Public Discourse in the Web Does Not Exhibit Group Polarization

Public Discourse in the W eb Do es Not Exhibit Group P olarization F ang W u and Bernardo A. Hub erman HP La bor atories P al o Alto, CA 943 04 No vem b er 2, 201 8 Abstract W e p erformed a massive study of the dy n a m i cs of group d e lib era- tion among sev eral websites con taining millions of opinions on topics ranging from b ooks to media. Con trary to the common phenomenon of group p olariza tion observed offline, we measured a strong tendency to w ard s mo derate views in th e course of time. This p henomenon p ossi- bly op e r a tes through a self-select ion bias whereby previous commen ts and ratings elicit contrarian views that soften the pr e vious opin io n s. 1 No aspect of the massiv e participation in con ten t creation that the w eb enables is more eviden t than in t he coun tless num b er of opinions, news and pro duct reviews that are constantly posted on the In ternet. Since these opinions pla y s uch an imp ortant role in trust b uilding and t he creation of consensus ab out many iss ues and pro ducts, there ha v e been a num ber of recen t of studies fo cused on the design, ev aluation and utilization of online opinion systems [5, 6 , 1 0, 11] (for a surv ey , see [7]). Giv en the imp o r t ance of group opinions to collectiv e so cial pro cesses such as group p olarization and information cascades [2, 3, 4 , 15] it is surprising that with the exception of one study [13], little researc h has b een done on the dynamic asp ects of online opinion formation. It remains unclear, f o r example, whether the opinions ab out b o oks, movies o r so cietal views fluctuate a lo ng time b efore r eac hing a final consensus , or they undergo any systematic c hanges as time go es o n. Th us the need to understand ho w online opinions are created and ev olve in time in order to dra w accurate conclusions from that data. Within this con text w e studied the dynamics o f o nline opinion expression b y analyzing the temp oral ev olution of a very larg e set of user views, rang ing from millions of online reviews of the b est selling b o oks at Amazon.c om , to thousands of mo vie reviews at the In ternet Mov ie D atabase IMDB.com . Surprisingly , our analysis rev ealed a trend that runs coun ter to the w ell kno wn herding effect studied under infor ma t ion cascades, and in t he smaller instance of group p o la rization. Online, a self selection mec hanism is at pla y whereb y previous commen ts and rat ing s elicit con traria n views that soften the previous opinions. It is well known that in the case of gro up p olarization, mem b ers of a discussion group tend to adv o cate more extreme p ositions and call for riskier courses of action than individuals who did not participate in an y such discus- sion [1, 17 ]. How ev er, o n the massiv e scale that the w eb offers, w e observ ed that later opinions in the course of time tend to sho w a la rge difference with previous ones, th us softening the ov erall discourse. This is a robust a nd quan titative observ ation fo r whic h w e can only offer a ten t a tiv e explanation in terms of the cost of expressing an opinion to the group at large. In order to p erform t his study w e first analyzed b o ok ratings p osted on Amazon.com . Our sample consisted of t he b o ok ratings of the top 4,000 2 b est-selling titles of Amazon in eac h of the follow ing 12 catego ries, as of July 1, 2007: art s & photo graph y , biographies & memoirs, history , lit era t ur e & fiction, m ystery & thrillers, r eference, religion & spiritualit y , sp orts, trav el, nonfiction, science, a nd entertainmen t. F or eac h of the 48,000 b o oks, a series of user ratings was collected in time order, where eac h rating is an in teger b et we en 1 and 5. Among the 48 ,0 00 b o oks, 16,4 5 4 b o oks hav e no less than 20 ratings, and 11,920 ha ve an av erage rating ab o ve 4. W e first che ck ed the av erage rating of the 16,454 b o oks a s a function of the index of the rating ( n = 1 , . . . , 20). As can b e seen from Fig. 1(a), E X n decreases almost linearly with n , so there is a clear dynamical trend in the ratings, whic h corrob o rates the observ ation rep orted in [13 ]. Later users tend to write differen t reviews from those of earlier users. Like in the exp erimental setup of group p olarization, an Amazon user observ es the existing av erage rating of that b o ok b efore she lea v es her own (usually shown at the top of the b o ok page, rig h t under the title). Ho w ev er, a s o pp osed to group p olarization, the o verall opinion on Amazon tends to decrease a wa y from the extreme ones. One p oin t to b e stressed is tha t these results do not necessarily imply that as time go es on the av erage opinion of the whole p opulation changes , for the late review ers might come fr o m a differen t group than the earlier ones and need not b e represen tativ e of the whole p opulation. This is seen when plotting the av erage “helpful ratio” a s a function o f star ra ting in Fig. 2 for users of Amazon . As can b e seen, the whole p opulation finds high ratings in general more helpful than lo w rat ing s, implying that the ma jorit y of the p opulation do es not necessarily agree with the lo w ra t ings. This additional data suggests that rather than indicating a real opinion shift in the whole p opulation, the observ ed dynamic trend is more of an expression bias. On reflection, it is rather surprising that p eople con tribute opinions and reviews of topics whic h hav e a lready b een extensiv ely cov ered b y others. While p osting views is easy to understand when it in volv es no effort, lik e clic king on a button of a w ebsite, it is more puzzling in situations where it is costly , suc h as comp osing a review. 1 If the opp ortunity to affect the ov erall 1 When a user of Amaz on decides to revie w a b o o k, she is r equired to write a shor t paragr aph of r e view in addition to a simple s tar ra ting. The average word co un t of Amazo n 3 5 10 15 20 4.3 4.4 4.5 4.6 n EX n extreme less extreme Figure 1: (a) The av erage rating of 16,45 4 b o oks on Amazon.com with more than 20 reviews. E X n is the sample av erage rating of all the 16,454 n ’th ratings. As o ne can see from the figure, E X n decreases b y 0.4 stars in 20 steps. W e did not obtain enough data fro m lo w selling b o oks to sho w the opp osite trend. opinion or rating diminishes with the n um b er of published ones, wh y do es an yone b other to incur the cost of contributing y et another review? F ro m a rational c hoice theory p oint of view, if the utilit y to b e gained do es not out we ig h the cost, p eople w o uld refra in fr om expressing their views. And ye t they do. This is reminiscen t of the w ell analyzed v oter’s paradox [9, 14, 16], where a rat io nal calculation of their success probability at determining the outcome of an election w ould make p eople sta y home rather t han v ote, and y et they sho w up at the p olls with high t ur no ut rat es. In con trast to a p olitical election, there is no concept of winning in online opinion systems. Rather, by contributing her own opinion to an existing opinion p o o l, a person affects the a ve ra ge or the distribution of opinions b y a marginal amount tha t diminishes with the size of that p o ol. One p ossible explanation for t hese results is that in cases lik e Amazon , p eople will deriv e more utilit y the more they can influence the ov erall rating, as in the voter’s para do x. T o b e precise , in cases where users’ opinions can reviews is 181.5 words [12], so the c o st o f o pinion expressio n is indeed high. 4 1 2 3 4 5 0.50 0.55 0.60 0.65 0.70 0.75 rating helpful ratio (a) 1 2 3 4 5 750 800 850 900 950 1000 1050 rating review length (characters) (b) Figure 2: (a) The av erage helpful ratio of fiv e differen t star ra tings. (b) The a ve ra ge review length of fiv e differen t star ratings in the nu mber of c hara cters. The data is calculated for 4,000 b estselling mys t ery b o oks. By comparing the tw o figures it is clear that p eople find high r a tings more helpful not just b ecause they are long er. F or instance, 5-star reviews are on av erage shorter than 4 -star and 3-star reviews but are nev ertheless more helpful. 5 b e quantified and ag gregated into an av erage v alue, the influence of a n online opinion can b e measured by ho w muc h its expression will c hange the av erage opinion. Supp ose that n users hav e expressed their opinions, X 1 , . . . , X n , o n a giv en topic at a webs ite, with X i denoting the quan tified v alue of the i ’th opinion. If the ( n + 1)’th p erson expresses a new o pinion X n +1 , it will mov e the a ve ra ge rating to ¯ X n +1 = n ¯ X n + X n +1 n + 1 , (1) and the a bsolute c hange in the a ve r a ge rating is give n by | ¯ X n +1 − ¯ X n | = | X n +1 − ¯ X n | n + 1 . (2) Th us a p erson is more lik ely to express her opinion whenev er | X n +1 − ¯ X n | is la r ge — a n opinion is lik ely to b e expressed if it deviates by a significan t amoun t from those already stated. Indeed, what is the p oin t of lea ving another 5-star review after one h undred p eople hav e already done so? 2 In order to test this hy p othesis, w e measured directly how muc h o ne’s rating deviates from t he observ ed a v erage rat ing . W e plot the exp ected deviation E d n = E | X n − ¯ X n − 1 | as a function of n in F ig. 3, where X n is the rating left by the n ’th user, and ¯ X n − 1 is the av erage rating the n ’th user observ es. As can b e seen, E d n increases with n . Since the exp ected deviation E d n of an i.i.d. sequence normally de cr e ases with n , this increasing trend is indeed significan t . This again supp o rts our conjecture that those users who disagree from the public opinion will b e more willing to express themselv es and thus soften the o v era ll opinion o f a give n b o ok. Next w e examined whether this dynamical trend is still prominent at the lev el o f eac h individual b o ok. W e defined d = ¯ X 20 − ¯ X 10 as a measure of the c hange in a b o ok’s rating ov er time. The histogram of 16,454 d ’s is sho wn in Fig. 4 . As can b e seen, most of the changes are negative. A t - test of the alternativ e hy p othesis “ d < 0” yields a p -v alue less than 0 . 0 001, whic h further confirms the declining trend. 2 This point has also b e e n ma de within the “brag-a nd-moan” mo del [8, 11] which as - sumes that consumers only choo se to write reviews when they ar e very satisfied with the pro ducts they purc has ed (brag), or very disgr un tled (moan). Note ho wever, that the brag- 6 5 10 15 20 0.60 0.65 0.70 0.75 0.80 n Ed n Figure 3: The av erage deviation of Amazon rating s increases with the n um b er of p eople. Histogram of d d Frequency −1.5 −1.0 −0.5 0.0 0.5 1.0 0 500 1000 1500 Figure 4: Histogram of the change in a ve ra ge b o ok rating s ( d = ¯ X 20 − ¯ X 10 ) on Amazon.com . Most of the c hanges are negative , testifying a declining trend in the a verage ratings. 7 While our hypothesis o f a costly expression bias seems to explain the softening of opinions observ ed in Amazon , it w o uld b e more conclusiv e if one could conduct a test that directly compares p eople’s opinions expressed at differen t cost lev els. In order to address this issue w e conducted a study of IMDB.com (The In ternet Mo vie D atabase). Unlike users of Amazon who are required to write a review when rating a b o ok, users of IMDB are free to cho ose the effort lev el when reviewing a movie. Sp ecifically , after observing the curren t a verage r a ting o f a mo vie, a user can either submit a quic k ra ting b y clic king on a scale of 10 stars, or can mak e the extra effort inv olv ed in writing a commen t b et w een 10 and 1000 w o rds. Our study fo cused o n t w o sets of mo vie titles. The first consists of the 50 most top-rated mo vies released after y ear 2000, whic h w e call the “go o d mo vies”, and the second consists of the 50 most low-rated, whic h w e call the “bad movies ”. F or each mo vie w e know its a verage rating (take n among a ll ratings with or without a commen t), as well a s the v alue and date-stamp of its eac h commen ted rating, but w e do no t ha ve any sp ecific information ab out eac h uncommen ted rat ing. The trend of the ratings asso ciated with commen ts of the tw o sets of mo vies is sho wn in Fig. 5. Similar to Amazon , a softening of the expressed view is o nce again observ ed for b oth sets. Tw o histograms of d = ¯ X 10 − ¯ X 5 for the g o o d mov ies and the bad movies are show n in Fig. 6. A t -test of the alternativ e hypothesis d < 0 for the g o o d mo vies yields a p -v alue 0 .44. A t -test o f d > 0 f or the bad mo vies yields a p -v alue 0.018. While it is not to o reliable to conclude that go o d mo vies tend to receiv e low er ratings ov er time, it is safer to conclude that bad movies accum ulate higher ratings as time go es on. W e also examined the difference b etw een the ov erall av erage rating (with or without a commen t) and the a v erage rating asso ciated with a commen t for eac h mo vie, and the result is sho wn in F ig . 7. It can b e seen that those who decide to sp end t he time to write a commen t tend to sp eak differen tly fro m the ma jority users, who simply leav e a star rating without an y justification. Fig. 7 is thus a direct v erification of our hypothesis that high cost induces and-moan mo del is static and th us predicts that ¯ X n is constant over time, in contradiction with the observed dynamical trends. 8 20 40 60 80 100 8.45 8.50 8.55 8.60 8.65 8.70 n EX n (a) Go o d mov ies 10 20 30 40 50 2.4 2.6 2.8 3.0 3.2 3.4 n EX n (b) Ba d mo vies Figure 5: Av erage rating asso ciated with a commen t of the (a) go o d and (b) bad movies , as a function of the n umber of existing ratings. It can b e seen that g o o d movies tend to receiv e low er ratings as time go es on, and bad mo vies tend to receiv e higher ratings. Good movies d Frequency −2 −1 0 1 2 3 0 2 4 6 8 10 (a) Bad movies d Frequency −3 −2 −1 0 1 2 3 0 5 10 15 20 (b) Figure 6: Histogra m of d = ¯ X 10 − ¯ X 5 for the go o d mo vies and bad mov ies. 9 2 4 6 8 10 2 4 6 8 10 average rating average rating with user comments + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Figure 7: Expression bias of commen t ed ratings. Eac h p oint in this figur e corresp onds to one movie title. The horizontal co ordinate represen ts the mo vie’s o ve ra ll av erage rating ( ¯ r ) take n ov er b oth commen ted and uncom- men ted ratings. The v ertical coo rdinate represen ts the mo vie’s a v erage rating tak en ov er only commente d rat ing s ( ¯ r c ). Go o d and bad movies ar e repre- sen ted by circles a nd crosses, resp ectiv ely . Clearly , those users who sp end the additional cost to write a commen t tend to sp eak opp o sitely to the ma- jorit y . A t -test of the alternativ e h yp othesis that ¯ r c < ¯ r for go o d movie s and a similar t -test of ¯ r c > ¯ r for ba d mo vies b o t h yield a p -v alue less than 0 . 001. expression bias. These results show that in the pro cess of a rticulating and expressing their views online, p eople tend to follow a different pattern fro m that observ ed in information cascades or group p olarization. What is observ ed is an a n ti p olar- ization effect, whereb y previous commen ts a nd rat ing s elicit con tra r ian views that soften the previous opinions. This is in con trast to the phenomenon of herding a nd opinion p ola rization observ ed in b oth group dynamic s and online sites. 3 3 W e p oint out that in a w ebs ite like J yte.c om , where it takes o nly one clic k to a gree or 10 In closing, b esides their intrinsic nov elt y , these results throw a cautionary note on the interpretation of online public opinion. This is b ecause a simple c hange in the order or fr equency of giv en sets of views can change the ongoing expression in the comm unity , and thus the p erceiv ed collectiv e wisdom that new users will find when accessing that information. References [1] Solo mon E. Asc h. Opinions and so cial pressure. Scientific Americ an , 193(5):31– 35, Nov em b er 1 9 55. [2] Abhijit V. Banerjee. A simple mo del of herd b eha vior,. [3] Sushil Bikhc ha nda ni, D a vid Hirshleifer, and Iv o W elch. A theory of fads, fashion, custom, and cultural c hange a s informational cascades. Journal of Politic al Ec onomy , 100(5):9 92–1026 , 1 9 92. [4] Judith A. Chev alier and Dina Mayz lin. The effect of w ord of mouth online: Online b o ok reviews . Journal of Marketing R ese ar ch , 43(3):345– 354, 2006. [5] D an Cosley , Sh yong K. Lam, Istv an Alb ert, Joseph A. Konstan, and John Riedl. Is seeing b elieving? Ho w recommender in t erfa ces affect users’ opinions. In CHI 200 3 , Ft. Lauderdale, Florida , USA, April 2 003. [6] Chrysan tho s Dellaro cas. Imm unizing online reputatio n rep orting sys- tems aga inst unfair ratings and discriminatory b eha vior. In A CM EC’00 , Minneap olis, Minnesota, USA, O cto b er 2000. [7] Chrysan tho s Dellaro cas. The digitization of word of mouth: Promise and challenges of online feedbac k mec ha nisms. Management Scien c e , 49(10):140 7–1424, Octob er 2003. disagree with a n arbitra ry claim, we did see a strong gr oup p ola rization [1 5]. It is poss ible that the latter is due to the fact that such a vote is costles s co mpared to the opinions on Amazon and IM DB . 11 [8] Chrysan tho s D ellaro cas and Rit u Nara yan. What motiv ates consumers to review a pro duct online? A study of t he pro duct-sp ecific anteced ents of online mo vie reviews. In WISE 2006 , Ev anston, Illinois, USA, D e- cem b er 2006. [9] Anthon y D o wns. An Ec onomic The ory of Demo cr acy . Harp er & Ro w, New Y ork, 1957. [10] G uo dong Gao, Bin Gu, a nd Mingfeng Lin. The dynamics of online consumer reviews . In WISE 2006 , Ev anston, Illinois, USA, Decem b er 2006. [11] Na n Hu, P aul A. P av lo u, a nd Jennifer Z hang. Can online reviews rev eal a pro duct’s t rue qualit y? Empirical findings and analytical mo deling of online word-of-mouth comm unication. In ACM EC’06 , Ann Arb or, Mic higan, USA, June 2006. [12] Erik Ketzan. Amazon.com and the new demo cracy o f opinion. http://www. themodernword.com/pynchon/papers_ketzan1.html , No v ember 200 2. [13] Xinxin L i and Lorin M. Hitt. Self selection and informa t ion role of online pro duct reviews. In WISE 2004 , Brisbane, Australia, Nov em b er 2004. [14] Willia m H. Rike r and P eter C. Or desho ok. A theory of the calculus of v oting. Americ an Politic al Scien c e R eview , 62:25–42, 1968. [15] Mat t hew J. Salganik, Pete r Sheridan D o dds, and Duncan J. W atts. Exp erimental study o f inequalit y and unpredictabilit y in an artificial cultural mark et. Sc i e nc e , 311(57 6 2):854–85 6, F ebruary 2006. [16] Alexander A. Sc h uessler. Expressiv e v oting. R ationality and S o ciety , 12(1):87–1 19, 200 0. [17] Cass R. Sunstein. Delib erativ e trouble? Wh y g roups g o to extremes. Y ale L aw Journal , 110(1) , Octob er 200 0. 12 Histogram of d d Frequency −5 0 5 0 500 1000 1500 2000