Decay Rates of Committed Online Community Members–an Epinions Case Study
By Eric Goldman
Building on several previous blog posts, I am writing up a extended discourse explaining why Wikipedia will fail. A key part of my argument is that committed Wikipedians will turn over faster than they can be replaced. Unfortunately, I’m not aware of a study of Wikipedian turnover rates. However, I had a rough sense of turnover rates in the Epinions community, and I was able to consult some publicly available data to do a quick ‘n’ dirty empirical study. Here’s what I came up with (I did the data review at the end of December 2008):
I looked at Epinions’ top 20 most popular authors in 1999 to see if they were still active on the site, which I defined as writing at least one opinion in the past 12 months (i.e., in 2008). I manually reviewed each of the top 20 reviewers’ profile pages. According to my definition, only seven of the top 20 (35%) still actively contribute to Epinions, meaning that 65% of those early power users have turned over in 9 years.
Indeed, six of the 20 (30%) have stopped coming to Epinions at all, and thus their accounts are no longer active. An explanation of inactive Epinions accounts: Epinions generally expires accounts if the person has not visited the site within a certain number of months (I’m not sure what the period is nowadays). Among other consequences of inactivity, Epinions wipes out any earnings that haven’t been cashed out before expiration. Most Epinions reviewers don’t earn a ton of money, but power reviewers with very popular opinions generally should be earning enough that they would be willing to expend a minimal effort to keep the cash coming. As a result, inactive members have an incentive to keep visiting the site occasionally and cashing out earnings even if they are no longer interested in continuing to write. The fact that 30% of them gave up the cash entirely is interesting to me.
The 1999 class of power Epinions reviewers is special in part because of the unique circumstances of the dot com bubble and the very high payouts that Epinions offered to “prime the pump” of reviews. Thus, many reviewers were drawn to the site by the opportunity to earn lucrative amounts, and it may not be surprising that these users turned over (in some cases, in disgust) when payout rates were significantly reduced during the dot com bust.
For comparison, I also looked at the top 20 most popular authors in 2003, many of whom either joined after the dot com bust or had acquiesced to Epinions’ reduced payouts by this time. This has been a much more stable group: 15 (75%) are still active contributors, and none of the 20 have inactive accounts.
Putting the two stats together (and acknowledging the many weaknesses of this limited empirical review), I see a community decay rate among Epinions’ most committed members of:
* 25% after 5 years
* 65% after 9 years
I’m not sure if this decay rate will look anything like WIkipedia’s. Among other major differences, the glue that binds these reviewers to Epinions is different than the binding force for Wikipedia, because Epinions reviewers get both paid for their contributions (even if it’s not a huge amount) and get much more reputational recognition than Wikipedians. (I’ll explain more about cash and credit in the paper). Also, by definition all Wikipedians joined during or after the dot com bust, so they may have different and more manageable expectations than the early Epinionators.
If you are aware of any other studies of community membership decay rates, specific to Wikipedia or more general, I would be very grateful for the referral. Please email me. Also, if you would be interested in exploring this topic with me in a more scientifically rigorous way, let me know–I might be interested in doing a real study of this phenomenon at some point.
UPDATE: In a short amount of time, I’ve already gotten some email from people who want me to build out the entire argument about Wikipedia–basically, to see the full paper. I’m gratified by this because most of the time people recoil in horror at the thought of reading my academic papers. (Yes, this includes my mom, who is otherwise the most sympathetic audience a son could ask for). In any case, I’ll be presenting the paper at Uniiv. of Colorado Boulder in early February and then I hope to post it online in a month or two after that. Thanks for your patience.
UPDATE 2: Jason Lee Miller’s comments.