Lawyer Rating Service Sued–Browne v. Avvo
By Eric Goldman
Browne v. Avvo, CV7 920 (W.D. Wash. complaint filed June 14, 2007)
You may have heard about Avvo, the latest entrant in the suddenly crowded field of lawyer ratings/rankings. Avvo pulls together a variety of third party data to spit out a numerical rating of lawyers.
Let’s start with the good news. It is very difficult for the average consumer to find and select a lawyer intelligently. There are many ways that consumers find lawyers, but two prominent methods:
1) word-of-mouth. A recommendation from a trusted source is better than nothing, but it doesn’t lend itself to easy comparison shopping, especially when consumers don’t get multiple referrals. Plus, these recommendations don’t any capture reputational information outside the referral source’s purview.
2) lawyer advertising, especially Yellow pages. In theory, the Yellow pages could encourage comparison shopping, but investigating multiple lawyers is time-consuming and possibly expensive, and the number of ads for certain services are so voluminous that it’s not even clear where to start. As a result, consumers often look at the internal credibility factors within the ad (ad size, testimonials, professional looking models, etc.) to navigate the mass of ads. Among other downsides, this selection process effectively bypasses any neutral sources of reputational information.
The process of helping consumers find lawyers would benefit greatly from reengineering. Consumers need more help making choices, and attorneys need more reputational accountability. To the extent that Avvo can improve consumer decision-making when selecting lawyers, Avvo is a very welcome entrant into the market.
Unfortunately, the first generation of Avvo isn’t very confidence-inspiring. The distillation of attorneys into a single numerical rating is inherently fraught with peril, and the media has picked up on numerous examples where the ratings are out of sync with common sense. There could be a number of reasons for this, including insufficient data to make accurate ratings or miscalibrated components of the rating algorithm. Either way, the numerical ratings look much more like a work-in-progress than a finished product, and I sure hope consumers aren’t actually relying on the numerical ratings. Regardless of any legal proceedings, I’m sure Avvo will be working hard to improve its tool by refining its algorithm, fixing any data errors and gathering more trustworthy data to rely on.
Naturally, where egos and money are on the line, lawyers aren’t going to stand by idly. Instead, a mere nine days after Avvo’s launch, we have the first class action lawsuit against Avvo. (I say first because it seems inevitable that more lawyers are going to want a piece of this action). The plaintiffs say (among other things) that “the Avvo Ratings are capricious and arbitrary, susceptible to manipulation by a number of sources that are neither objective nor mathematical nor indicative of a lawyer’s professional competence.” Based on the formula’s perceived deficiencies and Avvo’s wording choice in how it characterizes the factors and the data outputs, the plaintiffs allege violations of the Washington Consumer Protection Act.
Underneath the surface is a philosophical issue: what makes a good lawyer, and is there any way that lawyers can be objectively compared to each other?
There’s also a transparency issue. Do consumers need to know what factors went into the ratings in order to appreciate their import, or is simply telling consumers that a lawyer has an “excellent” rating enough even if the consumer doesn’t understand how the rating was produced? Obviously this raises parallel issues to search engine rankings, where consumers have no idea how search engines make their ordering decisions (both because the algorithms are secret and because consumers don’t understand search engine operations generally) but yet they trust those rankings almost implicitly. Transparency might help consumers understand Avvo’s ratings better, but I’m not sure transparency would actually improve consumer decision-making.
Let’s put all of this aside and focus on the doctrinal issue that seems preeminent: 47 USC 230. To the extent that the plaintiffs seek to hold Avvo liable under state consumer protection laws for third party content, this lawsuit should be cleanly preempted by 47 USC 230. As an example, I’m reasonably confident that eBay would argue vociferously that its numerical feedback rating is protected by 230 (among other doctrines).
But it’s hardly clear that Avvo gets the benefit of the statute. First, arguably, the lawsuit is based on the word choices that Avvo made in describing/characterizing the data and the output, not the underlying third party data. Second, this case goes straight to the doctrinal murkiness of the Roommates.com case. Recall Reinhardt’s reformulation that Roommates.com lost 230 protection because “Roommate categorizes, channels and limits the distribution of information, thereby creating another layer of information.” Isn’t this exactly what Avvo does too? I sure hope the Ninth Circuit cleans up the Roommates.com hairball before cases like this test its limits.
UPDATE: Tim Stanley thinks Avvo is purging judges’ ratings. If so, this might reinforce that perhaps Avvo went to public beta a little too early.
UPDATE 2: Yesterday, I spent an hour or so filling out my Avvo profile and boosted my score 1.7 points. Probably could have gone a little higher if I spent even more time. Looks like a pretty rigorous game-proof system when self-reporting moves me up 2 categories.
UPDATE 3: Avvo has tweaked its system to reduce the precision of its rankings when it has limited data. This is a wise move, but I think there are a lot more tweaks that will need to be made before the rating system is helpful.