What Would Happen If Search Engines Had to Give Higher Visibility to Less Relevant Results?
I’m reporting on a study by Bin Han, Chirag Shah, and Daniel Saelid called “Users’ Perception of Search Engine Biases and Satisfaction.” The authors showed two pages of Bing results to study participants. (An aside: Bing…really?) The first page was Bing’s standard results for a query. The second page replaced some of the standard results with lower-ranking Bing results for that query. The second page was designed to mimic how search results might look if regulators required search engines to combat search engine “bias,” such as exposing searchers to more diverse viewpoints.
Any guesses how searchers responded to the second page with the lower-ranked results? From the conclusion:
participants prefer the real search pages over the synthesized ones with a significant higher ratio. It indicates that adding more varieties makes the results less biased but less relevant and consistent to the queries, which hurts users’ satisfactions
Over the years, a number of folks have favored overriding search engines’ relevancy-based determinations to advance other normative goals (for way old-school proposals, see, e.g., Pandey et al, Sunstein, Pasquale; the new-school proposals are too numerous to inventory). Putting aside the obvious constitutional problems with such proposals, this study reinforces the intuition that such proposals actively thwart the searchers’ objectives and degrade the efficacy of search results. As I wrote 15 years ago:
regulatory intervention that promotes some search results over others does not ensure that searchers will find the promoted search results useful. Instead, government regulation rarely can do better than market forces at delivering results that searchers find relevant, so searchers likely will find some of the promoted results irrelevant. The clutter of unhelpful result may hinder searchers’ ability to satisfy their search objectives, undermining searchers’ confidence in search engines’ mind-reading abilities. In this case, regulatory intervention could counterproductively degrade search engines’ value to searchers. Whatever the adverse consequences of search engine bias, the consequences of regulatory correction are probably worse.
The idea of regulators telling search engines how to make relevancy determinations, or worse yet to override their editorial decisions about relevancy, has always been stupid and censorial. Which is why it remains so popular in some policy circles even today.