A pretty standard take on "Why is everything so bad online?" says, "It's the algorithms, stupid." They're in effect a demonic third party, an external agent that gets between us and corrupts otherwise reasonable minds into toxic trolls.
I'm skeptical. So it was keen to read Henry Farrell and Cosma Shalizi's paper, "Bias, Skew, and Search Engines Are Sufficient to Explain Online Toxicity." Their argument is that engagement-maximizing machine learning isn't actually necessary to get us to a highly polarized online environment. Rather, all you need are two things:
- 1.
Regular old confirmation bias
- 2.
Preferential attachment
Those are, let's say, omnipresent among us humans, and when you mix them, you get a pull not towards truth, but towards rationalization. Taking simple search engines, which pre-exist the social media that typically gets all the blame, if you have those things, then, first, search queries are likely to be formulated to defended or prop up already-existing identity-protective beliefs. You'll search for stuff that proves you right. And because there's so much out there, and so many other people who share your already-existing identity and its beliefs, you'll find a lot. So search returns confirmation because you want confirmation, and plenty of other people have supplied it. (And linked to it, driving such confirmation up the rankings.)
They point to Wikipedia as one of the few outliers, but argue that its architecture constrains those two kinds of bias, and it forces antagonistic actors to interact under very strict evidentiary norms and consensus requirements. Which is rad, but that structure is unique to what Wikipedia's trying to do. You can't really force it on social media.
The takeaway is algorithmic tweaking and content moderation let you tinker on margins, but aren't going to be a silver bullet. If toxicity is just what you get when ordinary human cognition bangs up against unconstrained sorting tools, then we're in trouble. We lack architectures to enforce epistemic friction, and we lack the discipline required to resist our own hunger for tribal confirmation.
Friday Field Notes is my collected marginalia and notes on what I'm reading as I think about upcoming essays and podcast episodes. Unlock them by becoming a paid supporter.
Behind the paywall:
Anil Dash says venture capital's shift from functional utility to speculative extraction is what's behind "enshittification."
Schopenhauer, de Beauvoir, and why getting board can be so damaging to societies.
Become a member to continue reading
Memberships start at $3/month