This is a large field test (RCT) of the effects of LLMs on human conflict
I've been researching the question of how machines mediate human conflict (from cooperation at one end to polarization, escalation, and violence at the other) for 15 years. There are some well-developed hypotheses in this field and some evidence from lab studies but we need field tests, and the easiest way to do that is through social media.
To kick off such tests, I raised money and ran an international competition which created eight AI algorithms.Now we are testing as many as we can prior to the 2024 election.
We will test these algorithms using a custom browser extension (already written) that changes what ~15,000 consenting participants see on Facebook, X, and Reddit for four months prior to the election. Primary outcomes include measures of conflict and polarization, well-being, and news knowledge.
We have raised over $600k so far and can test 3/8 algorithms. Each additional $60k lets us test one additional algorithm . The divisive US 2024 election is a unique opportunity to test theories about AI effects on political conflict.
See our fundraising info page. Subscribe to our newsletter for updates.
Today’s recommender algorithms optimize for clicks. We know this tends to amplify the most outrageous content, polarizing and radicalizing us. We think the capacity of modern AI systems can allow us to do better.
Teams from all over the world competed by creating prototypes, then our interdisciplinary panel of scientists selected eight algorithms to test. They use LLMs to do things like detect credible news, downrank toxicity, uprank nuance, and to select content that crosses political divides — an important idea called bridging-based ranking.
In the short term, this experiment will provide unique comparative data about the political effects of different LLM-based content selection algorithms across three social media platforms. (It will produce much more valuable data than many unconnected experiments, because the results for each algorithm will be strongly comparable). In the longer term, this work will contribute rare empirical knowledge to the question of how AI affects intergroup conflict -- a potential catastrophic risk from AGI.
The cost of testing an algorithm is driven by participant recruitment (~$20 per person) and payment for filling out surveys (~$15 per person) for ~1,500 people per arm. With server costs this comes out to about $60,000 per arm.
We have already raised and committed almost $600k to fixed costs. Each additional dollar will go directly to increasing the science return by funding additional algorithms in the same protocol.
Jonathan Stray, Senior Scientist at Center for Human-compatible AI at UC Berkeley, is the principal investigator. He has been producing industry-relevant research on the effects of technology on public discourse for many years, and is active in the international peacebuilding community.
Ian Baker leads the engineering team (about 10 people). He was formerly a senior ML engineer at Dropbox.
Julia Kamin is the lead researcher at the Civic Health Project, a non-profit which develops and funds technology-based solutions to political polarization.
Kylan Rutherford and George Beknazar-Yuzbashev are PhD students at Columbia. They lead participant recruitment, having previously completed a similar browser-based study.
Mateusz Stalinski (U. Warwick) is a professor of economics. Ceren Budak (U Michigan) is a professor of computational social science.
The project advisors (and contest judges) include many of the most prominent scientists studying conflict, AI alignment, and political communication: Mark Brandt (Michigan State), Amy Bruckman (Georgia Tech), Andy Guess (Princeton), Dylan Hadfield-Mennell (MIT), Michael Inzlicht (U Toronto), Alex Landry (Stanford), Yph Lelkes (U Penn), Paul Resnick (U Michigan), Lisa Schirch (U Notre Dame), Joshua Tucker (NYU), Robb Willer (Stanford), Magdalena Wojcieszak (UC Davis).
Recruitment cost is higher than in our pilot study. Software is not completed on time. Platforms decide they don't like us and take legal or technical counter-measures. Participant attrition is significantly higher than in pilot study.
DALHAP foundation, Project Liberty, and Google Jigsaw have together contributed about $600k.