THE UNIVERSITY OF CHICAGO
Response Adaptive Experiments
Project Details
The rise of social media as a primary source of news and political discourse presents both opportunities and challenges for political scientists. With the support of a Carnegie Fellowship, Molly Offer-Westort is studying the societal transformations brought about by the internet, particularly regarding political polarization and engagement. She has developed open source tools that promote digital field experiments, encouraging social science researchers to extend their online studies outside of online surveys to “real-world” online environments. These tools support adaptive algorithms that update treatment assignments based on observed responses—leveraging methods from multi-armed bandits and policy learning. This approach enables researchers to identify which interventions are most effective, and for whom, improving statistical efficiency and reducing ethical costs. By recruiting study participants directly from social media platforms and facilitating conversational studies over WhatsApp and Facebook Messenger, she runs experiments closer to respondents’ existing media environments, illuminating how online political conversations influence their attitudes and behaviors in these settings.
In 2022, in collaboration with Nick Feamster in the UChicago Department of Computer Science Department, Offer-Westort was awarded an intramural seed grant through the Data Science Institute/Center for Effective Government Data & Democracy Initiative. While her previous online messaging studies had used third party platforms integrated with Facebook Messenger, those platforms do not have support for complex randomization or for integration of language models that facilitate more naturalistic conversational experiences in the messaging studies.
Under the initial grant, Offer-Westort and Feamster developed a platform to run surveys in Spring 2023 on Facebook Messenger to collect over 5,000 survey responses in a study on deep canvassing. Using natural language processing to analyze user messages, the chatbot was designed to detect conversational topics and share relevant pre-scripted messages and third-person experiences. Findings showed that online conversations encouraging perspective-taking could move attitudes toward affected groups and relevant policies compared to a control condition, even when participants knew their conversational partner was an automated chatbot. This initial research provided a promising model for digital political outreach and policy campaigns, suggesting that personalized digital interventions can effectively influence public opinion on sensitive issues.
The Data & Democracy Initiative grant was renewed in 2023, when, along with Isaac Mehlhalff, also in the Department of Political Science at UChicago, Offer-Westort and Feamster have expanded the features of the chatbot platform, including incorporation of generative language models in messaging conversations. A current study aims to understand how citizens discuss challenging topics and coordinate across political divides in online spaces.
A key methodological innovation across these projects is the use of simulation-based procedures to design experiments and generate valid statistical inference when hypotheses are data-driven—an area where traditional tools often fail. Offer-Westort has developed novel algorithms that minimize variance in causal estimates while using inverse probability weighting estimators tailored for data collected via adaptive protocols. These tools ensure that results from complex, real-world experiments can be interpreted with the same rigor as conventional randomized trials.
Project Lead
Molly Offer-Westort integrates machine learning methods with experimental design to answer causal questions. She also has an ongoing substantive research program that examines online behavior to understand how people change their views and attitudes in response to the conversations they take part in and the information they engage with online. She combines these agendas in social media experiments, using approaches like adaptive assignment and policy learning, and incorporating natural language processing methods for flexible conversational interventions.
She has conducted social media experiments to identify the most effective interventions for curbing the spread of misinformation online, to optimally target informational messaging to people hesitant to adopt vaccines, and to measure the efficacy of online deep canvassing. Her work in statistical methodology develops and advances tools for experimental design and analysis, with a particular focus on adaptive experimentation.
Offer-Westort's PhD is joint in Political Science and Statistics & Data Science, conferred by Yale University in 2019; Offer-Westort also holds a Masters in Statistics, also from Yale, and a Masters in Public Affairs, from the Princeton School of Public and International Affairs.
Project Collaborators
Nick Feamster designs and deploys network protocols and systems that make the Internet work better. Using empirical network measurement and machine learning to understand and improve network performance, security, and privacy, his research often has implications to policy. He regularly works with federal and municipal organizations, including the Federal Communications Commission (FCC) and the City of Chicago on equitable Internet access, security, and privacy.
Isaac Mehlhaff’s research is driven by substantive questions in public opinion and political psychology: How and why do citizens change their attitudes on political issues? Under what conditions can political discussion exacerbate or ameliorate mass polarization? How is polarization causally related to other features of government and society? He approaches his work primarily as a computational social scientist, using and developing methods in natural language processing, machine learning, and Bayesian modeling. He received his PhD in 2023 from The University of North Carolina at Chapel Hill. He also holds an MA from UNC-Chapel Hill and a BA from the University of Wisconsin-Madison. At UChicago he is affiliated with the Data Science Institute, Committee on Data Science, and Program in Political Economy.