AutoPersuade: A Framework for Evaluating and Explaining Persuasive Arguments
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 16325-16342.

Abstract
We introduce AutoPersuade, a three-part framework for constructing persuasive messages. First, we curate a large dataset of arguments with human evaluations. Next, we develop a novel topic model to identify argument features that influence persuasiveness. Finally, we use this model to predict the effectiveness of new arguments and assess the causal impact of different components to provide explanations. We validate AutoPersuade through an experimental study on arguments for veganism, demonstrating its effectiveness with human studies and out-of-sample predictions.
Related Publications
- [Conference Proceedings] STATe-of-Thoughts: Structured Action Templates for Tree-of-Thoughts
- [Paper] State Media Control Influences Large Language Models
- [Paper] Fine-Tuned Large Language Models Can Replicate Expert Coding Better Than Trained Coders: A Study on Informative Signals Sent by Interest Groups
- [Conference Proceedings] GPT Deciphering Fedspeak: Quantifying Dissent Among Hawks and Doves
- [Working Paper] Where's the Evidence that Respondents Understand Your Survey Questions?
- [Paper] Using Large Language Model Annotations for the Social Sciences: A General Framework of Using Predicted Variables in Downstream Analyses
- [Book] Text as Data: A New Framework for Machine Learning and the Social Sciences
- [Conference Proceedings] TopicCheck: Interactive Alignment for Assessing Topic Model Stability