Experimental Methods: Design & Analysis
- Readings (password required)
- Discussion questions Google Doc
- Assignment submissions folder
- Lecture slides
- Topics covered include: causality; internal and external validity; ethics and deception; lab experiments; survey experiments; sampling; social desirability; power analysis; audit studies; field and natural experiments; writing about experimental data
- Example code
Causality is of fundamental interest to most social scientists. Theories and theoretically motivated questions are almost always causal in nature. While many statistical advances have aided our ability to test for causality in observational data, experiments remain the gold standard for causal inference. Primarily because of experiments' unrivaled ability to test for causality, use of experimental studies has grown rapidly in many social science fields such as economics, political science, and sociology.
Some experimental traditions, such as laboratory experiments, have a long and influential history within many fields. Newer methods such as field, audit, and survey experiments offer exciting possibilities to test causal ideas in newer and more diverse settings and with more diverse samples. While the class will begin by covering fundamental experimental methodology and laboratory experiments, the majority of the class will focus on experiments conducted outside of the laboratory: from conducting audit studies on hiring managers to fielding survey experiments in nationally representative samples (and everything in between).
We will spend roughly 75% of the course time focusing on experimental methodology and design; i.e. what types of experiments are possible and how to design and conduct methodologically sound studies. The course is meant to be practical, teaching concrete best practices rather than only abstract concepts. In addition, we will read exemplar research articles throughout the semester alongside methodological pieces, illustrating the diverse and contemporary topics that are currently being studied with experimental methods. We will also cover issues of grant funding as many experimental questions require paying participants.
The other 25% of course time will focus on issues of data analysis and the presentation of experimental results. Some aspects of experimental design make experimental data easier to analyze than survey data; other aspects make it more difficult. Best practices for analyzing and presenting experimental data are rarely covered in sociology, political science, or other related fields' graduate training and the tools needed can be slightly different than those commonly used with survey data. A primary goal of the course is for you to not only learn how to conduct experimental studies, but also how to best understand and present them.