Longer description of your proposed project

A typical meta-analysis is painfully credulous to its literature. Reading one will almost always give you a rosier picture about the validity and durability of a field’s findings than you’d get from reading a random subset of the analyzed papers. Simonsohn et al. call for meta-analyses to become more “selective, design-focused and evaluative," rather than being a simple averaging exercise.

As a partial remedy to this, I will write, in collaboration with coauthors, three papers that aim to fix this problem.

The first paper will document the meta-analytic aims and methods of Betsy Levy Paluck’s lab. The goal of this paper is to reduce friction to writing meta-analyses for other labs and explain how to write meta-analyses that aim for curious, dispassionate data analysis rather than validation. We'll also compose an R package that implements these defaults, available in a first draft here.

Second, in collaboration with Benny Smith and Maya Mathur, I will apply the methods and thinking of that paper to the question of vegan advocacy, looking specifically to see what theories of change drive the biggest effects. Here’s a first draft of that paper.

We're now revising the paper to be more about measurement bias and less about what works. In my revised estimate, we really only know what works at a handful of elite colleges, and that's not a very robust evidence base.

Third, my advisor Donald P. Green and I will write a meta-analytic piece on design deficiencies in sports science. There have been quite a few papers noting statistical problems in the field (here, here and here, for example), but less attention has been paid to four deeper issues: representative populations, realistic interventions, strict adherence to randomized comparisons, and real-world outcomes. We aim to remedy this by systematically reviewing and meta-analyzing articles published in three field journals. Here’s an outline. (This paper is more in flux than the others. We might just end up writing it as a "how to" guide to field experiments and identifications strategies, though I'd like to include a meta-analytic component as well.)

Describe why you think you're qualified to work on this

I’ve co-authored three meta-analyses — one published, one in preprint stage, and one available as a first draft — and worked as an RA on a fourth. I've been thinking about these questions for a long time.

Other ways I can learn about you

Check out my portfolio! I am also somewhat active on the EA forum, and here is my Google Scholar.

How much money do you need?

I currently have funding for the first two pieces but not third (the exercise piece). Let's say that it would take 2 months of work to get that piece to the finish line -- I'd call that ~$10,000.

Estimate your probability of succeeding if you get the amount of money you asked for

I will probably write some version of these three papers no matter what. The vegan one in particular is like 40% done. Where a grant will help is getting things done more quickly. if I can work on this full-time (i.e. delay looking for a full-time job), I can get all three done in 2024, whereas if it's a hobby, it will probably take years. Also, there are more and less ambitious versions of these papers. Without funding, we'll probably scale these papers back to just make the core points.