RCTs Against the Machine: Can Machine Learning Prediction Methods Recover Experimental Treatment Effects?
RFF researchers examine how well machine learning counterfactual prediction tools can estimate causal treatment effects.
We investigate how well machine learning counterfactual prediction tools can estimate causal treatment effects. We use three prediction algorithms—XGBoost, random forests, and LASSO—to estimate treatment effects using observational data. We compare those results to causal effects from a randomized experiment for electricity customers who faced critical-peak pricing and information treatments. Our results show that each algorithm replicates the true treatment effects, even when using data from treated households only. Additionally, when using both treatment households and nonexperimental comparison households, simpler difference-in-differences methods replicate the experimental benchmark, suggesting little benefit from ML approaches over standard program evaluation methods.
Click "Download" above to read the full paper.
Brian C. Prest
Fellow; Director, Social Cost of Carbon Initiative
Brian Prest is an economist and Fellow at Resources for the Future specializing in climate change, oil and gas, and energy economics.
Casey J. Wichman
Casey Wichman is a university fellow at RFF. He performs research at the intersection of environmental and public economics, with an emphasis on examining the ways in which individuals make decisions in response to environmental policies.
RFF Live — Sep 13, 2022
Pathways toward Grid Decarbonization
New modeling illuminates opportunities to reduce greenhouse gas emissions from the US electric power sector
Media Highlight — May 12, 2022
S&P Global: "Capacity Markets, though Flawed, Seen as Critical in US Energy Transition"
This story details a May 11 RFF Live event on capacity markets, as well as a new book on the same topic coauthored by RFF University Fellow Todd Aagaard.