RCTs against The Machine: Can Machine Learning Prediction Methods Recover Experimental Treatment Effects?
RFF researchers investigate how successfully ML prediction algorithms can be used to estimate causal treatment effects in electricity demand applications with nonexperimental data.
We investigate how successfully ML prediction algorithms can be used to estimate causal treatment effects in electricity demand applications with nonexperimental data. We use three prediction algorithms—XGBoost, random forests, and LASSO—to generate counterfactuals using observational data. Using those counterfactuals, we estimate nonexperimental treatment effects and compare them to experimental treatment effects from a randomized experiment for electricity customers who faced critical-peak pricing and information treatments. Our results show that nonexperimental treatment effects based on each algorithm replicate the true treatment effects even when only using data from treated households. Additionally, when using both treatment households and nonexperimental comparison households, standard two-way fixed effects regressions replicate the experimental benchmark, suggesting little benefit from ML approaches over standard program evaluation methods in that setting.
Brian C. Prest
Fellow; Director, Social Cost of Carbon Initiative
Brian Prest is an economist and fellow at Resources for the Future specializing in climate change, oil and gas, and energy economics.
Casey J. Wichman
Casey Wichman is a university fellow at RFF. He performs research at the intersection of environmental and public economics, with an emphasis on examining the ways in which individuals make decisions in response to environmental policies.
Explainer — Sep 22, 2023
Transmission 102: Building New Transmission Lines
This explainer reviews the actors involved in transmission expansion and the barriers involved in constructing new lines.
Explainer — Sep 22, 2023
Transmission 101: Transmission Planning
This explainer discusses the nuts and bolts of transmission planning, relevant actors, and recent policy developments.
Report — Sep 21, 2023
Expanding the Possibilities: When and Where Can Grid-Enhancing Technologies, Distributed Energy Resources, and Microgrids Support the Grid of the Future?
This report discusses three categories of solutions that can bolster resilience, reliability, and affordability of electricity transmission: grid-enhancing technologies, distributed energy resources, and microgrids.
Resources Radio — Sep 19, 2023
Climate Policy and Environmental Justice in New York, with Victoria Sanders and Molly Robertson
Victoria Sanders and Molly Robertson discuss the Climate Leadership and Community Protection Act in New York State, options for implementing the law, and the different benefits that these options could provide among different communities.