The Treatment of Uncertainty in EPA’s Analysis of Air Pollution Rules: A Status Report

An understanding of the uncertainty in benefit and cost estimates is a critical part of a benefit–cost analysis. Without a quantitative treatment of uncertainty, it is difficult to know how much confidenceto place in these estimates. In 2002, an NRC report recommended that EPA move toward conducting probabilistic, multiple-source uncertainty analyses in its RIAs with the specification of probabilitydistributions for major sources of uncertainty in the benefit estimates. In 2006, reports by GAO and RFF found that EPA had begun to address the NRC recommendations, but that much remained to be done to meet the NRC concerns. This paper provides a further review of EPA’s progress in developing a quantitative assessment of the uncertainties in its health benefits analyses for the RIAs for four recent NAAQS rulemakings. In conclusion, EPA’s recent RIAs present the results of its uncertainty analyses in piecemeal fashion rather than providing an overall, comprehensive statement of the uncertainty in its estimates. In addition, its recent RIAs continue to focus on the concentration-response relationship and largely fail to address the uncertainty associated with the other key elements of the benefits analysis.

In the United States, outdoor air pollution results in roughly 65,000 deaths per year, with estimated annual health damages ranging from $70–$270 billion. When considering new air quality rules, the U.S. Environmental Protection Agency (EPA) estimates the health benefits of reducing air pollution as a way of understanding the consequences (costs and benefits) of its regulatory actions. EPA’s analysis of the uncertainty in its estimates has been criticized in recent years, and a new RFF analysis suggests that the agency’s methodology still doesn’t measure up to expectations.

In a 2002 report titled Estimating the Public Health Benefits of Proposed Air Pollution Regulations, the National Research Council (NRC) of the National Academy of Sciences recommended that EPA update its health benefits analysis by conducting a more comprehensive quantitative assessment of uncertainty for its regulatory impact analysis (RIA).

In a new discussion paper (DP 10-04), The Treatment of Uncertainty in EPA’s Analysis of Air Pollution Rules, RFF Visiting Scholar Arthur G. Fraas provides a status report of EPA’s progress in carrying out the NRC’s recommendations by examining the RIAs for four recent proposed and final National Ambient Air Quality Standards (NAAQS) rulemakings. The author says that although some, largely minor, improvements have been made, EPA’s basic approach to presenting the uncertainty in its health benefits estimates remains largely unchanged in critical ways:

• The responsibility of assessing and combining uncertainty information is left largely up to the reader.
• Estimates focus on the concentration-response relationship and largely fail to address the uncertainty associated with other key elements in the benefits analysis.
• Expert elicitation studies provides useful information, but overall, the current PM analysis falls far short of yielding a more comprehensive, quantitative representation of uncertainty in health benefits.
• The PM expert elicitation study does not address the uncertainty in the concentration-response relationship for the other criteria pollutants subject to the NAAQS.

Fraas notes that EPA’s tight rulemaking schedule and limited staff and budget resources represent strong barriers to improving its uncertainty analysis. Nevertheless, he says, EPA should make improving this system a priority.