7.6. EPA's 1987 Review of RIA Process
EPA's Use of Benefit-Cost Analysis: 1981-1986 is in part a response to GAO’s study but largely goes beyond it to make an affirmative case for EPA’s RIAs. The EPA document cites a number of GAO’s criticisms, never disagreeing with them. They seem to be accepted as valid and constructive criticism. Two examples follow:
- "The GAO report pointed to a number of limitations in the three analyses it reviewed, and a number of subsequent journal articles have addressed the subject.
- "In general, there are three major types of limitations discussed in these reviews:
- 1. Those inherent in the nature of economic analyses in general,
- 2. Those caused by gaps in available information and deficiencies in analytic techniques, and
- 3. Those that are the result of errors and omissions in the execution of the analysis.
- EPA recognizes the importance of these limitations, addressed more fully in the body of this report, and has taken them into consideration in the Agency’s guidelines for preparing RIAs." (p. S-5)
Although EPA’s RIA guidelines call for a full discussion of all analytic uncertainties in benefit-cost analyses, the GAO report and EPA's study both found that these guidelines were not always followed adequately. The GAO report recommended more discussion of uncertainties and EPA’s study recommended that more sensitivity analyses be performed whenever considerable uncertainties in the data or analyses exist. (p. 6-4)
Although EPA accepts all noted criticisms, GAO’s comments are not presented in a systematic or comprehensive fashion so the reader does not know where EPA stands on the unmentioned ones. For example, GAO was critical of the selection of regulatory alternatives in certain RIAs but this important comment is not responded to. However, and in fairness to EPA, the subject document is never represented as a point-by-point rejoinder to GAO.
In presenting the positive case that RIAs improve regulations, EPA focuses on major regulations – those requiring RIAs under the Executive Order – promulgated in the five years preceding the subject report. There were eighteen such regulations. Three were exempted from the requirement and RIAs were prepared for the other fifteen. EPA, op. cit., p. S-2.
However, of the fifteen, only six included monetized benefit estimates making them complete benefit-cost studies:
- "In general, the benefit-cost analyses prepared for air and water regulations were more likely to include monetized benefits estimates than those for other program areas. This is because the data and analytic techniques necessary for the analysis of pollutant quantities, exposures, and adverse effects are available more often for air and water regulations than for some of the other regulations. ... Six of these analyses traced benefit estimates completely from improved ambient conditions through reduced exposures and adverse effects to estimates of the monetized values of the benefits and net benefits of the regulations." (p. S-2)
Viewed negatively, 60% of the studies were not complete benefit-cost analyses. They were cost-effectiveness studies having their own, albeit, lesser virtues.
EPA clearly feels that the six complete studies were of great value:
- "These six analyses provided EPA with direct comparisons of the benefits and costs of regulatory alternatives. Two of these analyses (lead in fuels and NAAQS-PM) showed that more stringent standards could lead to greater benefits for society. One analysis (surface coal mines) showed that costs would exceed benefits for two of the three alternatives proposed. Another (organic chemicals) revealed hitherto unnoticed inter-media pollution effects that EPA is now taking into consideration. The remaining two analyses (iron and steel and PCBs) confirmed the positive net benefits of the preferred regulatory alternatives." (p. S-3)
Even some of the cost-effectiveness were apparently of some use:
- "Two of these analyses (used oil and national contingency plan) assisted EPA in selecting regulatory alternatives that will result in greater environmental benefits at less cost. Another (TSCA premanufacture review) showed how the costs of the regulation could be reduced considerably with no significant reduction in benefits. A fourth analysis (small quantity generators), on the other hand, showed that greater benefits could be achieved with only a small increase in costs." (p. S-3)
While these potential improvements cannot be attributed solely to benefit-cost analysis, it is fair to say that the analyses played major roles in bringing about regulatory improvements. (p. 5-2)
The paper contains a thumbnail sketch describing how nearly every RIA improved the contemplated regulation. An excerpt from one of those sketches, for lead in fuels, follows as a sample:
- "EPA officials realized in 1984 that the benefits of a further reduction in lead content might be substantial. Accordingly, EPA fully analyzed the benefits and costs of the alternatives.
- "This analysis revealed that reducing the lead content in gasoline from 1.1 to 0.1 grams per gallon would reduce adverse health effects ...; would reduce emissions of other pollutants; and would improve fuel economy and reduce motor vehicle maintenance costs. The present value of the net benefits ... from ... lowering the lead standard ... was calculated to be [at least] $6.7 billion ...." (p. 4-6)
This is certainly impressive but it’s not clear that economists can take credit for the entirety of the huge gain in welfare. Even a few percent of it is impressive, though.
EPA provides a general statement supporting the efficacy of the RIA process in enhancing regulatory development:
- "When trade-offs between benefits and costs have been central issues in choosing among regulatory alternatives and when legislative authority has allowed the full consideration of benefits and costs, the benefit-cost analyses have played an important role in the decision-making process." (p. 4-5)
It is unsurprising to find that each RIA tends to enhance the regulation for which it is written as is its purpose. What is somewhat surprising is to learn that the RIAs have positive spillover effects. The revelation and discussion of these effects may be the most interesting and novel aspect of the EPA paper.
- [RIAs] "have increased awareness of the environmental results of EPA’s regulations, provided a framework for comparing regulations both within a single medium and across media, identified cross-media effects, and improved analytic techniques." (p. 5-1)
Regarding “increased awareness of the environmental results of ... regulations,” the EPA document notes that the understanding of health benefits derived from studying reduced lead in gasoline has helped EPA “to evaluate the benefits of reducing lead in drinking water.”
The second spillover effect, “creating a consistent framework for evaluating environmental initiatives,"for evaluating environmental initiatives,” permits comparing
- "different regulations and environmental programs across media. This makes it possible to begin to examine the relative effectiveness of different regulations and different programs." (p. 5-3)
Unfortunately, the document does not explain the benefit from cross-program comparisons.
The third spillover effect, identifying cross-media effects results from benefit-cost analyses because they cover all media. As an example, the document cites the analysis of the treatment of wastewaters in the plastics and organic chemicals industries.
- "EPA learned that the systems being considered to treat water pollution would volatize many organic compounds, thereby creating a potential air pollution problem ...." (p. 5-3)
The fourth spillover effect is improvement in analytic technique. The furnished example concerns the National Ambient Air Quality Standard for Particulate Matter.
- "EPA was able to break new ground for this analysis. The analysis used several techniques for estimating indicators of environmental benefits, including exposure-response estimates, hedonic models of changes in property values or wages, and direct economic models of behavioral responses of individuals and firms." (p. 5-3)
Regrettably, it isn’t clear from the limited exposition how much of the cited technological progress was attributable to the study as opposed to merely used in it.
The report includes substantial material describing the shortcomings of benefit-cost analysis in general and EPA’s RIAs in particular. The distribution of benefits and costs is one consideration. Benefit-cost analyses ideally produce a measure of net benefits. However, the benefits typically accrue to society generally but the costs are incurred by the polluter.
Valuation of human health is another contentious issue. The report cites arguments by some that hedonic wage studies often used to derive mortality and morbidity values are unreliable because workers are uninformed and for other reasons.
A closely related objection concerns discounting when applied to the value of human lives. Here, ethical objections exist. As to specific failings in EPA’s RIAs, the frequent lack of distributions rather than point estimates has already been noted. Information gaps and analytic uncertainties also are noted.
EPA acknowledges a list of deficiencies in execution for which it has been criticized:
- "These include (1) using unrealistic assumptions when preparing exposure estimates, (2) not assigning dollar values to health benefits (3) not using common time periods when estimating benefits and costs, (4) failing to consider distributional effects, (5) improperly considering employment consequences and (6) not incorporating overall market trends into the economic analyses." (p. 6-5)
The report discusses the cost of preparing RIAs:
- "The cost of each RIA ranged from $210,000 to $2,380,000 with an average cost of approximately $675,000.
- "When compared with the costs of at least $100 million per year that are associated with each major regulation, a one-time cost of less than $1 million for each benefit-cost analysis seems modest. ... benefit-cost analyses often result in significant regulatory improvements worth many times the costs of the analyses. Three of the benefit-cost analyses covered by this report helped to increase the net benefits associated with their respective regulations by over $10 billion." (p. 6-5)