7.1. Morgenstern's Review
Richard Morgenstern is one of several researchers who have attempted to evaluate the regulatory impact assessments prepared by and for EPA and other agencies. A book that he edited, Economic Analyses at EPA: Assessing Regulatory Impact, contains twelve case study chapters (see Table 7.1.1) written by various authors, each reviewing an EPA RIA. Morgenstern prepared the introductory chapters and a conclusion that assesses and integrates the case studies. It is Morgenstern’s chapters, especially the conclusion, that we will consider.
|EPA RIA Analyzed|
|4||Lead in gasoline (air)|
|5||Organic chemicals, plastics, and synthetic fibers industry (water)|
|6||Stratospheric ozone depletion (air)|
|7||Asbestos (toxic substances)|
|8||Lead in drinking water (drinking water)|
|9||Municipal landfill management (solid waste)|
|10||Visibility at the Grand Canyon and the Navajo Generating Station (air)|
|11||Agricultural pesticides and worker protection (pesticides)|
|12||Vehicle inspection/maintenance (air)|
|13||Municipal sewage sludge management (water)|
|14||Reformulated gasoline (air)|
|15||Great Lakes water quality guidance (water)|
Morgenstern begins by informing us that, of the twelve rules considered, “… the benefits of three rules alone – Lead in gasoline, lead in drinking water and CFCs – far exceed the costs of all twelve rules combined.” This result is derived from the RIAs but the significance could be slight if the RIAs did not influence the rules but merely reported on them. It is a comment on the rules more than on the RIAs.
What becomes clear from reading Morgenstern’s evaluation is that the RIAs are not simple binary documents that either pass or fail a proposed regulation. Rather, there is interaction between the regulation – which should still be in a malleable stage when the assessment is undertaken – and the assessment itself. Although it is difficult to prove causality, an example of increased benefits or decreased costs probably resulting from the RIA is given for each of the 12 case studies. Morgenstern, op. cit., page 458, Table 1. However, Morgenstern notes, “… the case study authors … believe that the economic analyses played a significant role in stimulating these specific changes.” Morgenstern, op. cit., page 457.
Morgenstern emphasizes this theme of feedback from RIA to rule:
"In all cases examined, the economic analyses did, in fact, contribute to improving the rules: the value of such improvements likely dwarfs the one-time cost of conducting the analyses." (p. 456)
"In all twelve cases, the economic analyses supported specific cost-saving rule improvements. … In the case of lead phase-down, the economic analysis is cited as the source of the innovative banking and trading system." (p. 457)
"Even a cursory examination of the rule modifications associated with the RIAs suggests that, overall, they are quite significant. … in a number of cases the improvements were quite sizeable, amounting to a third or more of the benefits or costs of the rule. In both lead rules [gasoline and drinking water], for example, some might argue that most of the benefits are attributable to the economic analysis." (p. 459)
Clearly, Morgenstern sees RIAs as part of an interactive process of rule development. However, he does not see RIA development as entirely satisfactory.
Where are RIAs deficient? He identifies three actual or potential areas:
"The underlying scientific and risk information was so uncertain that it provided an insufficient basis on which to conduct an economic analysis.
The economic analysis itself was technically flawed in one or more critical ways.
The economic analysis was not designed to address a sufficiently rich array of policy options and was thus rendered irrelevant to actual policy and regulatory decisions." (p. 472)
Morgenstern regards the first and third areas as significant deficiencies but, as regards the second, he is satisfied with the quality of economic analysis, especially in the twelve case studies. However, one might contend that the third area is just a subset of the second, i.e., the analyst must choose appropriate options to consider or the analysis is flawed. There is a narrowly semantic character to this objection and it might be countered that the options, in some instances, are chosen for the analyst. In any case, Morgenstern concentrates on the first and third areas.
It is difficult to argue with his contention that a good economic analysis cannot be built on a poor scientific one. The benefits side of benefit-cost study typically begins with emissions reduction, later reaches some harm reduction and ultimately arrives at a benefit value in a long chain. The first few links in the chain are “science” or risk assessment and the last few are economic analysis. Morgenstern focuses on the Navajo Generating Station study as the most egregious offender in regard to relying on an unsatisfactory risk assessment. Morgenstern, op. cit., page 464. Besides improving the science, his remedy for dealing with a deficient risk assessment is simple:
"The agency might be well served to abandon any pretext that it is possible to assess whether or not benefits exceed costs. Rather, it might be better to conduct a more limited cost-effectiveness analysis and focus on a frank discussion of the science, including the uncertainties." (p. 465)
Morgenstern’s other grievance with some RIAs is their analysis of only a limited range of policy options:
"The omission of relevant policy options … can obscure key differences among alternatives. Perhaps the clearest example of the failure to define and analyze a broad set of policy options is the 1992 inspection and maintenance (I/M) rule, which mandated procedures for states to follow … In that case the agency did not give serious attention to approaches other than the IM240 model … Had EPA considered and analyzed other options in the original RIA, the need to revisit the issue in 1993, with the attendant political and economic costs, might have been avoided. As [case study author] Ramsden notes, 'Viewed perhaps cynically, this indicates that the policy choice selected the form of the RIA, not the other way around.'” (p. 470)
Admittedly, failing to consider certain policy options is a grievous flaw in a benefit-cost study. Valid as this criticism is, it is not operational. Morgenstern does not tell us how to generate the proper or full range of options worthy of consideration. There probably is no systematic method for generating that set. What a reader of an RIA notes as an obvious option omission in hindsight might not have been apparent to the researcher or to any researcher. Nevertheless, the criticism remains valid.
Morgenstern offers three recommendations for improving the conduct and use of economic analysis at EPA.
Conveniently, Morgenstern provides us with his assessment of the case study RIAs but also of the RIA process at EPA. Regarding the former, he states,
"If we were grading these economic analyses, most of them would get grades of 'B' or 'C.' At least four of the twelve deserve 'A’s:' lead phase down, CFCs, lead in drinking water, and asbestos. None is guilty of the type of errors that would merit a 'D' or an 'F.'" (p. 466)
Morgenstern obligingly provides a brief benefit-cost analysis of the benefit-cost analyses. He summarizes it as follows:
"For the group of cases in this volume, the present value of the annual increase in benefits and/or decrease in costs of the rules attributable to the economic analyses clearly outweighs the one-time cost of carrying out the analyses." (p.463)
However, lest one come away with an altogether felicitous view of RIAs through Morgenstern’s eyes, it should be noted that he expresses a less sanguine view earlier: “Later chapters … show that these Presidential directives have so far failed to produce consistently high quality and relevant economic assessments." Morgenstern, op. cit., page 21. Clearly, Morgenstern perceives that economic analysis at EPA works but that it is flawed. He also believes specific remedial steps would be effective.