7.4. API Review
In a paper for the American Petroleum Institute (API), Rusin et al. also reviewed the content of several regulatory impact assessments. The first two chapters of this paper provide analysis, review and comments on the regulatory process. Each of the remaining seven chapters evaluates an RIA. Most of the RIAs deal with issues of particular import to the petroleum industry.
The API study emphasizes four conclusions.
First, “existing legislation is the major administrative impediment to use of benefit cost analysis as a decision tool.” API, op. cit., p. 3. Several environmental statutes preclude or limit the application of benefit cost analysis even though it is required under Executive Orders. Note that this assertion is contradicted to some extent by Hahn.
- "Adequate cost benefit analyses can be done, but they are difficult and expensive. ... Existing RIAs range in quality and usefulness from high quality, complete decision making tools to poor and incomplete." (p. 3)
Perhaps this second comment might be construed to mean merely that the task of producing an adequate cost benefit analysis is not impossible. The authors note that in some instances, a better RIA would have resulted from simply avoiding mistakes whereas other RIAs needed more resources invested in them.
- "With relatively few exceptions, the theoretical underpinnings and empirical methods of benefit cost analysis are well established and agreed upon." (p. 4)
This comment pertains to the field generally and not just to the case studies. The authors note that non-use or existence values may be an exception to their “agreed upon” claim. Possibly they could add contingent valuation as another contentious matter. The field may not be as well settled as they perceive.
- "Industry comments on RIA’s are predominantly technical; industry comments on economic topics are usually limited to presenting and commenting on expenditure estimates." (p. 4)
API represents industry. In the case studies, industry comments on draft RIAs are analyzed.
Included in the introductory chapters is a section dealing with the cost of pollution abatement. Here the argument is advanced that social cost typically differs from compliance cost. RIAs rely on the latter nearly exclusively which is a criticism of all of them.
Social cost may be more or less than compliance cost, the authors explain, because the latter neglects general equilibrium considerations; there is no adaptation or substitution, just control. Further, dynamic adjustments are ignored. Social cost may be substantially more or less than compliance cost.
The extensive introductory material that precedes the case studies discusses the theory and evolution of RIAs in a general way and, although excellent, does not, for the most part, constitute a commentary on RIAs generally or specifically. We need to look at the case studies to see what the API authors perceive.
1 Reducing Lead in Gasoline
The first RIA reviewed by the API authors dealt with reducing lead in gasoline. They note:
- "This RIA was one of the first, if not the first, comprehensive analysis and quantification of both costs and benefits of a proposed environmental rule." (p. 26)
How good was the first full benefit-cost study? The API study faults the RIA for improperly estimating nearly all benefit categories. Regarding children’s health, they object that:
- "The methods EPA used for monetizing benefits in many cases were not the 'willingness to pay' methods that are now conceptually preferred. Instead, the method used to look at such things as the direct and indirect costs of illness, including medical costs saved, changes in earnings, etc." (p. 25)
They level the same objection at the largest benefit category, blood pressure reduction in adults. API, op. cit., p.26.
As to the value of lives saved, they note:
- "Current life saved values, even adjusted for inflation from 1983 values, are usually higher than the $1 million used in the RIA." (p. 25)
The API authors’ harshest criticism, however, was reserved for the purported economies of using unleaded gasoline:
- "EPA maintained that removal of lead and lead scavengers would lead to significant and substantial improvements in engine life and reductions in maintenance requirements and cost. Reduction in maintenance costs in the peak year (1986) were estimated at an astonishing $900 million.
- "EPA also estimated fuel savings on the order of $100 million per year ....
- "It is difficult to accept the magnitude of EPA’s estimate of savings in maintenance and fuel economy. These estimated savings alone exceed EPA’s estimate of unleaded gasoline manufacture costs. A necessary premise is that there is some form of imperfection or failure in the market for gasoline, preventing ordinary market forces from providing unleaded gasoline and realizing the maintenance and economy savings, thereby making both buyers and sellers of unleaded gasoline better off." (p. 26)
On the cost side, the authors suspect that an underestimate was made:
- "EPA’s estimate of the cost of lead removal was about two cents per gallon, considerably less than the observed differential in price between leaded and unleaded grades over the phase-down period. This piece of evidence suggests that EPA did underestimate costs." (p. 28)
The review’s overall comment on the cost-benefit analysis is sanguine even though significant objections were directed at nearly all the elements of the calculation:
- "In summary, while all the categories of monetized benefits have areas where criticisms can be leveled, particularly the maintenance and fuel economy savings, which probably are high, most are reasonable. EPA can be accused of some upward bias in benefit estimates and downward bias in cost, but no reasonable adjustment would alter any of the estimates sufficiently to change the overall conclusions." (p. 26)
Some benefit estimates seemed to the reviewers high, mainly maintenance and fuel savings, and others low, such as morbidity and mortality. Whether the resulting overall benefit estimate is biased upwards as claimed seems unclear.
That “no reasonable adjustment would ... change the overall conclusions” seems valid given that benefits were more than an order of magnitude greater than costs. The API authors, having made a telling criticism against nearly every cost and benefit item, observe correctly that the conclusion is nevertheless true. That is, the RIA succeeds despite itself because the case for the regulation is so compelling. However, fairness and charity require remarking that the API review was written more than a decade after the RIA during which time the art of benefit-cost analysis had progressed significantly.
2 The National Ambient Air Quality Standard for Carbon Monoxide
The Clean Air Act requires EPA to review scientific criteria and ambient air quality with a view to revising national ambient air quality standards (NAAQS) and to do so every five years. For this purpose, EPA reviewed the carbon monoxide (CO) standard in 1985 by preparing an RIA. Four regulatory alternatives were considered.
The API authors note that, “EPA did not attempt to monetize benefits; whether it had the technical ability to do so in 1985 is debatable ...” API, op. cit., p.31. A form of cost-effectiveness analysis was used.
They conclude by observing:
- "This RIA tries to support an action that by itself will have relatively modest costs relative to the existing standard. However, the mobile source controls called for in the Clean Air Act rest in large part on the determination that many areas of the country do not meet EPA’s NAAQS for CO. Had the NAAQS been set at modestly higher levels, some of the stringent mobile source controls would not have been required. Since the overall cost of the nation’s effort to control is far from trivial – amounting to several billion dollars annually – a more comprehensive RIA that does document the economic benefits of alternative standards would have been illuminating and might have changed the level of control sought for CO. It could be, for example, that the adverse health effects (and the economic value thereof) of alternative 2 [other regulatory approaches, e.g., performance standards for mobile and stationary sources] or alternative 3 [market-oriented alternatives such as charges and marketable permits] were zero or very close to zero.
- "One useful step would have EPA embark on a program to support scientific studies aimed at narrowing the uncertainty. A second, and related step, would have EPA sponsor studies geared toward establishing economic values (willingness to pay) for health effects of potential concern. With the results of better scientific and economic information, EPA could render a much more informed decision. Inasmuch as the NAAQS for criteria pollutants must be revised periodically and a number of areas remain out of attainment for CO, it is not too late for EPA to take these actions." (p. 32)
One could conclude that the API authors perceive the economics and underlying science in the RIA as deficient and that they hope for improvement on both counts as EPA repeats this process periodically.
3 Phase I and II Gasoline Volatility Regulations
EPA prepared a draft RIA in 1987 and a final RIA in 1990 dealing with gasoline volatility. The objective was to reduce evaporative losses of hydrocarbons, a precursor of ozone for which there are national standards. Neither RIA contains a complete benefit-cost analysis.
- This RIA does not estimate the economic benefits of reducing gasoline’s RVP [Reid Vapor Pressure, a measure of volatility]. Instead, ... EPA describes ... the environmental impacts of the proposed change. EPA does not translate changes in emissions levels to changes in health or other impacts (e.g., effects on materials, forest, or crops). (p. 35)
The RIA stops two steps short of monetizing benefits. What EPA did was to study cost-effectiveness.
- EPA calculated changes in cost and emission reduction incrementally, recognizing the increasing nature of unit decreases in RVP ... (p. 36)
API’s harshest criticism of the RIA is that it failed to analyze other control options such as onboard canisters. The marginal cost of RVP reduction is sensitive to the baseline which is established by other policies. Hence, policies are interrelated and, at least in principle, should therefore be analyzed jointly to find the least cost combination for a given abatement level. However, this point is to some extent contradicted by a retrospective view of the RVP regulation and the other controls:
- "The benefit was achieved with a reasonable cost-effectiveness, particularly given how much more costly further controls on vehicles and fuels have turned out to be." (p. 37)
4 Control of the Sulfur and Aromatic Hydrocarbon Content of Highway Diesel Fuel
EPA prepared an RIA leading to a 1985 rulemaking concerning vehicular particulate emissions. EPA considered only two regulatory alternatives, both affecting only fuel, which led to API’s vigorous objection:
- "EPA examined two regulatory alternatives: (1) reduce sulfur levels from approximately 0.25 weight percent to .05 weight percent; and (2) reduce sulfur levels as in (1) and also reduce aromatic content from about 35 volume percent to about 20 volume percent. EPA did not explicitly consider other target sulfur level reductions, a major omission.
- "For all intents and purposes, consideration of these and only these two alternatives dictated the results. Other target sulfur levels might have yielded better information and results and should have been considered." (p. 39)
As to the analysis of compliance costs, API notes:
- "The RIA noted but did not quantify the increased costs associated with segregation, distribution, and marketing of low sulfur diesel, i.e., effectively adding another grade to the distillate pool." (p. 39)
API expresses skepticism about EPA claims of better engine wear resulting from reduced fuel sulfur:
- "The RIA also concludes that approximately half of engines burning low sulfur fuel will benefit by 18% longer intervals between rebuilding, and longer life. This results in substantial cost savings, ranging from 8.4 to 30.7 cents per gallon of fuel used, depending on vehicle type. This is almost an order of magnitude greater than desulfurization cost, raising the question of whether or not truck operators either knew of or would agree with this assessment. If the RIA is correct and truck operators were aware of this potential savings, we would have expected them to demand and use low sulfur fuel. We did not observe this to be the case, suggesting that either the RIA is wrong, or truck operators are ignorant. The former seems more plausible." (p. 40)
The questionable engine wear claims drive the economic analysis according to API:
- "Economic analysis is limited to cost effectiveness. The RIA does not monetize any of the air quality changes. The RIA takes such large credits for reduced engine wear and other improvements to vehicle operation that these credits in most cases become larger than the cost of desulfurization. Thus:
- "'Beginning in the year 2000, net costs for sulfur control in the maximum segregation scenario actually become negative, indicating that societal savings will outweigh the cost to refiners of producing low sulfur fuel.' (RIA, page 7-14)
- "It is as if, given the “free lunch” quality of the desulfurization of diesel fuel, there is no need to go further in quantifying benefits. Curiously, the RIA refers to a 'Chapter 8' where welfare benefits are quantified. Curiously, there is no Chapter 8, and none is listed in the table of contents." (pp. 40-41)
In its concluding remarks on this RIA, API states with reference to the engine wear matter,
- "The conclusions of the RIA appear implausible.
- "In order for this analysis to be of any use in comparing benefits and costs, a more realistic analysis of the effects ... is needed, as is a monetization of air quality benefits." (p. 41)
5 The Proposed Great Lakes Water Quality Guidance
The Guidance document deals with point source discharges into the Great Lakes. It provides direction to the states on water quality standards and specifies criteria for certain pollutants.
API’s criticism begins with trading considerations:
- "An important regulatory alternative not considered in the RIA was the use of effluent trading as a means of ensuring cost-effective controls across point sources." (p. 44)
The cost side of the analysis proceeded by dealing with a sample of point source dischargers and geographic subsets of the Great Lakes. API expressed reservations:
- "The approach used to select the sample of major dischargers was based on time and resources available, not on a predetermined level of statistical significance. It is therefore questionable whether the cost results are statistically significant ....
- "Potentially significant factors not considered in the RIA analysis include possible control of non-point sources, the costing of emission control when allowable levels are set below analytical detection limits, and future criteria development (as new criteria are developed for Tier II pollutants, limits will be incorporated into permits under the Guidance). Other factors not considered include additional costs to expanding plants, costs incurred by new facilities locating in the Great Lakes area, and regional impacts." (pp. 47-48)
In addition to questioning the RIA cost estimates, API refers to higher, industry-supplied estimates:
- "If industry cost estimates are compared with EPA derived benefits, the conclusion would be that net societal welfare would decrease with the implementation of the Guidance. ... the sensitivity analysis of EPA demonstrated that the costs of the Guidance could exceed the benefits." (pp. 54)
The API authors were similarly concerned about the benefits side:
- "An important assumption in the calculation of benefits was the determination of the fraction of a given benefit estimate attributable to the Guidance. The limited amount of information that exists suggests that approximately 90 percent or more of the current concentrations of toxins at the mouth of the Lower Fox River [one of the areas studied] are due to sediment (reflecting historic rather than on-going loadings to the system), and on the order of 1 percent are due to point sources. Assigning a value of 50 percent of the benefit estimates to the Guidance appears to be unwarranted based on current information and was not based on empirical evidence. As explained in the RIA and illustrated in Table 8 [not reproduced], the benefits are highly sensitive to this assumption.
- "Under an attribution assumption of 10 percent or less, dramatically different conclusions than those arrived at in the RIA are obtained. ... costs would exceed benefits and therefore ... net social welfare would be decreased through implementation of the Guidance. ... On the other hand, various categories of benefits were not estimated, e.g., human health, subsistence fishing, waterfowl hunting, and recreational boating and swimming ... These benefits are likely to be significant. The conclusion that must must be drawn is that sufficient information does not exist to determine either the sign or the magnitude of net social benefits ...." (p. 51)
The API authors conclude:
- "In sum, the omission of major sources of discharge (non-point sources), combined with the existing regulatory, technological, and statistical uncertainties, results in inconclusive net benefit estimates." (p. 55)
6 Emissions of Hazardous Air Pollutants from Petroleum Refineries (NESHAPS)
EPA is required under the Clean Air Act to control emissions of hazardous air pollutants (HAP) from refineries which is the ostensible purpose of the proposed rulemaking for which the RIA was developed. However, most of the benefits derive from the ancillary congenial effect of diminished emissions of volatile organic compounds (VOC), precursors of ozone. API, op. cit., pp. 62-63.
On the cost side of the analysis, API expressed reservations. The EPA model involves a supply function shifting because of regulation against a demand function. Welfare triangles are summed to derive cost. The API authors dispute the elasticities used for both functions. API, op. cit., p. 59. Whether the numeric inputs to the cost calculations are appropriate can be debated but the structure conforms more closely to the social cost concept advocated in the introduction to the API document than the accounting or compliance cost measure used in other RIAs although it is partial not general equilibrium.
The API authors note with some reservation that benefits are calculated using the method of benefits transfer:
- "There are also some problems in EPA’s treatment of benefits. One, it relies on benefits transfer from an OTA study that did not claim to have a comprehensive estimate of ozone control benefits. It has also been contended that some studies published (although not all peer reviewed) in the seven years since the OTA document was issued have shown health impacts at lower levels of exposure than previously had been identified. Second, the OTA study does not value possible reductions in chronic effects ...." (p. 62)
Of the seven RIAs they review, this one seems to come closest to garnering the approval of the API authors. It treats costs in a sophisticated way. It considers at least some alternatives. It is a complete benefit-cost study. But they don’t find it perfect. They wonder if
- "An alternative approach, targeted more specifically at ozone reduction in particular or air quality benefits in general , can be constructed more efficiently." (p. 62)
The Occupational Safety and Health Administration (OSHA) undertook to regulate exposure to asbestos under the OSHA Act and the Construction Safety Act. Asbestos is a carcinogen and can induce asbestosis.
The API authors commend OSHA for developing incrementally more stringent rules for incrementally greater exposure risk:
- "The fourfold classification of activities and the assignment of incremental controls for each category is a positive step toward regulatory flexibility." (p. 67)
On the benefits side, OSHA took the analysis nearly to a successful conclusion:
- "Summing up, OSHA discussed benefits of the final rules in some detail. The agency discussed major quantifiable and non-quantifiable benefits and identified other ancillary benefits. OSHA’s reluctance to attach a monetary value to averted deaths is distressing. The existence of a well-developed literature on valuation of a statistical life, and the pains OSHA took to estimate the number of avoided deaths both made that task very easy." (p. 68)
The API authors are even less sanguine about the cost side:
- "Discussion with an API expert suggests OSHA grossly underestimated costs of certain compliance related activities and totally ignored at least one exposure-causing activity. For example, API has documented compliance costs sever orders of magnitude higher than the OSHA estimate for removal asbestos gaskets. API also pointed out OSHA had completely ignored costs related to removal of underground pipeline coating material." (p. 69)
After noting that the analysis of costs stops with firms’ compliance cost rather than capturing social costs, the API document sums up the cost side flaws in the OSHA RIA:
- "It is difficult to escape the impression that OSHA devoted a lot more attention to costs than it did to benefits from the regulation. ... Yet the analysis of costs is quite incomplete despite the attention given to it. Besides not considering the dead-weight loss to the society, the analysis does not acknowledge costs such as the cost of monitoring compliance ..., and the adjustment costs to firms." (p. 69)
The API authors conclude by attempting to integrate the cost and incomplete benefit analyses:
- "OSHA did not quantify benefits; nor did it attempt to compare costs to benefits. A back-of-the-envelope calculation of health benefits presented above suggests they were substantial and probably of the order of a third of the estimated compliance costs. The magnitudes of costs and benefits may be closer if secondary non-health benefits are also considered, as they should be. Another measure of the efficiency of the regulation is the cost to society per statistical life saved. OSHA did not calculate this. Using the agency’s estimates of compliance cost and prevented deaths, we obtain a value of approximately $8.48 million as the cost per statistical life saved. ... These constructs are above the upper end of the range most experts are comfortable with for the value of a statistical life. One may that the final standards were probably marginally inefficient." (p. 70)