|Project Status Reports:
For the year 1997
Objective: The goal of this project is to investigate the claim made in Gregory, et. al. 1993 (Journal of Risk and Uncertainty, Vol. 7), that multi-attribute utility (MAU) questioning techniques hold promise for direct valuation of environmental goods and services because they: (a) reduce the cognitive demands on lay respondents by simplifying the questions asked; and (b) are congruent with the multi-dimensional character of many problem settings, ecosystem valuation in particular. Specifically, using a forest valuation case study, we are examining whether and how MAU: (1) can be implemented in a manner consistent with the case; and (2) affects how lay respondents consider six-dimensional descriptions of forests.
Our approach involved the creation of an MAU valuation survey instrument that is based on a six-dimensional description of a southern appalachian forest. The dimensions are intended to be ecologically meaningful and yet relevant to respondents’ judgments about the value of forests to them. (We do not prejudge the identities of the sources of these values or the suitability of any particular forest relative to any particular one of these sources.)
The six dimensions or attributes that we are using are listed in Figure 1, which is the first response work sheet from the survey. The questions on this sheet ask respondents to identify their most- and least-preferred levels of the attributes. They do this as they view visual representations and read descriptive material concerning the attributes.
The other steps in the survey are sufficiently straightforward to be within the capability of even respondents with severely limited education. These steps are: (1) to list the attributes in order of declining importance (triggered by a question asking which attribute the respondent would change first, from least to most preferred level, if they had the power); (2) to supply “swing” weights that quantify the relative importance list; and (3) to answer willingness to pay (WTP) questions concerning the most important attribute. (These questions ask about WTP to ensure that the attribute will be found at its most, rather than its least, preferred level in a forest that the respondent could easily visit.) Linearity and independence assumptions make these answers sufficient to determine what we might call the “sub-WTP” functions for each attribute and respondent. (There is some residual uncertainty regarding parts of the functions for attributes for which a respondent has picked an interior most-preferred level in the first step.)
The last questions of the survey, and the key to the test of whether MAU makes a difference, concern three “blended” forests – forests that are described using the same six attributes in three different combinations, with the combinations presented all at once. Respondents are asked to state their preferences for forest 1 versus forest 2, for forest 2 versus forest 3, and for 1 versus 3 (the last question providing them with enough scope to display intransitivity). Respondents are also asked to supply WTP judgments for the difference between their preferred and not preferred choice in each of the first two pairings. Based on their answers to the MAU questions, we can calculate WTP numbers for each respondent and blended forest. These can then be compared with the stated preferences and WTP numbers from the last part of the survey.
Progress Summary: Our findings should be considered very preliminary. The first data come from a “deliberative polling” exercise held in Nashville this past fall, from which we obtained about 75 completed surveys. Another, larger, event will be held in early March. This mode of administration has been necessary because we believe the instrument is far too long for a successful mail survey, and our budget will not support one-on-one interviewing.
We find that MAU does work in the sense already noted; i.e., that the tricky business of asking questions concerning a multi-dimensional ecosystem can be simplified enough that poorly educated respondents can answer. But this achievement comes at a price. Despite our many simplifications, the survey is long.
On the crucial matter of the blended forests, our first examination of the data reveals that people who have worked through all of the material do not have much trouble when asked to consider changing combinations of all six attributes. Thus, we have seen no intransitivity implied by the preference statements, suggesting no serious confusion. Further, by and large, the stated preference orderings are the same as the orderings implied by the answers to the MAU questions. And the stated WTPs are at least in the same realm as the values implied by the sub-WTP functions derived from the MAU responses.
While we do not want to base conclusions on these early numbers, they suggest that a large investment of time and effort in familiarizing respondents with aspects of a complex problem may be as important as the details of the questioning technique employed to seek their preferences and even their WTPs.
Future Activities: Our next steps will involve additional data gathering, computation of all relevant quantities for all respondents, and development of appropriate formal tests for the variety of comparisons possible within the data.
Figure 1. Response Work Sheet. - see attachment
Clifford Russell, Virginia Dale, Junsoo Lee, Molly Hadley Jensen, Michael Kane, and Robin Gregory, Applying Multi-attribute Utility Techniques to Environmental Valuation: A Forest Ecosystem Example, Project Report to EPA, December 10, 1997.
The three objectives of this research project were to:
1. Devise a general method of describing an ecological system that: (a) is connected to underlying ecological indicator measurements in a well-defined way; (b) relates both to the multiple system functions that lie at the root of human valuations of the system and to the principles that mediate the response of individuals to those functional capabilities; and (c) reduces the problem of dimensionality without taking the draconian step of describing an ecosystem only in terms of suitability for one function.
2. Flesh out and apply the method of value elicitation (or "construction"), based on the multi-attribute utility (MAU) approach to decision making (following the suggestion of Gregory, et al., 1993).
3. Apply the above techniques in a forest ecosystem case study context and comparing the performance of the MAU method with the more traditional alternative of a take-it-or-leave-it (discrete choice) elicitation method, first, with the description to be valued based on the "dimensions" as above in (1) and, second, with the description of a summary that attempted to "smear" the dimensions.
The use of willingness-to-pay (WTP) survey techniques based on MAU approaches has been recommended by some authors as a way to deal simultaneously with two difficulties that increasingly plague environmental valuation. The first of these is that, as valuation exercises come to involve less familiar and more subtle environmental effects, such as ecosystem management, lay respondents are less likely to have any idea, in advance, of the value they would attach to a described result. The second is that valuation questions may increasingly be about multidimensional effects (e.g., changes in ecosystem function) as opposed to changes in visibility from a given point.
MAU has been asserted to allow the asking of simpler questions, even in the context of difficult subjects. It is, as the name suggests, inherently multidimensional.
This project asked whether MAU techniques can be shown to "make a difference" in the context of questions about preferences over, and valuation of differences between, alternative descriptions of a forest ecosystem. Making a difference was defined in terms of internal consistency of answers to preference and WTP questions involving three 5-attribute forest descriptions.
The method involved first asking MAU-structured questions attribute-by-attribute. The responses to these questions allowed us to infer each respondent's preferences and WTP. Second, the same respondents were asked directly about their preferences and WTPs.
Our positive findings were:
- It is possible to construct an MAU-based survey instrument, embodying multiple independent dimensions of a complex valuation problem (in our case, forests). The questions about preferences over the scale of each dimension, relative importance of the dimensions (numerically expressed), and WTP to alter one of the dimensions can be answered, even by people with limited education.
- Participants, who ranged in age from high school students to volunteers from a nursing home, were generally quite willing to work through the tasks given to them and to think about valuation in the context of multiple attributes for a forest ecosystem. This positive result underlies the appeal of a constructive approach to valuation (Payne, et al., 1992) and its fundamental assumption that our notions of value are built up, piece-by-piece, much as a building is constructed. Of course, some buildings are built better than others, and protocols for the design of multi-attribute environmental evaluation efforts are still at an early stage (Gregory and Slovic, 1997). Nevertheless, the willingness of diverse respondents to undertake this rather lengthy task, and to stick with it through to a monetary valuation, suggests a fit between the way the questions were posed and how many participants naturally think about the types of policy questions that might affect management of a forest ecosystem.
- Their answers, combined with a quite restrictive linearity assumption, allow the derivation of a "subwillingness to pay function" for each dimension or attribute. These functions can, in turn, be used to infer at least relative values for the particular multidimensional good at issue described by combinations of the attributes. In particular, it is possible to make judgments among alternative possible goods, either on the basis of "votes" (aggregating ordinal preferences) or total WTP.
- The inferred preferences and WTP figures approximate, though they do not perfectly match, the stated preferences and WTP numbers obtained directly from respondents.
- The stated WTP answers themselves appear, in general, to be sensibly related to key socio-economic characteristics of the respondents.
A more skeptical person might question the importance of these findings by pointing to some awkward facts, such as:
It is not clear that the MAU process makes much difference in the chosen setting, multidimensional though it is, because:
Thus, it may be that MAU could be useful in more complex problem settings, for which the vector comparisons would be overwhelming—if there were more dimensions, for example. However, a skeptic might well say, in addition, something along the following lines:
- Subsample asked for blended forest preferences without the benefit of the MAU educational process exhibited even less cyclicity (taken as evidence of confusion about the vector comparisons).
- The mean WTP answers of this group for the differences between blended forests were in one case identical to the mean from the "educated" sample and in one case different.
- The stated preference orderings over the blended forests were not significantly different for the uneducated and educated samples.
- Even granting that each question is quite simple, the facts are that: (1) the overall instrument took a long time to complete, so it was almost certainly not a good candidate for a mail survey, which in turn implies the technique may be expensive to use; and (2) only 75 percent of those who sat down to do the survey successfully (completely) finished.
- The several stated WTPs are about the only results that seem to "make sense," if the test is: Can we explain the variation across respondents by their characteristics and self-reported experiences (with forests)? In particular, there appears to be no straightforward explanation of variation in the matches between implied and predicted preferences and WTP numbers for the blended forests pairs.
So, it seems clear that the jury is still out on the promise of MAU as an alternative to the conventional contingent valuation technique for problems such as ecosystem valuation. The approach cannot be rejected as without promise, but it also cannot be embraced as the answer to the problems of cognitive challenges—especially multidimensionality—identified in the literature and likely to become more common as the boundaries of the search for dollar values in the environment are pushed out by the needs of policy makers.
References: Gregory R, Lichtenstein S, Slovic P. Valuing environmental resources: a constructive approach. Journal of Risk and Uncertainty 1993;7:177-197. Gregory R, Slovic P. A constructive approach to environmental valuation. Ecological Economics 1997;21(3):175-182.
Russell C, Dale V, Lee J, Jensen MH, Kane M, Gregory R. Experimenting with multi-attribute utility survey methods in a multi-dimensional valuation problem. Ecological Economics (submitted for publication).
Schiller A, Konar V, Hunsaker C, Kane M, Wolfe A, Dale V, Suter G, Russell C, Pion G, Jensen MH. Communicating ecological indicators to decision makers and the public. Conservation Ecology August 2000 (revised and resubmitted for publication).
Russell C. Experimenting with multi-attribute utility survey methods in a multi-dimensional valuation problem. Presented at the Norwegian Agriculture University, February 1997.
Russell C. Experimenting with multi-attribute utility survey methods in a multi-dimensional valuation problem. Presented at the University of East Anglia, Norwich, United Kingdom, March 8, 1997.
Russell C. Experimenting with multi-attribute utility survey methods in a multi-dimensional valuation problem. Presented at Beijer Institute, Royal Swedish Academy of Sciences, Stockholm, Sweden, April 15, 1997.