The Use (and Abuse) of Meta-Analysis in Environmental and Natural Resource Economics: An Assessment
Tóm tắt
Motivated by the 2006 report of a Work Group appointed by the Environmental Protection Agency (EPA), this paper examines the present state of meta-analysis in environmental economics and offers recommendations for its future use. To this end we summarize and assess 140 meta-analyses from 125 published and unpublished studies, covering 17 topical categories in environmental and resource economics. First, we provide several generic meta-analysis models as reference points and discuss major estimation issues. Five econometric issues are identified as part of a complete analysis: (1) sample selection criteria; (2) basic data summary; (3) primary data heterogeneity; (4) heteroskedasticity; and (5) non-independence of multiple observations from primary studies. Second, a tabular summary is presented for the 140 meta-analyses with respect to estimation methods. Third, a narrative summary is presented for 19 meta-analyses, including the three value-of-statistical-life studies examined by the EPA Work Group and one analysis from each of 16 other categories. Fourth, we offer a set of “best practice” guidelines for future meta-analyses in this and other areas of economics. Last, the paper comments on the use of meta-analytic methods for benefit transfers of environmental values.
Tài liệu tham khảo
Aigner DJ (1973) Regression with a binary independent variable subject to errors of observation. J Econom 1: 49–60
Asenso-Boadi F, Peters TJ, Coast J (2008) Exploring differences in empirical time preference rates for health: an application of meta-regression. Health Econ 17: 235–248
Baltagi BH (2005) Econometric analysis of panel data, 3rd edn. Wiley, New York
Banzhaf HS, Smith VK (2007) Meta-analysis in model implementation: choice sets and the valuation of air quality improvements. J Appl Econom 22: 1013–1031
Bateman IJ, Jones AP (2003) Contrasting conventional with multi-level modeling approaches to meta-analysis: expectation consistency in UK woodland recreation values. Land Econ 79: 235–258
Bellavance F, Dionne G, Lebeau M (2007) The value of a statistical life: a meta-analysis with a mixed effects regression model. Working paper 06–12, Ecole des HEC Montreal
Bergstrom JC, Taylor LO (2006) Using meta-analysis for benefits transfer: theory and practice. Ecol Econ 60: 351–360
Berk R (2007) Statistical inference and meta-analysis. J Exp Criminol 3: 247–270
Bickel R (2007) Multilevel analysis for applied research. Guilford Press, New York
Bijmolt THA, Pieters RGM (2001) Meta-analysis in marketing when studies contain multiple measurements. Mark Lett 12: 157–169
Bom PRD, Ligthart JE (2008) How productive is public capital? a meta-analysis. Unpublished paper, Tilburg University
Brander LM, Florax RJGM, Vermaat JE (2006) The empirics of wetland valuation: a comprehensive summary and a meta-analysis of the literature. Environ Resource Econ 33: 223–250
Breusch TS, Mizon GE, Schmidt P (1989) Efficient estimation using panel data. Econometrica 57: 695–700
Brons M, Nijkamp P, Pels E, Rietveld P (2008) A meta-analysis of the price elasticity of gasoline demand: a SUR approach. Energy Econ 30: 2015–2022
Brouwer R, Langford IH, Bateman IJ, Turner RK (1999) A meta-analysis of wetland contingent valuation studies. Reg Environ Chang 1: 47–57
Brown M, Forsythe A (1992) Robust tests for the equality of variances. J Am Stat Assoc 69: 364–367
Cameron AC, Trivedi PK (2005) Microeconometrics: methods and applications. Cambridge University Press, Cambridge
Cavlovic TA, Baker KH, Berrens RP, Gawande K (2000) A meta-analysis of environmental Kuznets curve studies. Agric Resour Econ Rev 29: 32–42
Christensen P (2003) Topics in meta-analysis: a literature survey. Institute of Transport Economics, Oslo
Cooper H, Hedges LV (1994) The handbook of research synthesis. Russell Sage Foundation, New York
Dalhuisen JM, Florax RJGM, de Groot HLF, Nijkamp P (2003) Price and income elasticities of residential water demand: a meta-analysis. Land Econ 79: 292–308
Daniel VE, Florax RJGM, Rietveld P (2007) Flooding risk and housing values: an economic assessment of environmental hazard. Working paper 07–02, Purdue University
Day B (1999) A meta-analysis of wage-risk estimates of the value of statistical life. In: EC, Benefits transfer and the economic valuation of environmental damages, European Commission, Brussels, pp 1–29
de Blaeij A, Florax RJGM, Rietveld P, Verhoef E (2003) The value of statistical life in road safety: a meta-analysis. Accid Anal Prev 35: 973–986
Debrezion G, Pels E, Rietveld P (2007) The impact of railway stations on residential and commercial property value: a meta-analysis. J Real Estate Fin Econ 35: 161–80
Desvousges WH, Johnson FR, Banzhaf HS (1998) Environmental policy analysis with limited information: principles and applications of the transfer method. Elgar, Cheltenham
Elvik R (2005) Can we trust the results of meta-analyses. Transp Res Rec 1909: 221–29
Espey M (1996) Explaining the variation in elasticity estimates of gasoline demand in the United States. Energy J 17: 49–60
Espey JA, Espey M (2004) Turning on the lights: a meta-analysis of residential electricity demand elasticities. J Agric Appl Econ 36: 65–81
Feld LP, Heckemeyer JH (2008) FDI and taxation: a meta-study. Unpublished paper, University of Heidelberg and ZEW Mannheim
Florax RJGM (2002a) Accounting for dependence among study results in meta-analysis: methodology and applications to the valuation and use of natural resources. Research memorandum 2002–2005, Free University of Amsterdam
Florax RJGM (2002b) Methodological pitfalls in meta-analysis: publication bias. In: Florax RJGM, Nijkamp P, Willis KG (eds) Comparative environmental assessment. Elgar, Cheltenham, pp 177–207
Florax RJGM, Nijkamp P, Willis KG (2002) Comparative environmental assessment. Elgar, Cheltenham
Florax RJGM, Travisi CM, Nijkamp P (2005) A meta-analysis of the willingness to pay for reductions in pesticide risk exposure. Eur Rev Agric Econ 32: 441–467
Gelman A, Hill J (2007) Data analysis using regression and multilevel/hierarchical models. Cambridge University Press, Cambridge
Ghermandi A, van den Bergh JCJM, Brander LM, de Groot HLF, Nunes PALD (2007) Exploring diversity: a meta-analysis of wetland conservation and creation. Working paper, Free University of Amsterdam
Glass GV (1976) Primary, secondary, and meta-analysis of research. Educ Res 10: 3–8
Goldfarb RS, Stekler HO (2002) Meta-analysis. J Econ Perspect 16: 225–226
Greene WH (2008) Econometric analysis, 6th edn. Pearson Prentice Hall, Upper Saddle River
Hausman JA, Taylor WE (1981) Panel data and unobservable individual effects. Econometrica 49: 1377–1398
Hedges LV (1992) Meta-analysis. J Educ Stat 17: 279–296
Hedges LV (1994) Fixed effects models. In: Cooper H, Hedges LV (eds) The handbook of research synthesis. Russell Sage Foundation, New York, pp 285–299
Hedges LV, Olkin I (1985) Statistical methods for meta-analysis. Academic Press, New York
Higgins JPT, Thompson SG (2002) Quantifying heterogeneity in a meta-analysis. Stat Med 21: 1539–1558
Hoehn JP (2006) Methods to address selection effects in the meta regression and transfer of ecosystem values. Ecol Econ 60: 389–398
Horowitz JK, McConnell KE (2002) A review of WTA/WTP studies. J Environ Econ Manag 44: 426–447
Hox JJ, de Leeuw ED (2003) Multilevel models for meta-analysis. In: Reise SP, Duan N (eds) Multilevel modeling: methodological advances and applications. Lawrence Erlbaum, Mahwah, pp 90–111
Hunter JE, Schmidt FL (2004) Methods of meta-analysis: correcting error and bias in research findings. Sage, Thousand Oaks
Jarrell SB, Stanley TD (1999) A meta-analysis of the union wage gap. Ind Labor Relat Rev 44: 54–67
Jeppesen T, List JA, Folmer H (2002) Environmental regulations and new plant location decisions: evidence from a meta-analysis. J Reg Sci 42: 19–49
Johnson FR, Fries EE, Banzhaf HS (1997) Valuing morbidity: an integration of the willingness-to-pay and health-status index literatures. J Health Econ 16: 641–665
Johnston RJ, Besedin EY, Wardwell RF (2003) Modeling relationships between use and nonuse value for surface water quality: a meta-analysis. Water Resour Res 39: 1363–1371
Johnston RJ, Besedin EY, Iovanna R, Miller CJ, Wardwell RF, Ranson MH (2005) Systematic variation in willingness to pay for aquatic resource improvements and implications for benefit transfer: a meta-analysis. Can J Agric Econ 53: 221–248
Johnston RJ, Ranson MH, Besedin EY, Helm EC (2006) What determines willingness to pay per fish? a meta-analysis of recreational fishing values. Mar Resour Econ 21: 1–32
Kennedy PE (1981) Estimation with correctly interpreted dummy variables in semilogarithmic equations. Am Econ Rev 71: 802
Kennedy PE (2002) Sinning in the basement: what are the rules? the ten commandments of applied econometrics. J Econ Surv 16: 569–589
Kennedy PE (2008) A guide to econometrics, 6th edn. Blackwell, Malden
Kiel KA, Williams M (2007) The impact of Superfund sites on local property values: are all sites the same.J Urban Econ 61: 170–192
Kochi I, Hubbell B, Kramer R (2006) An empirical Bayes approach to combining and comparing estimates of the value of a statistical life for environmental policy analysis. Environ Resource Econ 34: 385–406
Koetse MJ, Florax RJGM, de Groot HLF (2005) Correcting for primary study misspecification in meta-analysis. Tinbergen Institute Discussion Paper TI 2005-029/3, Free University of Amsterdam
Koetse MJ, Florax RJGM, de Groot HLF (2007) The impact of effect size heterogeneity on meta-analysis: a Monte Carlo experiment. Tinbergen Institute Discussion Paper TI 2007-052/3, Free University of Amsterdam
Koetse MJ, de Groot HLF, Florax RJGM (2008) Capital-energy substitution and shifts in factor demand: a meta-analysis. Energy Econ 30: 2236–2257
Lindhjem H, Navrud S (2008) How reliable is meta-analysis for international benefit transfers?. Ecol Econ 66: 425–435
Lipsey MW, Wilson DB (2001) Practical meta-analysis. Sage, Thousands Oaks
Loomis JB, White DS (1996) Economic benefits of rare and endangered species: summary and meta-analysis. Ecol Econ 18: 197–206
Messer KD, Schultze WD, Hackett KF, Cameron TA, McClelland GH (2006) Can stigma explain large property value losses? psychology and economics of superfund. Environ Resource Econ 33: 299–324
Moeltner K, Boyle KJ, Paterson RW (2007) Meta-analysis and benefit transfer for resource valuation–addressing classical challenges with Bayesian modeling. J Environ Econ Manag 53: 250–269
Moulton BR (1990) An illustration of a pitfall in estimating the effect of aggregate variables on microeconomic units. Rev Econ Stat 72: 334–338
Mrozek JR, Taylor LO (2002) What determines the value of life? a meta-analysis. J Policy Anal Manag 21: 253–270
Mulatu A, Florax RJGM, Withagen CA (2003) Environmental regulation and competitiveness: an exploratory meta-analysis. In: Böhringer C, Löschel A (eds) Empirical modeling of the economy and the environment. Physica Verlag, Berlin, pp 23–54
Murphy JJ, Allen PG, Stevens TH, Weatherhead D (2005) A meta-analysis of hypothetical bias in stated preference valuation. Environ Resource Econ 30: 313–325
Nelson JP (2004) Meta-analysis of airport noise and hedonic property values: problems and prospects. J Transp Econ Policy 38: 1–28
Oltmer K, Nijkamp P, Florax R, Brouwer F (2000) A meta-analysis of environmental impacts of agri-environmental policies in the European Union. Tinbergen Institute Discussion Paper TI 2000-058/3, Free University of Amsterdam
Patuelli R, Nijkamp P, Pels E (2005) Environmental tax reform and the double dividend: a meta-analytical performance assessment. Ecol Econ 55: 564–583
Poe GL, Boyle KJ, Bergstrom JC (2001) A preliminary meta analysis of contingent values for ground water quality revisited. In: Bergstrom JC, Boyle KJ, Poe GL (eds) The economic value of water quality. Elgar, Cheltenham, pp 137–162
Quantitative Micro Software (2007) EViews 6.0 User’s Guide, Part II. QMS, Irvine
Rabe-Hesketh S, Skrondal A (2008) Multilevel and longitudinal modeling using Stata, 2nd edn. Stata Press, College Station
Raudenbush SW (1994) Random effects models. In: Cooper H, Hedges LV (eds) The handbook of research synthesis. Russell Sage Foundation, New York, pp 301–321
Raudenbush SW, Bryk AS (2002) Hierarchical linear models: applications and data analysis methods, 2nd edn. Sage, Thousand Oaks
Ringquist EJ (2005) Assessing evidence of environmental inequities: a meta-analysis. J Policy Anal Manag 24: 223–247
Roberts CJ, Stanley TD (2005) Meta-regression analysis: issues of publication bias in economics. Blackwell Publishing, Oxford
Rosenberger RS, Johnston RJ (2008) Selection effects in meta-analysis and benefit transfer: avoiding unintended consequences. Land Econ (in press)
Rosenberger RS, Loomis JB (2000a) Using meta-analysis for benefit transfer: in-sample convergent validity tests of an outdoor recreation database. Water Resour Res 36: 1097–1107
Rosenberger RS, Loomis JB (2000b) Panel stratification in meta-analysis of economic studies: an investigation of its effects in the recreation valuation literature. J Agric Appl Econ 32: 459–470
Rosenberger RS, Stanley TD (2007) Publication bias in the recreation use value literature: a preliminary investigation. Working paper, Oregon State University
Rothstein HR, Sutton AJ, Borenstein M (2005) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, New York
Santos JML (1998) The economic valuation of landscape changes: theory and policies for land use and conservation. Elgar, Cheltenham
Saxonhouse GR (1976) Estimated parameters as dependent variables. Am Econ Rev 66: 178–183
Schulze R (2004) Meta-analysis: a comparison of approaches. Hogrefe & Huber, Cambridge
Smith VK, Huang J-C (1993) Hedonic models and air pollution: twenty-five years and counting. Environ Resource Econ 3: 381–94
Smith VK, Huang J-C (1995) Can markets value air quality? a meta-analysis of hedonic property value models. J Polit Econ 103: 209–227
Smith VK, Kaoru Y (1990a) What have we learned since Hotelling’s letter? a meta-analysis. Econ Lett 32: 267–272
Smith VK, Kaoru Y (1990b) Signals or noise? Explaining the variation in recreation benefit estimates. Am J Agric Econ 72: 419–433
Smith VK, Osborne LL (1996) Do contingent valuation estimates pass a ‘scope’ test? a meta-analysis. J Environ Econ Manag 31: 287–301
Smith VK, Pattanayak SK (2002) Is meta-analysis a Noah’s Ark for non-market valuation?. Environ Resource Econ 22: 271–296
Stanley TD (2001) Wheat from chaff: meta-analysis as quantitative literature review. J Econ Perspect 15:131–150
Stanley TD (2005) Beyond publication bias. J Econ Surv 19: 309–345
Stanley TD (2008) Meta-regression methods for detecting and estimating empirical effects in the presence of publication selection. Oxf Bull Econ Stat 70: 103–127
Stanley TD, Jarrell SB (1989) Meta-regression analysis: a quantitative method of literature surveys. J Econ Surv 3: 161–170
StataCorp (2007) Stata statistical software, release 10, base reference manual, vol 3. Stata Press, College Station
Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F (2000) Methods for meta-analysis in medical research. Wiley, Chichester
U.S. Environmental Protection Agency (2000) Guidelines for preparing economic analyses, EPA 240-R-00-003. EPA, Washington, DC
U.S. Environmental Protection Agency (2006) Report of the EPA work group on VSL meta-analysis, report NCEE-0494. National Center for Environmental Economics, EPA, Washington, DC. http://yosemite.epa.gov/ee/epa/eerm.nsf/vwRepNumLookup/EE-0494?OpenDocument
U.S. Environmental Protection Agency (2007) SAB advisory on EPA’s issues in valuing mortality risk reduction, EPA-SAB-08-001. National Center for Environmental Economics, EPA, Washington, DC
van den Bergh JCJM, Button KJ, Nijkamp P, Pepping GC (1997) Meta-analysis in environmental economics. Kluwer, Dordrecht
Van Garderen KJ, Shah C (2002) Exact interpretation of dummy variables in semilogarithmic equations with estimation uncertainty. Econom J 5: 149–159
Van Houtven G, Powers J, Pattanayak SK (2007) Valuing water quality improvements in the United States using meta-analysis: is the glass half-full or half-empty for national policy analysis. Resour Energy Econ 29: 206–228
van Kooten GC, Eagle AJ, Manley J, Smolak T (2004) How costly are carbon offsets? a meta-analysis of carbon forest sinks. Environ Sci Policy 7: 239–251
van Kooten GC, Laaksonen-Craig S, Wang Y (2007) Costs of creating carbon offsets via forestry activities: a meta- regression analysis. Working paper 2007-2003, University of Victoria
Viscusi WK, Aldy JE (2003) The value of a statistical life: a critical review of market estimates throughout the world. J Risk Uncertain 27: 5–76
Walsh RG, Johnson DM, McKean JR (1989) Issues in nonmarket valuation and policy applications: a retrospective glance. West J Agric Econ 14: 178–188
Walsh RG, Johnson DM, McKean JR (1990) Nonmarket values from two decades of research on recreation demand. In: Link A, Smith VK (eds) Advances in applied microeconomics. JAI Press, Greenwich, pp 167–193
Weitzman ML, Kruse DL (1990) Profit sharing and productivity. In: Blinder A (eds) Paying for productivity: a look at the evidence. Brookings Institution, Washington, DC, pp 95–140
Wood JA (2008) Methodology for dealing with duplicate study effects in a meta-analysis. Organ Res Methods 11: 79–95
Wooldridge JM (2003) Cluster-sample methods in applied econometrics. Am Econ Rev Pap Proc 93: 133–138
Zamparini L, Reggiani A (2007) Meta-analysis and the value of travel time savings: a transatlantic perspective in passenger transport. Netw Spat Econ 7: 377–96
Ziliak ST, McCloskey DN (2008) The cult of statistical significance. University of Michigan Press, Ann Arbor