Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation

Monthly Weather Review - Tập 133 Số 5 - Trang 1098-1118 - 2005
Tilmann Gneiting1, Adrian E. Raftery1, Anton H. Westveld1, Tom Goldman1
1Department of Statistics, University of Washington, Seattle, Washington

Tóm tắt

Abstract Ensemble prediction systems typically show positive spread-error correlation, but they are subject to forecast bias and dispersion errors, and are therefore uncalibrated. This work proposes the use of ensemble model output statistics (EMOS), an easy-to-implement postprocessing technique that addresses both forecast bias and underdispersion and takes into account the spread-skill relationship. The technique is based on multiple linear regression and is akin to the superensemble approach that has traditionally been used for deterministic-style forecasts. The EMOS technique yields probabilistic forecasts that take the form of Gaussian predictive probability density functions (PDFs) for continuous weather variables and can be applied to gridded model output. The EMOS predictive mean is a bias-corrected weighted average of the ensemble member forecasts, with coefficients that can be interpreted in terms of the relative contributions of the member models to the ensemble, and provides a highly competitive deterministic-style forecast. The EMOS predictive variance is a linear function of the ensemble variance. For fitting the EMOS coefficients, the method of minimum continuous ranked probability score (CRPS) estimation is introduced. This technique finds the coefficient values that optimize the CRPS for the training data. The EMOS technique was applied to 48-h forecasts of sea level pressure and surface temperature over the North American Pacific Northwest in spring 2000, using the University of Washington mesoscale ensemble. When compared to the bias-corrected ensemble, deterministic-style EMOS forecasts of sea level pressure had root-mean-square error 9% less and mean absolute error 7% less. The EMOS predictive PDFs were sharp, and much better calibrated than the raw ensemble or the bias-corrected ensemble.

Từ khóa


Tài liệu tham khảo

Anderson, 1996, A method for producing and evaluating probabilistic forecasts from ensemble model integrations., J. Climate, 9, 1518, 10.1175/1520-0442(1996)009<1518:AMFPAE>2.0.CO;2

Atger, 2003, Spatial and interannual variability of the reliability of ensemble-based probabilistic forecasts: Consequences for calibration., Mon. Wea. Rev, 131, 1509, 10.1175//1520-0493(2003)131<1509:SAIVOT>2.0.CO;2

Birgé, 1993, Rates of convergence for minimum contrast estimators., Probab. Theory Rel. Fields, 97, 113, 10.1007/BF01199316

Dawid, 1984, Statistical theory: The prequential approach., J. Roy. Stat. Soc, A147, 278, 10.2307/2981683

Déqué, 1994, Formulation of Gaussian probability forecasts based on model extended-range integrations., Tellus, 46A, 52, 10.3402/tellusa.v46i1.15430

Eckel, F. A. , 2003: Effective mesoscale, short-range ensemble forecasting. Ph.D. dissertation, University of Washington, 224 pp. [Available online at www.atmos.washington.edu/ens/pubs_n_pres.html.].

Eckel, 1998, Calibrated probabilistic quantitative precipitation forecasts based on the MRF ensemble., Wea. Forecasting, 13, 1132, 10.1175/1520-0434(1998)013<1132:CPQPFB>2.0.CO;2

Gel, 2004, Calibrated probabilistic mesoscale weather field forecasting: The geostatistical output perturbation (GOP) method (with discussion and rejoinder)., J. Amer. Stat. Assoc, 99, 575, 10.1198/016214504000000872

Glahn, 1972, The use of model output statistics (MOS) in objective weather forecasting., J. Appl. Meteor, 11, 1203, 10.1175/1520-0450(1972)011<1203:TUOMOS>2.0.CO;2

Gneiting, T., and A. E.Raftery, 2004: Strictly proper scoring rules, prediction, and estimation. Tech. Rep. 463, Department of Statistics, University of Washington, 29 pp. [Available online at www.stat.washington.edu/tech.reports.].

Gneiting, T., A. E.Raftery, F.Balabdaoui, and A.Westveld, 2003: Verifying probabilistic forecasts: Calibration and sharpness. Proc. Workshop on Ensemble Weather Forecasting in the Short to Medium Range, Val-Morin, QC, Canada. [Available online at www.cdc.noaa.gov/people/tom.hamill/ef_workshop_2003_schedule.html.].

Good, 1952, Rational decisions., J. Roy. Stat. Soc, B14, 107

Grimit, 2002, Initial results of a mesoscale short-range ensemble forecasting system over the Pacific Northwest., Wea. Forecasting, 17, 192, 10.1175/1520-0434(2002)017<0192:IROAMS>2.0.CO;2

Grimit, E. P., and C. F.Mass, 2004: Forecasting mesoscale uncertainty: Short-range ensemble forecast error predictability. Preprints, 16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., CD-ROM, 24.3. [Available online at www.atmos.washington.edu/ens/pubs_n_pres.html.].

Hamill, 2001, Interpretation of rank histograms for verifying ensemble forecasts., Mon. Wea. Rev, 129, 550, 10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2

Hamill, 1997, Verification of Eta-RSM short-range ensemble forecasts., Mon. Wea. Rev, 125, 1312, 10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2

Hamill, 1998, Evaluation of Eta-RSM ensemble probabilistic precipitation forecasts., Mon. Wea. Rev, 126, 711, 10.1175/1520-0493(1998)126<0711:EOEREP>2.0.CO;2

Hamill, 2004, Ensemble reforecasting: Improving medium-range forecast skill using retrospective forecasts., Mon. Wea. Rev, 132, 1434, 10.1175/1520-0493(2004)132<1434:ERIMFS>2.0.CO;2

Hersbach, 2000, Decomposition of the continuous ranked probability score for ensemble prediction systems., Wea. Forecasting, 15, 559, 10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2

Houtekamer, 1996, A system simulation approach to ensemble prediction., Mon. Wea. Rev, 124, 1225, 10.1175/1520-0493(1996)124<1225:ASSATE>2.0.CO;2

Huber, 1964, Robust estimation of a location parameter., Ann. Math. Stat, 35, 73, 10.1214/aoms/1177703732

Huber, 1981, Robust Statistics., 10.1002/0471725250

Jewson, 2004, A new parametric model for the assessment and calibration of medium-range ensemble temperature forecasts., Atmos. Sci. Lett, 5, 96, 10.1002/asl.69

Kharin, 2002, Climate predictions with multimodel ensembles., J. Climate, 15, 793, 10.1175/1520-0442(2002)015<0793:CPWME>2.0.CO;2

Krishnamurti, 1999, Improved weather and seasonal climate forecasts from multimodel superensemble., Science, 285, 1548, 10.1126/science.285.5433.1548

Krishnamurti, 2000, Multimodel ensemble forecasts for weather and seasonal climate., J. Climate, 13, 4196, 10.1175/1520-0442(2000)013<4196:MEFFWA>2.0.CO;2

Mass, 2003, IFPS and the future of the National Weather Service., Wea. Forecasting, 18, 75, 10.1175/1520-0434(2003)018<0075:IATFOT>2.0.CO;2

Mass, 2003, Regional environmental prediction over the Pacific Northwest., Bull. Amer. Meteor. Soc, 84, 1353, 10.1175/BAMS-84-10-1353

McCullagh, 1989, Generalized Linear Models., 10.1007/978-1-4899-3242-6

Molteni, 1996, The ECMWF ensemble prediction system: Methodology and validation., Quart. J. Roy. Meteor. Soc, 122, 73, 10.1002/qj.49712252905

Murphy, 1979, Probabilistic temperature forecasts: The case for an operational program., Bull. Amer. Meteor. Soc, 60, 12, 10.1175/1520-0477(1979)060<0012:PTFTCF>2.0.CO;2

Pfanzagl, 1969, On the measurability and consistency of minimum contrast estimates., Metrika, 14, 249, 10.1007/BF02613654

Press, 1992, Numerical Recipes in FORTRAN: The Art of Scientific Computing.

Raftery, 2005, Using a Bayesian model averaging to calibrate forecast ensembles., Mon. Wea. Rev, 133, 1155, 10.1175/MWR2906.1

Richardson, 2001, Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of sample size., Quart. J. Roy. Meteor. Soc, 127, 2473, 10.1002/qj.49712757715

Rosenblatt, 1952, Remarks on a multivariate transformation., Ann. Math. Stat, 23, 470, 10.1214/aoms/1177729394

Roulston, 2002, Evaluating probabilistic forecasts using information theory., Mon. Wea. Rev, 130, 1653, 10.1175/1520-0493(2002)130<1653:EPFUIT>2.0.CO;2

Roulston, 2003, Combining dynamical and statistical ensembles., Tellus, 55A, 16, 10.3402/tellusa.v55i1.12082

Saetra, 2004, Effects of observation errors on the statistics for ensemble spread and reliability., Mon. Wea. Rev, 132, 1487, 10.1175/1520-0493(2004)132<1487:EOOEOT>2.0.CO;2

Scherrer, 2004, Analysis of the spread-skill relations using the ECMWF ensemble prediction system over Europe., Wea. Forecasting, 19, 552, 10.1175/1520-0434(2004)019<0552:AOTSRU>2.0.CO;2

Stefanova, 2002, Interpretation of seasonal climate forecast using Brier score, the Florida State University superensemble, and the AMIP-I dataset., J. Climate, 15, 537, 10.1175/1520-0442(2002)015<0537:IOSCFU>2.0.CO;2

Stensrud, 2003, Short-range predictions of 2-m temperature and dewpoint temperature over New England., Mon. Wea. Rev, 131, 2510, 10.1175/1520-0493(2003)131<2510:SEPOMT>2.0.CO;2

Talagrand, O., R.Vautard, and B.Strauss, 1997: Evaluation of probabilistic prediction systems. Proc. Workshop on Predictability, Reading, United Kingdom, European Centre for Medium-Range Weather Forecasts, 1–25.

Tibshirani, 1996, Regression shrinkage and selection via the lasso., J. Roy. Stat, B58, 267

Toth, 1997, Ensemble forecasting at NCEP and the breeding method., Mon. Wea. Rev, 125, 3297, 10.1175/1520-0493(1997)125<3297:EFANAT>2.0.CO;2

Toth, 2003, Probability and ensemble forecasts.

Unger, D. A. , 1985: A method to estimate the continuous ranked probability score. Preprints, Ninth Conf. on Probability and Statistics in Atmospheric Sciences, Virginia Beach, VA, Amer. Meteor. Soc., 206–213.

Van den Dool, 1994, On the weights for an ensemble-averaged 6–10-day forecast., Wea. Forecasting, 9, 457, 10.1175/1520-0434(1994)009<0457:OTWFAE>2.0.CO;2

Weigend, 2000, Predicting daily probability distributions of S&P500 returns., J. Forecasting, 19, 375, 10.1002/1099-131X(200007)19:4<375::AID-FOR779>3.0.CO;2-U

Whitaker, 1998, The relationship between ensemble spread and ensemble mean skill., Mon. Wea. Rev, 126, 3292, 10.1175/1520-0493(1998)126<3292:TRBESA>2.0.CO;2

Wilks, 1995, Statistical Methods in the Atmospheric Sciences.

Wilks, 2002, Smoothing forecast ensembles with fitted probability distributions., Quart. J. Roy. Meteor. Soc, 128, 2821, 10.1256/qj.01.215

Wilson, 1999, A strategy for verification of weather element forecasts from an ensemble prediction system., Mon. Wea. Rev, 127, 956, 10.1175/1520-0493(1999)127<0956:ASFVOW>2.0.CO;2

Thông tin
Thông tin xuất bản

Monthly Weather Review

Tập 133 Số 5

1098-1118

Thông tin tác giả