Using Bayesian Model Averaging to Calibrate Forecast Ensembles
Tóm tắt
Ensembles used for probabilistic weather forecasting often exhibit a spread-error correlation, but they tend to be underdispersive. This paper proposes a statistical method for postprocessing ensembles based on Bayesian model averaging (BMA), which is a standard method for combining predictive distributions from different sources. The BMA predictive probability density function (PDF) of any quantity of interest is a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights are equal to posterior probabilities of the models generating the forecasts and reflect the models' relative contributions to predictive skill over the training period. The BMA weights can be used to assess the usefulness of ensemble members, and this can be used as a basis for selecting ensemble members; this can be useful given the cost of running large ensembles. The BMA PDF can be represented as an unweighted ensemble of any desired size, by simulating from the BMA predictive distribution.
The BMA predictive variance can be decomposed into two components, one corresponding to the between-forecast variability, and the second to the within-forecast variability. Predictive PDFs or intervals based solely on the ensemble spread incorporate the first component but not the second. Thus BMA provides a theoretical explanation of the tendency of ensembles to exhibit a spread-error correlation but yet be underdispersive.
The method was applied to 48-h forecasts of surface temperature in the Pacific Northwest in January–June 2000 using the University of Washington fifth-generation Pennsylvania State University–NCAR Mesoscale Model (MM5) ensemble. The predictive PDFs were much better calibrated than the raw ensemble, and the BMA forecasts were sharp in that 90% BMA prediction intervals were 66% shorter on average than those produced by sample climatology. As a by-product, BMA yields a deterministic point forecast, and this had root-mean-square errors 7% lower than the best of the ensemble members and 8% lower than the ensemble mean. Similar results were obtained for forecasts of sea level pressure. Simulation experiments show that BMA performs reasonably well when the underlying ensemble is calibrated, or even overdispersed.
Từ khóa
Tài liệu tham khảo
Anderson, 1996, A method for producing and evaluating probabilistic forecasts from ensemble model integrations., J. Climate, 9, 1518, 10.1175/1520-0442(1996)009<1518:AMFPAE>2.0.CO;2
Buizza, 1997, Potential forecast skill of ensemble prediction and spread and skill distributions of the ECMWF ensemble prediction system., Mon. Wea. Rev, 125, 99, 10.1175/1520-0493(1997)125<0099:PFSOEP>2.0.CO;2
Buizza, 1999, Stochastic representation of model uncertainties in the ECMWF ensemble prediction system., Quart. J. Roy. Meteor. Soc, 125, 2887, 10.1002/qj.49712556006
Buizza, 2005, A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems., Mon. Wea. Rev, 133, 1076, 10.1175/MWR2905.1
Carter, 1989, Statistical forecasts based on the National Meteorological Center's numerical weather prediction system., Wea. Forecasting, 4, 401, 10.1175/1520-0434(1989)004<0401:SFBOTN>2.0.CO;2
Casella, 2001, Statistical Inference.
Dempster, 1977, Maximum likelihood from incomplete data via the EM algorithm., J. Roy. Stat. Soc, 39B, 1
Eckel, 1998, Calibrated probabilistic quantitative precipitation forecasts based on the MRF ensemble., Wea. Forecasting, 13, 1132, 10.1175/1520-0434(1998)013<1132:CPQPFB>2.0.CO;2
Eckel, 2003, Towards an effective short-range ensemble forecast system.
Fisher, 1922, On the mathematical foundations of theoretical statistics., Philos. Trans. Roy. Soc. London, 222A, 309
Gel, 2004, Calibrated probabilistic mesoscale weather field forecasting: The geostatistical output perturbation (GOP) method (with discussion)., J. Amer. Stat. Assoc, 99, 575, 10.1198/016214504000000872
Glahn, 1972, The use of model output statistics (MOS) in objective weather forecasting., J. Appl. Meteor, 11, 1202, 10.1175/1520-0450(1972)011<1203:TUOMOS>2.0.CO;2
Gneiting, 2003, Verifying probabilistic forecasts: Calibration and sharpness.
Good, 1952, Rational decisions., J. Roy. Stat. Soc, 14, 107
Grimit, 2002, Initial results of a mesoscale short-range ensemble forecasting system over the Pacific Northwest., Wea. Forecasting, 17, 192, 10.1175/1520-0434(2002)017<0192:IROAMS>2.0.CO;2
Hamill, 2001, Interpretation of rank histograms for verifying ensemble forecasts., Mon. Wea. Rev, 129, 550, 10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2
Hamill, 1997, Verification of Eta–RSM short-range ensemble forecasts., Mon. Wea. Rev, 125, 1312, 10.1175/1520-0493(1997)125<1312:VOERSR>2.0.CO;2
Hamill, 1998, Evaluation of Eta–RSM ensemble probabilistic precipitation forecasts., Mon. Wea. Rev, 126, 711, 10.1175/1520-0493(1998)126<0711:EOEREP>2.0.CO;2
Hamill, 2000, A comparison of probabilistic forecasts from bred, singular-vector, and perturbed observation ensembles., Mon. Wea. Rev, 128, 1835, 10.1175/1520-0493(2000)128<1835:ACOPFF>2.0.CO;2
Hamill, 2004, Ensemble re-forecasting: Improving medium-range forecast skill using retrospective forecasts., Mon. Wea. Rev, 132, 1434, 10.1175/1520-0493(2004)132<1434:ERIMFS>2.0.CO;2
Hersbach, 2000, Decomposition of the continuous ranked probability score for ensemble prediction systems., Wea. Forecasting, 15, 559, 10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2
Hersbach, 2000, A short-range to early-medium-range ensemble prediction system for the European area., Mon. Wea. Rev, 128, 3501, 10.1175/1520-0493(2000)128<3501:ASRTEM>2.0.CO;2
Hoeting, 1999, Bayesian model averaging: A tutorial (with discussion)., Stat. Sci, 14, 382
Hou, 2001, Objective verification of the SAMEX'98 ensemble forecast., Mon. Wea. Rev, 129, 73, 10.1175/1520-0493(2001)129<0073:OVOTSE>2.0.CO;2
Houtekamer, 1993, Global and local skill forecasts., Mon. Wea. Rev, 121, 1834, 10.1175/1520-0493(1993)121<1834:GALSF>2.0.CO;2
Houtekamer, 1995, Methods for ensemble prediction., Mon. Wea. Rev, 123, 2181, 10.1175/1520-0493(1995)123<2181:MFEP>2.0.CO;2
Houtekamer, 1998, Data assimilation using an ensemble Kalman filter technique., Mon. Wea. Rev, 126, 796, 10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2
Houtekamer, 2001, A sequential ensemble Kalman filter for atmospheric data assimilation., Mon. Wea. Rev, 129, 123, 10.1175/1520-0493(2001)129<0123:ASEKFF>2.0.CO;2
Kharin, 2002, Climate predictions with multimodel ensembles., J. Climate, 15, 793, 10.1175/1520-0442(2002)015<0793:CPWME>2.0.CO;2
Krishnamurti, 1999, Improved weather and seasonal climate forecasts from multimodel superensembles., Science, 258, 1548, 10.1126/science.285.5433.1548
Leamer, 1978, Specification Searches.
Leith, 1974, Theoretical skill of Monte-Carlo forecasts., Mon. Wea. Rev, 102, 409, 10.1175/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2
McLachlan, 1997, The EM Algorithm and Extensions.
Molteni, 1996, The ECMWF ensemble system: Methodology and validation., Quart. J. Roy. Meteor. Soc, 122, 73, 10.1002/qj.49712252905
Pellerin, 2003, Increasing the horizontal resolution of ensemble forecasts at CMC., Nonlinear Processes Geophys, 10, 463, 10.5194/npg-10-463-2003
Raftery, 1993, Bayesian model selection in structural equation models.
Raftery, 2003, Long-run performance of Bayesian model averaging., J. Amer. Stat. Assoc, 98, 931, 10.1198/016214503000000891
Roulston, 2002, Evaluating probabilistic forecasts using information theory., Mon. Wea. Rev, 130, 1653, 10.1175/1520-0493(2002)130<1653:EPFUIT>2.0.CO;2
Roulston, 2003, Combining dynamical and statistical ensembles., Tellus, 55A, 16, 10.3402/tellusa.v55i1.12082
Scherrer, 2004, Analysis of the spread-skill relations using the ECMWF ensemble prediction system over Europe., Wea. Forecasting, 19, 552, 10.1175/1520-0434(2004)019<0552:AOTSRU>2.0.CO;2
Stefanova, 2002, Interpretation of seasonal climate forecast using Brier skill score, the Florida State University superensemble, and the AMIP-I dataset., J. Climate, 15, 537, 10.1175/1520-0442(2002)015<0537:IOSCFU>2.0.CO;2
Stensrud, 1999, Using ensembles for short-range forecasting., Mon. Wea. Rev, 127, 433, 10.1175/1520-0493(1999)127<0433:UEFSRF>2.0.CO;2
Stern, 1984, A model fitting analysis of daily rainfall data (with discussion)., J. Roy. Stat. Soc, 147A, 1
Talagrand, O., R.Vautard, and B.Strauss, 1997: Evaluation of probabilistic prediction systems. Proc. ECMWF Workshop on Predictability, Reading, United Kingdom, ECMWF, 1–25. [Available from ECMWF, Shinfield Park, Reading, Berkshire RG2 9AX, United Kingdom.].
Toth, 1993, Ensemble forecasting at the NMC: The generation of perturbations., Bull. Amer. Meteor. Soc, 74, 2317, 10.1175/1520-0477(1993)074<2317:EFANTG>2.0.CO;2
Toth, 2001, The use of ensembles to identify forecasts with small and large uncertainty., Wea. Forecasting, 16, 463, 10.1175/1520-0434(2001)016<0463:TUOETI>2.0.CO;2
Van den Dool, 1994, On the weights for an ensemble-averaged 6–10-day forecast., Wea. Forecasting, 9, 457, 10.1175/1520-0434(1994)009<0457:OTWFAE>2.0.CO;2
Wilks, 1995, Statistical Methods in the Atmospheric Sciences.
Wilks, 2002, Smoothing forecast ensembles with fitted probability distributions., Quart. J. Roy. Meteor. Soc, 128, 2821, 10.1256/qj.01.215