On the Comparison of Interval Forecasts

Ross Askanazi, Francis X. Diebold, Frank Schorfheide, Minchul Shin

Research output: Contribution to journalArticlepeer-review


We explore interval forecast comparison when the nominal confidence level is specified, but the quantiles on which intervals are based are not specified. It turns out that the problem is difficult, and perhaps unsolvable. We first consider a situation where intervals meet the Christoffersen conditions (in particular, where they are correctly calibrated), in which case the common prescription, which we rationalize and explore, is to prefer the interval of shortest length. We then allow for mis-calibrated intervals, in which case there is a calibration-length tradeoff. We propose two natural conditions that interval forecast loss functions should meet in such environments, and we show that a variety of popular approaches to interval forecast comparison fail them. Our negative results strengthen the case for abandoning interval forecasts in favor of density forecasts: Density forecasts not only provide richer information, but also can be readily compared using known proper scoring rules like the log predictive score, whereas interval forecasts cannot.

Original languageEnglish (US)
Pages (from-to)953-965
Number of pages13
JournalJournal of Time Series Analysis
Issue number6
StatePublished - Nov 2018


  • Forecast accuracy
  • forecast evaluation
  • prediction

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty
  • Applied Mathematics


Dive into the research topics of 'On the Comparison of Interval Forecasts'. Together they form a unique fingerprint.

Cite this