Universal linear least squares prediction: Upper and lower bounds

Andrew C. Singer, Suleyman S. Kozat, Meir Feder

Research output: Contribution to journalLetterpeer-review


We consider the problem of sequential linear prediction of real-valued sequences under the square-error loss function. For this problem, a prediction algorithm has been demonstrated [1]-[3] whose accumulated squared prediction error, for every bounded sequence, is asymptotically as small as the best fixed linear predictor for that sequence, taken from the class of all linear predictors of a given order p. The redundancy, or excess prediction error above that of the best predictor for that sequence, is upper-bounded by A 2 p ln(n)/n, where n is the data length and the sequence is assumed to be bounded by some A. In this correspondence, we provide an alternative proof of this result by connecting it with universal probability assignment. We then show that this predictor is optimal in a min-max sense, by deriving a corresponding lower bound, such that no sequential predictor can ever do better than a redundancy of A 2p ln(n)/n.

Original languageEnglish (US)
Pages (from-to)2354-2362
Number of pages9
JournalIEEE Transactions on Information Theory
Issue number8
StatePublished - Aug 2002


  • Min-max
  • Prediction
  • Sequential probability assignment
  • Universal algorithms

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences


Dive into the research topics of 'Universal linear least squares prediction: Upper and lower bounds'. Together they form a unique fingerprint.

Cite this