TY - JOUR

T1 - Universal linear least squares prediction

T2 - Upper and lower bounds

AU - Singer, Andrew C.

AU - Kozat, Suleyman S.

AU - Feder, Meir

N1 - Funding Information:
Manuscript received August 24, 2000; revised March 6, 2002. This work was supported in part by the National Science Foundation under Grants CCR-0092598 (CAREER), CCR 99-79381, and ITR 00-85929, and by the Office of Naval Research under Award N000140110117. A. C. Singer and S. S. Kozat are with the Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801 USA (e-mail: acsinger@uiuc.edu; kozat@ifp.uiuc.edu). M. Feder is with the Department of Electrical Engineering–Systems, Tel-Aviv University, Ramat-Aviv, Tel–Aviv 69978, Israel (e-mail: meir@eng.tau.ac.il). Communicated by G. Lugosi, Associate Editor for Nonparametric Estimation, Classification, and Neural Networks. Publisher Item Identifier 10.1109/TIT.2002.800489.

PY - 2002/8

Y1 - 2002/8

N2 - We consider the problem of sequential linear prediction of real-valued sequences under the square-error loss function. For this problem, a prediction algorithm has been demonstrated [1]-[3] whose accumulated squared prediction error, for every bounded sequence, is asymptotically as small as the best fixed linear predictor for that sequence, taken from the class of all linear predictors of a given order p. The redundancy, or excess prediction error above that of the best predictor for that sequence, is upper-bounded by A 2 p ln(n)/n, where n is the data length and the sequence is assumed to be bounded by some A. In this correspondence, we provide an alternative proof of this result by connecting it with universal probability assignment. We then show that this predictor is optimal in a min-max sense, by deriving a corresponding lower bound, such that no sequential predictor can ever do better than a redundancy of A 2p ln(n)/n.

AB - We consider the problem of sequential linear prediction of real-valued sequences under the square-error loss function. For this problem, a prediction algorithm has been demonstrated [1]-[3] whose accumulated squared prediction error, for every bounded sequence, is asymptotically as small as the best fixed linear predictor for that sequence, taken from the class of all linear predictors of a given order p. The redundancy, or excess prediction error above that of the best predictor for that sequence, is upper-bounded by A 2 p ln(n)/n, where n is the data length and the sequence is assumed to be bounded by some A. In this correspondence, we provide an alternative proof of this result by connecting it with universal probability assignment. We then show that this predictor is optimal in a min-max sense, by deriving a corresponding lower bound, such that no sequential predictor can ever do better than a redundancy of A 2p ln(n)/n.

KW - Min-max

KW - Prediction

KW - Sequential probability assignment

KW - Universal algorithms

UR - http://www.scopus.com/inward/record.url?scp=0036672580&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0036672580&partnerID=8YFLogxK

U2 - 10.1109/TIT.2002.800489

DO - 10.1109/TIT.2002.800489

M3 - Letter

AN - SCOPUS:0036672580

SN - 0018-9448

VL - 48

SP - 2354

EP - 2362

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 8

ER -