TY - GEN

T1 - Learning from compressed observations

AU - Raginsky, Maxim

N1 - Copyright:
Copyright 2011 Elsevier B.V., All rights reserved.

PY - 2007

Y1 - 2007

N2 - The problem of statistical learning is to construct a predictor of a random variable Y as a function of a related random variable X on the basis of an i.i.d. training sample from the joint distribution of (X, Y). Allowable predictors are drawn from some specified class, and the goal is to approach asymptotically the performance (expected loss) of the best predictor in the class. We consider the setting in which one has perfect observation of the X-part of the sample, while the Y-part has to be communicated at some finite bit rate. The encoding of the Y-values is allowed to depend on the X-values. Under suitable regularity conditions on the admissible predictors, the underlying family of probability distributions and the loss function, we give an information-theoretic characterization of achievable predictor performance in terms of conditional distortion-rate functions. The ideas are illustrated on the example of nonparametric regression in Gaussian noise.

AB - The problem of statistical learning is to construct a predictor of a random variable Y as a function of a related random variable X on the basis of an i.i.d. training sample from the joint distribution of (X, Y). Allowable predictors are drawn from some specified class, and the goal is to approach asymptotically the performance (expected loss) of the best predictor in the class. We consider the setting in which one has perfect observation of the X-part of the sample, while the Y-part has to be communicated at some finite bit rate. The encoding of the Y-values is allowed to depend on the X-values. Under suitable regularity conditions on the admissible predictors, the underlying family of probability distributions and the loss function, we give an information-theoretic characterization of achievable predictor performance in terms of conditional distortion-rate functions. The ideas are illustrated on the example of nonparametric regression in Gaussian noise.

UR - http://www.scopus.com/inward/record.url?scp=46749098369&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=46749098369&partnerID=8YFLogxK

U2 - 10.1109/ITW.2007.4313111

DO - 10.1109/ITW.2007.4313111

M3 - Conference contribution

AN - SCOPUS:46749098369

SN - 1424415640

SN - 9781424415649

T3 - 2007 IEEE Information Theory Workshop, ITW 2007, Proceedings

SP - 420

EP - 425

BT - 2007 IEEE Information Theory Workshop, ITW 2007, Proceedings

T2 - 2007 IEEE Information Theory Workshop, ITW 2007

Y2 - 2 September 2007 through 6 September 2007

ER -