TY - GEN
T1 - Adaptive sequential learning
AU - Wilson, Craig
AU - Veeravalli, Venugopal
N1 - Funding Information:
This work was supported by the NSF under award CCF 1111342 through the University of Illinois at Urbana-Champaign.
Publisher Copyright:
© 2016 IEEE.
PY - 2017/3/1
Y1 - 2017/3/1
N2 - A framework for learning a sequence of slowly changing tasks, where the parameters of the learning algorithm are obtained by minimizing a loss function to a desired accuracy using optimization algorithms such as stochastic gradient descent (SGD) is considered. The tasks change slowly in the sense that the optimum values of the learning algorithm parameters change at a bounded rate. An adaptive sequential learning algorithm is developed to solve such a slowly varying sequence of tasks. The adaptive sequential learning algorithm is extended to handle cross validation and a cost based approach to selecting the number of samples used to compute approximate solutions. Experiments with synthetic and real data are used to validate theoretical results.
AB - A framework for learning a sequence of slowly changing tasks, where the parameters of the learning algorithm are obtained by minimizing a loss function to a desired accuracy using optimization algorithms such as stochastic gradient descent (SGD) is considered. The tasks change slowly in the sense that the optimum values of the learning algorithm parameters change at a bounded rate. An adaptive sequential learning algorithm is developed to solve such a slowly varying sequence of tasks. The adaptive sequential learning algorithm is extended to handle cross validation and a cost based approach to selecting the number of samples used to compute approximate solutions. Experiments with synthetic and real data are used to validate theoretical results.
KW - adaptive algorithms
KW - gradient methods
KW - machine learning
KW - stochastic optimization
UR - http://www.scopus.com/inward/record.url?scp=85016274900&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85016274900&partnerID=8YFLogxK
U2 - 10.1109/ACSSC.2016.7869052
DO - 10.1109/ACSSC.2016.7869052
M3 - Conference contribution
AN - SCOPUS:85016274900
T3 - Conference Record - Asilomar Conference on Signals, Systems and Computers
SP - 326
EP - 330
BT - Conference Record of the 50th Asilomar Conference on Signals, Systems and Computers, ACSSC 2016
A2 - Matthews, Michael B.
PB - IEEE Computer Society
T2 - 50th Asilomar Conference on Signals, Systems and Computers, ACSSC 2016
Y2 - 6 November 2016 through 9 November 2016
ER -