TY - JOUR
T1 - “Neural-Gas” Network for Vector Quantization and its Application to Time-Series Prediction
AU - Martinetz, Thomas M.
AU - Berkovich, Stanislav G.
AU - Schulten, Klaus J.
N1 - Funding Information:
Manuscript received November 11, 1991; revised September 13, 1992. This work was supported by the National Science Foundation under Grant DIR-90-15561 and by a Fellowship of the Volkswagen Foundation to T. M. Martinetz. The authors are with the Beckman Institute and the Department of Physics, University of Illinois at Urbana-Champaign, Urbana, IL 61801, IEEE Log Number 9204650.
PY - 1993/7
Y1 - 1993/7
N2 - As a data compression technique, vector quantization requires the minimization of a cost function - the distortion error - which, in general, has many local minima. In this paper, a neural network algorithm based on a 'soft-max' adaptation rule is presented that exhibits good performance in reaching the optimum, or at least coming close. The soft-max rule employed is an extension of the standard K-means clustering procedure and takes into account a 'neighborhood ranking' of the reference (weight) vectors. It is shown that the dynamics of the reference (weight) vectors during the input-driven adaptation procedure 1) is determined by the gradient of an energy function whose shape can be modulated through a neighborhood determining parameter, and 2) resembles the dynamics of Brownian particles moving in a potential determined by the data point density. The network is employed to represent the attractor of the Mackey-Glass equation and to predict the Mackey-Glass time series, with additional local linear mappings for generating output values. The results obtained for the time-series prediction compare very favorably with the results achieved by back-propagation and radial basis function networks.
AB - As a data compression technique, vector quantization requires the minimization of a cost function - the distortion error - which, in general, has many local minima. In this paper, a neural network algorithm based on a 'soft-max' adaptation rule is presented that exhibits good performance in reaching the optimum, or at least coming close. The soft-max rule employed is an extension of the standard K-means clustering procedure and takes into account a 'neighborhood ranking' of the reference (weight) vectors. It is shown that the dynamics of the reference (weight) vectors during the input-driven adaptation procedure 1) is determined by the gradient of an energy function whose shape can be modulated through a neighborhood determining parameter, and 2) resembles the dynamics of Brownian particles moving in a potential determined by the data point density. The network is employed to represent the attractor of the Mackey-Glass equation and to predict the Mackey-Glass time series, with additional local linear mappings for generating output values. The results obtained for the time-series prediction compare very favorably with the results achieved by back-propagation and radial basis function networks.
UR - http://www.scopus.com/inward/record.url?scp=0027632248&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0027632248&partnerID=8YFLogxK
U2 - 10.1109/72.238311
DO - 10.1109/72.238311
M3 - Article
C2 - 18267757
AN - SCOPUS:0027632248
SN - 1045-9227
VL - 4
SP - 558
EP - 569
JO - IEEE Transactions on Neural Networks
JF - IEEE Transactions on Neural Networks
IS - 4
ER -