Causal coding of markov sources with continuous alphabets

We study, for kth-order Markov sources, the structure of optimal causal encoders minimizing total rate subject to a mean-square distortion constraint over a finite horizon. The class of sources considered is general alphabet sources, and the encoder is allowed to be variable-rate. This leads to an optimization problem in an infinite-dimensional space, for which a solution exists. We show that the optimal causal encoder for a kth-order Markov source uses only the most recent k source symbols and the information available at the receiver. We also consider the infinite-horizon version, but for k=1, provide an existence result for an optimal stationary solution, discuss the effect of randomization on performance, and show that a recurrence-based time sharing of at most two deterministic quantizers is optimal. We further argue that for real-valued processes innovation encoding is not in general optimal, but for coding of a stable linear or non-linear source, the quantization of the innovation is an almost optimal scheme in the limit of low-distortion. This class of problems has natural applications in remote control of linear and non-linear systems with quantization.