Compactly encoded optical-flow fields for bidirectional prediction and video coding

We have previously developed an optical-flow-based motion estimator that produces dense, spatially-coherent motion fields under bit-rate constraints. These motion estimates target video coding applications, including post-processing applications in which no additional motion estimation step at the decoder is required. In frame-interpolation applications, significant improvements have been obtained over methods that rely upon the standard block-matching algorithm for motion estimation. We now extend these ideas to the case of video coders that use bidirectionally predicted B frames; the use of B frames provides temporal scalability and good compression performance. We develop a novel scheme to handle the problems caused by the presence of covered/uncovered regions. The scheme uses a label field to optimally weight the contributions of the forward and backward predictions. The label field is dense, with label values in the range (0, 1); we introduce a multiscale algorithm for jointly estimating and compressing the label field. In coding experiments on the Susie sequence, the use of label fields resulted in substantial visual and PSNR gains, especially in the fast moving parts of the sequence.