Work-in-Progress: Algorithms for Canvas-Based Attention Scheduling with Resizing

In real-time machine inference literature, canvas-based attention scheduling was recently introduced as an effective scheduling algorithm for real-time perception pipelines. In this framework, a notion of focus locales is maintained, referring to those locales on which the perception subsystem 'focuses its attention'. Data from these locales (e.g., parts of input video frames corresponding to objects of interest) are packed into smaller bins called canvas frames that are then processed by the AI pipeline. The practice saves resources compared to processing the entirety of the original full frames. While prior work on canvas-based scheduling derived a schedulability bound, their bound applies only if focus locales are quantized into a small set of allowable container sizes for ease of packing into the canvas. In this work, we explore the possibility of removing this limiting assumption thus obviating quantization for a new bound, and generalizing the scheduling policy to allow for object resizing. Experiments on a representative AI-powered embedded platform with a real-world video dataset demonstrate improvements in efficiency in the presence and empirically validate the new bound. The result informs the design and capacity planning of modern real-time machine perception pipelines.