Underprovisioned GPUs: On Sufficient Capacity for Real-Time Mission-Critical Perception

Recent work suggests that computing resources, such as GPUs in real-time edge-based perception systems, need not have sufficient capacity to keep up with the input frame rates of all input devices (e.g., cameras) at their full-frame resolution. Rather, they can be under-provisioned because only parts of any given frame need to be inspected (i.e., paid attention to). This paper derives an attention allocation policy, called canvas-based attention scheduling that decides which parts of each frame of each device to inspect, and a corresponding schedulability condition that relates the spatiotemporal properties of surrounding objects to the ability of the edge-based perception subsystem to keep up with the state of the environment in real-time. It provides a quantitative estimate of adequate computing capacity for the expected perception workload. We implement a canvas-based attention scheduler for an object detection application and perform an empirical comparative study based on actual GPU hardware and surveillance videos. Results show that canvas-based attention scheduling keeps up with the environment while using a much smaller GPU capacity, compared with prior approaches.