Self-Cueing Real-Time Attention Scheduling in Criticality-Aware Visual Machine Perception

Shengzhong Liu, Xinzhe Fu, Maggie Wigness, Philip David, Shuochao Yao, Lui Sha, Tarek Abdelzaher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents a self-cueing real-time frame-work for attention prioritization in AI-enabled visual perception systems that minimizes a notion of state uncertainty. By attention prioritization we refer to inspecting some parts of the scene before others in a criticality-aware fashion. By self-cueing, we refer to not needing external cueing sensors for prioritizing attention, thereby simplifying design. We show that attention prioritization saves resources, thus enabling more efficient and responsive real-time object tracking on resource-limited embedded platforms. The system consists of two components: First, an optical flow-based module decides on the regions to be viewed on a subframe level, as well as their criticality. Second, a novel batched proportional balancing (BPB) scheduling policy decides how to schedule these regions for inspection by a deep neural network (DNN), and how to parallelize execution on the GPU. We implement the system on an NVIDIA Jetson Xavier platform, and empirically demonstrate the superiority of the proposed architecture through an extensive evaluation using a real-word driving dataset.

Original languageEnglish (US)
Title of host publicationProceedings - 28th IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages173-186
Number of pages14
ISBN (Electronic)9781665499989
DOIs
StatePublished - 2022
Event28th IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2022 - Milan, Italy
Duration: May 4 2022May 6 2022

Publication series

NameProceedings of the IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS
Volume2022-May
ISSN (Print)1545-3421

Conference

Conference28th IEEE Real-Time and Embedded Technology and Applications Symposium, RTAS 2022
Country/TerritoryItaly
CityMilan
Period5/4/225/6/22

ASJC Scopus subject areas

  • General Engineering

Fingerprint

Dive into the research topics of 'Self-Cueing Real-Time Attention Scheduling in Criticality-Aware Visual Machine Perception'. Together they form a unique fingerprint.

Cite this