On Exploring Image Resizing for Optimizing Criticality-based Machine Perception

Yigong Hu, Shengzhong Liu, Tarek Abdelzaher, Maggie Wigness, Philip David

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

On-board computing capacity remains a key bottleneck in modern machine inference pipelines that run on embedded hardware, such as aboard autonomous drones or cars. To mitigate this bottleneck, recent work proposed an architecture for segmenting input frames of complex modalities, such as video, and prioritizing downstream machine perception tasks based on criticality of the respective segments of the perceived scene. Criticality-based prioritization allows limited machine resources (of lower-end embedded GPUs) to be spent more judiciously on tracking more important objects first. This paper explores a novel dimension in criticality-based prioritization of machine perception; namely, the role of criticality-dependent image resizing as a way to improve the trade-off between perception quality and timeliness. Given an assessment of criticality (e.g., an object's distance from the autonomous car), the scheduler is allowed to choose from several image resizing options (and related inference models) before passing the resized images to the perception module. Experiments on an AI-powered embedded platform with a real-world driving dataset demonstrate significant improvements in the trade-off between perception accuracy and response time when the proposed resizing algorithm is used. The improvement is attributed to two advantages of the proposed scheme: (i) improved preferential treatment of more critical objects by reducing time spent on less critical ones, and (ii) improved image batching within the GPU, thanks to re-sizing, leading to better resource utilization.

Original languageEnglish (US)
Title of host publicationProceedings - 2021 IEEE 27th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages169-178
Number of pages10
ISBN (Electronic)9781665441889
DOIs
StatePublished - Aug 2021
Event27th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2021 - Virtual, Houston, United States
Duration: Aug 18 2021Aug 20 2021

Publication series

NameProceedings - 2021 IEEE 27th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2021

Conference

Conference27th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2021
Country/TerritoryUnited States
CityVirtual, Houston
Period8/18/218/20/21

Keywords

  • Cyber-Physical Systems
  • Machine Perception
  • Real-Time Scheduling

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Information Systems and Management

Fingerprint

Dive into the research topics of 'On Exploring Image Resizing for Optimizing Criticality-based Machine Perception'. Together they form a unique fingerprint.

Cite this