TY - GEN
T1 - On Exploring Image Resizing for Optimizing Criticality-based Machine Perception
AU - Hu, Yigong
AU - Liu, Shengzhong
AU - Abdelzaher, Tarek
AU - Wigness, Maggie
AU - David, Philip
N1 - Funding Information:
This research was sponsored in part by DARPA award W911NF-17-C-0099 and the Army Research Laboratory under Cooperative Agreements W911NF-17-2-0196.
Publisher Copyright:
© 2021 IEEE.
PY - 2021/8
Y1 - 2021/8
N2 - On-board computing capacity remains a key bottleneck in modern machine inference pipelines that run on embedded hardware, such as aboard autonomous drones or cars. To mitigate this bottleneck, recent work proposed an architecture for segmenting input frames of complex modalities, such as video, and prioritizing downstream machine perception tasks based on criticality of the respective segments of the perceived scene. Criticality-based prioritization allows limited machine resources (of lower-end embedded GPUs) to be spent more judiciously on tracking more important objects first. This paper explores a novel dimension in criticality-based prioritization of machine perception; namely, the role of criticality-dependent image resizing as a way to improve the trade-off between perception quality and timeliness. Given an assessment of criticality (e.g., an object's distance from the autonomous car), the scheduler is allowed to choose from several image resizing options (and related inference models) before passing the resized images to the perception module. Experiments on an AI-powered embedded platform with a real-world driving dataset demonstrate significant improvements in the trade-off between perception accuracy and response time when the proposed resizing algorithm is used. The improvement is attributed to two advantages of the proposed scheme: (i) improved preferential treatment of more critical objects by reducing time spent on less critical ones, and (ii) improved image batching within the GPU, thanks to re-sizing, leading to better resource utilization.
AB - On-board computing capacity remains a key bottleneck in modern machine inference pipelines that run on embedded hardware, such as aboard autonomous drones or cars. To mitigate this bottleneck, recent work proposed an architecture for segmenting input frames of complex modalities, such as video, and prioritizing downstream machine perception tasks based on criticality of the respective segments of the perceived scene. Criticality-based prioritization allows limited machine resources (of lower-end embedded GPUs) to be spent more judiciously on tracking more important objects first. This paper explores a novel dimension in criticality-based prioritization of machine perception; namely, the role of criticality-dependent image resizing as a way to improve the trade-off between perception quality and timeliness. Given an assessment of criticality (e.g., an object's distance from the autonomous car), the scheduler is allowed to choose from several image resizing options (and related inference models) before passing the resized images to the perception module. Experiments on an AI-powered embedded platform with a real-world driving dataset demonstrate significant improvements in the trade-off between perception accuracy and response time when the proposed resizing algorithm is used. The improvement is attributed to two advantages of the proposed scheme: (i) improved preferential treatment of more critical objects by reducing time spent on less critical ones, and (ii) improved image batching within the GPU, thanks to re-sizing, leading to better resource utilization.
KW - Cyber-Physical Systems
KW - Machine Perception
KW - Real-Time Scheduling
UR - http://www.scopus.com/inward/record.url?scp=85116623726&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85116623726&partnerID=8YFLogxK
U2 - 10.1109/RTCSA52859.2021.00027
DO - 10.1109/RTCSA52859.2021.00027
M3 - Conference contribution
AN - SCOPUS:85116623726
T3 - Proceedings - 2021 IEEE 27th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2021
SP - 169
EP - 178
BT - Proceedings - 2021 IEEE 27th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 27th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2021
Y2 - 18 August 2021 through 20 August 2021
ER -