Real-Time Task Scheduling for Machine Perception in In Intelligent Cyber-Physical Systems

Shengzhong Liu, Shuochao Yao, Xinzhe Fu, Huajie Shao, Rohan Tabish, Simon Yu, Ayoosh Bansal, Heechul Yun, Lui Sha, Tarek Abdelzaher

Research output: Contribution to journalArticlepeer-review


This paper explores criticality-based real-time scheduling of neural-network-based machine inference pipelines in cyber-physical systems (CPS) to mitigate the effect of algorithmic priority inversion. We specifically focus on the perception subsystem, an important subsystem feeding other components. In current machine perception software, significant priority inversion occurs because resource allocation to the underlying neural network models does not differentiate between critical and less critical data within a scene. To remedy this problem, in recent work, we proposed an architecture to partition the input data into regions of different criticality, then formulated a utility-based optimization problem to batch and schedule their processing in a manner that maximizes confidence in perception results, subject to criticality-based time constraints. This journal extension matures the work in several directions: (i) We extend confidence maximization to a generalized utility optimization formulation that accounts for criticality in the utility function itself, offering finer-grained control over resource allocation within the perception pipeline; (ii) we further instantiate two different criticality metrics to understand their relative advantages; and (iii) we explore the limitations of the approach, specifically how inaccuracies in criticality-based attention cueing affect performance. All experiments are conducted on the NVIDIA Jetson AGX Xavier platform with a real-world driving dataset.

Original languageEnglish (US)
Pages (from-to)1770-1783
Number of pages14
JournalIEEE Transactions on Computers
Issue number8
StateAccepted/In press - 2021


  • Algorithmic Priority Inversion
  • Computer architecture
  • Cyber-physical systems
  • Cyber-Physical Systems (CPS)
  • Distance measurement
  • Machine Intelligence
  • Neural networks
  • Pipelines
  • Real-Time Scheduling
  • Real-time systems
  • Resource management

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture
  • Computational Theory and Mathematics


Dive into the research topics of 'Real-Time Task Scheduling for Machine Perception in In Intelligent Cyber-Physical Systems'. Together they form a unique fingerprint.

Cite this