MLModelScope: Evaluate and introspect cognitive pipelines

Cheng Li, Abdul Dakkak, Jinjun Xiong, Wen Mei Hwu

Research output: Chapter in Book/Report/Conference proceedingConference contribution


The current landscape of cognitive pipelines exercises many Machine Learning (ML) and Deep Learning (DL) building blocks. These ML and DL building blocks leverage non-uniform frameworks, models, and system stacks. Currently, there is no end-to-end tool that facilitates ML and DL building blocks evaluation and introspection within cognitive pipelines. Due to the absence of such tools, the current practice for evaluating and comparing the benefits of hardware or software innovations on end-to-end cognitive pipelines is both arduous and error-prone - stifling the rate of adoption of innovations. We propose MLModelScope: a hardware/software agnostic platform to facilitate evaluation and introspection of cognitive pipelines in the cloud or on the edge. We describe the design and implementation of MLModelScope and show how it provides a holistic view of the execution of components within cognitive pipelines. MLModelScope aids application developers in experimenting with and discovering cognitive models, data scientists in comparing and evaluating published algorithms, and system architects in optimizing system stacks for cognitive applications.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE World Congress on Services, SERVICES 2019
EditorsCarl K. Chang, Peter Chen, Michael Goul, Katsunori Oyama, Stephan Reiff-Marganiec, Yanchun Sun, Shangguang Wang, Zhongjie Wang
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages4
ISBN (Electronic)9781728138510
StatePublished - Jul 2019
Event2019 IEEE World Congress on Services, SERVICES 2019 - Milan, Italy
Duration: Jul 8 2019Jul 13 2019

Publication series

NameProceedings - 2019 IEEE World Congress on Services, SERVICES 2019


Conference2019 IEEE World Congress on Services, SERVICES 2019


  • AI Software
  • Deep Learning
  • Machine Learning
  • Performance Profiling

ASJC Scopus subject areas

  • Hardware and Architecture
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality
  • Management Science and Operations Research
  • Artificial Intelligence
  • Computer Networks and Communications
  • Information Systems


Dive into the research topics of 'MLModelScope: Evaluate and introspect cognitive pipelines'. Together they form a unique fingerprint.

Cite this