MLModelScope: Evaluate and introspect cognitive pipelines

Cheng Li, Abdul Dakkak, Jinjun Xiong, Wen-Mei W Hwu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The current landscape of cognitive pipelines exercises many Machine Learning (ML) and Deep Learning (DL) building blocks. These ML and DL building blocks leverage non-uniform frameworks, models, and system stacks. Currently, there is no end-to-end tool that facilitates ML and DL building blocks evaluation and introspection within cognitive pipelines. Due to the absence of such tools, the current practice for evaluating and comparing the benefits of hardware or software innovations on end-to-end cognitive pipelines is both arduous and error-prone - stifling the rate of adoption of innovations. We propose MLModelScope: a hardware/software agnostic platform to facilitate evaluation and introspection of cognitive pipelines in the cloud or on the edge. We describe the design and implementation of MLModelScope and show how it provides a holistic view of the execution of components within cognitive pipelines. MLModelScope aids application developers in experimenting with and discovering cognitive models, data scientists in comparing and evaluating published algorithms, and system architects in optimizing system stacks for cognitive applications.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE World Congress on Services, SERVICES 2019
EditorsCarl K. Chang, Peter Chen, Michael Goul, Katsunori Oyama, Stephan Reiff-Marganiec, Yanchun Sun, Shangguang Wang, Zhongjie Wang
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages335-338
Number of pages4
ISBN (Electronic)9781728138510
DOIs
StatePublished - Jul 2019
Event2019 IEEE World Congress on Services, SERVICES 2019 - Milan, Italy
Duration: Jul 8 2019Jul 13 2019

Publication series

NameProceedings - 2019 IEEE World Congress on Services, SERVICES 2019

Conference

Conference2019 IEEE World Congress on Services, SERVICES 2019
CountryItaly
CityMilan
Period7/8/197/13/19

Fingerprint

Pipelines
Learning systems
Innovation
Hardware
Deep learning
Machine learning
Evaluation
Software

Keywords

  • AI Software
  • Deep Learning
  • Machine Learning
  • Performance Profiling

ASJC Scopus subject areas

  • Hardware and Architecture
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality
  • Management Science and Operations Research
  • Artificial Intelligence
  • Computer Networks and Communications
  • Information Systems

Cite this

Li, C., Dakkak, A., Xiong, J., & Hwu, W-M. W. (2019). MLModelScope: Evaluate and introspect cognitive pipelines. In C. K. Chang, P. Chen, M. Goul, K. Oyama, S. Reiff-Marganiec, Y. Sun, S. Wang, ... Z. Wang (Eds.), Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019 (pp. 335-338). [8817116] (Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SERVICES.2019.00093

MLModelScope : Evaluate and introspect cognitive pipelines. / Li, Cheng; Dakkak, Abdul; Xiong, Jinjun; Hwu, Wen-Mei W.

Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019. ed. / Carl K. Chang; Peter Chen; Michael Goul; Katsunori Oyama; Stephan Reiff-Marganiec; Yanchun Sun; Shangguang Wang; Zhongjie Wang. Institute of Electrical and Electronics Engineers Inc., 2019. p. 335-338 8817116 (Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Li, C, Dakkak, A, Xiong, J & Hwu, W-MW 2019, MLModelScope: Evaluate and introspect cognitive pipelines. in CK Chang, P Chen, M Goul, K Oyama, S Reiff-Marganiec, Y Sun, S Wang & Z Wang (eds), Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019., 8817116, Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019, Institute of Electrical and Electronics Engineers Inc., pp. 335-338, 2019 IEEE World Congress on Services, SERVICES 2019, Milan, Italy, 7/8/19. https://doi.org/10.1109/SERVICES.2019.00093
Li C, Dakkak A, Xiong J, Hwu W-MW. MLModelScope: Evaluate and introspect cognitive pipelines. In Chang CK, Chen P, Goul M, Oyama K, Reiff-Marganiec S, Sun Y, Wang S, Wang Z, editors, Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 335-338. 8817116. (Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019). https://doi.org/10.1109/SERVICES.2019.00093
Li, Cheng ; Dakkak, Abdul ; Xiong, Jinjun ; Hwu, Wen-Mei W. / MLModelScope : Evaluate and introspect cognitive pipelines. Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019. editor / Carl K. Chang ; Peter Chen ; Michael Goul ; Katsunori Oyama ; Stephan Reiff-Marganiec ; Yanchun Sun ; Shangguang Wang ; Zhongjie Wang. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 335-338 (Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019).
@inproceedings{5430ab9d888d492fa765377ebd60a9ac,
title = "MLModelScope: Evaluate and introspect cognitive pipelines",
abstract = "The current landscape of cognitive pipelines exercises many Machine Learning (ML) and Deep Learning (DL) building blocks. These ML and DL building blocks leverage non-uniform frameworks, models, and system stacks. Currently, there is no end-to-end tool that facilitates ML and DL building blocks evaluation and introspection within cognitive pipelines. Due to the absence of such tools, the current practice for evaluating and comparing the benefits of hardware or software innovations on end-to-end cognitive pipelines is both arduous and error-prone - stifling the rate of adoption of innovations. We propose MLModelScope: a hardware/software agnostic platform to facilitate evaluation and introspection of cognitive pipelines in the cloud or on the edge. We describe the design and implementation of MLModelScope and show how it provides a holistic view of the execution of components within cognitive pipelines. MLModelScope aids application developers in experimenting with and discovering cognitive models, data scientists in comparing and evaluating published algorithms, and system architects in optimizing system stacks for cognitive applications.",
keywords = "AI Software, Deep Learning, Machine Learning, Performance Profiling",
author = "Cheng Li and Abdul Dakkak and Jinjun Xiong and Hwu, {Wen-Mei W}",
year = "2019",
month = "7",
doi = "10.1109/SERVICES.2019.00093",
language = "English (US)",
series = "Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "335--338",
editor = "Chang, {Carl K.} and Peter Chen and Michael Goul and Katsunori Oyama and Stephan Reiff-Marganiec and Yanchun Sun and Shangguang Wang and Zhongjie Wang",
booktitle = "Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019",
address = "United States",

}

TY - GEN

T1 - MLModelScope

T2 - Evaluate and introspect cognitive pipelines

AU - Li, Cheng

AU - Dakkak, Abdul

AU - Xiong, Jinjun

AU - Hwu, Wen-Mei W

PY - 2019/7

Y1 - 2019/7

N2 - The current landscape of cognitive pipelines exercises many Machine Learning (ML) and Deep Learning (DL) building blocks. These ML and DL building blocks leverage non-uniform frameworks, models, and system stacks. Currently, there is no end-to-end tool that facilitates ML and DL building blocks evaluation and introspection within cognitive pipelines. Due to the absence of such tools, the current practice for evaluating and comparing the benefits of hardware or software innovations on end-to-end cognitive pipelines is both arduous and error-prone - stifling the rate of adoption of innovations. We propose MLModelScope: a hardware/software agnostic platform to facilitate evaluation and introspection of cognitive pipelines in the cloud or on the edge. We describe the design and implementation of MLModelScope and show how it provides a holistic view of the execution of components within cognitive pipelines. MLModelScope aids application developers in experimenting with and discovering cognitive models, data scientists in comparing and evaluating published algorithms, and system architects in optimizing system stacks for cognitive applications.

AB - The current landscape of cognitive pipelines exercises many Machine Learning (ML) and Deep Learning (DL) building blocks. These ML and DL building blocks leverage non-uniform frameworks, models, and system stacks. Currently, there is no end-to-end tool that facilitates ML and DL building blocks evaluation and introspection within cognitive pipelines. Due to the absence of such tools, the current practice for evaluating and comparing the benefits of hardware or software innovations on end-to-end cognitive pipelines is both arduous and error-prone - stifling the rate of adoption of innovations. We propose MLModelScope: a hardware/software agnostic platform to facilitate evaluation and introspection of cognitive pipelines in the cloud or on the edge. We describe the design and implementation of MLModelScope and show how it provides a holistic view of the execution of components within cognitive pipelines. MLModelScope aids application developers in experimenting with and discovering cognitive models, data scientists in comparing and evaluating published algorithms, and system architects in optimizing system stacks for cognitive applications.

KW - AI Software

KW - Deep Learning

KW - Machine Learning

KW - Performance Profiling

UR - http://www.scopus.com/inward/record.url?scp=85072777402&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072777402&partnerID=8YFLogxK

U2 - 10.1109/SERVICES.2019.00093

DO - 10.1109/SERVICES.2019.00093

M3 - Conference contribution

AN - SCOPUS:85072777402

T3 - Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019

SP - 335

EP - 338

BT - Proceedings - 2019 IEEE World Congress on Services, SERVICES 2019

A2 - Chang, Carl K.

A2 - Chen, Peter

A2 - Goul, Michael

A2 - Oyama, Katsunori

A2 - Reiff-Marganiec, Stephan

A2 - Sun, Yanchun

A2 - Wang, Shangguang

A2 - Wang, Zhongjie

PB - Institute of Electrical and Electronics Engineers Inc.

ER -