Large-Scale System Monitoring Experiences and Recommendations

Ville Ahlgren; Stefan Andersson; Jim Brandt; Nicholas Cardo; Sudheer Chunduri; Jeremy James Enos; Parks Fields; Ann Gentile; Richard Gerber; Michael Gienger; Joe Greenseid; Annette Greiner; Bilel Hadri; Yun He; Dennis Hoppe; Urpo Kaila; Kaki Kelly; Mark Klein; Alex Kristiansen; Steve Leak; Mike Mason; Kevin Pedretti; Jean Guillaume Piccinali; Jason Repik; Jim Rogers; Susanna Salminen; Mike Showerman; Cary Whitney; Jim Williams

doi:10.1109/CLUSTER.2018.00069

Large-Scale System Monitoring Experiences and Recommendations

Ville Ahlgren, Stefan Andersson, Jim Brandt, Nicholas Cardo, Sudheer Chunduri, Jeremy James Enos, Parks Fields, Ann Gentile, Richard Gerber, Michael Gienger, Joe Greenseid, Annette Greiner, Bilel Hadri, Yun He, Dennis Hoppe, Urpo Kaila, Kaki Kelly, Mark Klein, Alex Kristiansen, Steve LeakMike Mason, Kevin Pedretti, Jean Guillaume Piccinali, Jason Repik, Jim Rogers, Susanna Salminen, Mike Showerman, Cary Whitney, Jim Williams

National Center for Supercomputing Applications (NCSA)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Monitoring of High Performance Computing (HPC) platforms is critical to successful operations, can provide insights into performance-impacting conditions, and can inform methodologies for improving science throughput. However, monitoring systems are not generally considered core capabilities in system requirements specifications nor in vendor development strategies. In this paper we present work performed at a number of large-scale HPC sites towards developing monitoring capabilities that fill current gaps in ease of problem identification and root cause discovery. We also present our collective views, based on the experiences presented, on needs and requirements for enabling development by vendors or users of effective sharable end-to-end monitoring capabilities.

Original language	English (US)
Title of host publication	Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	532-542
Number of pages	11
ISBN (Electronic)	9781538683194
DOIs	https://doi.org/10.1109/CLUSTER.2018.00069
State	Published - Oct 29 2018
Event	2018 IEEE International Conference on Cluster Computing, CLUSTER 2018 - Belfast, United Kingdom Duration: Sep 10 2018 → Sep 13 2018

Publication series

Name	Proceedings - IEEE International Conference on Cluster Computing, ICCC
Volume	2018-September
ISSN (Print)	1552-5244

Other

Other	2018 IEEE International Conference on Cluster Computing, CLUSTER 2018
Country/Territory	United Kingdom
City	Belfast
Period	9/10/18 → 9/13/18

Keywords

HPC monitoring
Monitoring architecture
System administration

ASJC Scopus subject areas

Software
Hardware and Architecture
Signal Processing

Online availability

10.1109/CLUSTER.2018.00069

Library availability

Discover UIUC Full Text

Cite this

Ahlgren, V., Andersson, S., Brandt, J., Cardo, N., Chunduri, S., Enos, J. J., Fields, P., Gentile, A., Gerber, R., Gienger, M., Greenseid, J., Greiner, A., Hadri, B., He, Y., Hoppe, D., Kaila, U., Kelly, K., Klein, M., Kristiansen, A., ... Williams, J. (2018). Large-Scale System Monitoring Experiences and Recommendations. In Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018 (pp. 532-542). Article 8514913 (Proceedings - IEEE International Conference on Cluster Computing, ICCC; Vol. 2018-September). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CLUSTER.2018.00069

Large-Scale System Monitoring Experiences and Recommendations. / Ahlgren, Ville; Andersson, Stefan; Brandt, Jim et al.
Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 532-542 8514913 (Proceedings - IEEE International Conference on Cluster Computing, ICCC; Vol. 2018-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ahlgren, V, Andersson, S, Brandt, J, Cardo, N, Chunduri, S, Enos, JJ, Fields, P, Gentile, A, Gerber, R, Gienger, M, Greenseid, J, Greiner, A, Hadri, B, He, Y, Hoppe, D, Kaila, U, Kelly, K, Klein, M, Kristiansen, A, Leak, S, Mason, M, Pedretti, K, Piccinali, JG, Repik, J, Rogers, J, Salminen, S, Showerman, M, Whitney, C & Williams, J 2018, Large-Scale System Monitoring Experiences and Recommendations. in Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018., 8514913, Proceedings - IEEE International Conference on Cluster Computing, ICCC, vol. 2018-September, Institute of Electrical and Electronics Engineers Inc., pp. 532-542, 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018, Belfast, United Kingdom, 9/10/18. https://doi.org/10.1109/CLUSTER.2018.00069

Ahlgren V, Andersson S, Brandt J, Cardo N, Chunduri S, Enos JJ et al. Large-Scale System Monitoring Experiences and Recommendations. In Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 532-542. 8514913. (Proceedings - IEEE International Conference on Cluster Computing, ICCC). doi: 10.1109/CLUSTER.2018.00069

@inproceedings{e910a071d88f46dda630a77b4838ab18,

title = "Large-Scale System Monitoring Experiences and Recommendations",

abstract = "Monitoring of High Performance Computing (HPC) platforms is critical to successful operations, can provide insights into performance-impacting conditions, and can inform methodologies for improving science throughput. However, monitoring systems are not generally considered core capabilities in system requirements specifications nor in vendor development strategies. In this paper we present work performed at a number of large-scale HPC sites towards developing monitoring capabilities that fill current gaps in ease of problem identification and root cause discovery. We also present our collective views, based on the experiences presented, on needs and requirements for enabling development by vendors or users of effective sharable end-to-end monitoring capabilities.",

keywords = "HPC monitoring, Monitoring architecture, System administration",

author = "Ville Ahlgren and Stefan Andersson and Jim Brandt and Nicholas Cardo and Sudheer Chunduri and Enos, {Jeremy James} and Parks Fields and Ann Gentile and Richard Gerber and Michael Gienger and Joe Greenseid and Annette Greiner and Bilel Hadri and Yun He and Dennis Hoppe and Urpo Kaila and Kaki Kelly and Mark Klein and Alex Kristiansen and Steve Leak and Mike Mason and Kevin Pedretti and Piccinali, {Jean Guillaume} and Jason Repik and Jim Rogers and Susanna Salminen and Mike Showerman and Cary Whitney and Jim Williams",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018 ; Conference date: 10-09-2018 Through 13-09-2018",

year = "2018",

month = oct,

day = "29",

doi = "10.1109/CLUSTER.2018.00069",

language = "English (US)",

series = "Proceedings - IEEE International Conference on Cluster Computing, ICCC",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "532--542",

booktitle = "Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018",

address = "United States",

}

TY - GEN

T1 - Large-Scale System Monitoring Experiences and Recommendations

AU - Ahlgren, Ville

AU - Andersson, Stefan

AU - Brandt, Jim

AU - Cardo, Nicholas

AU - Chunduri, Sudheer

AU - Enos, Jeremy James

AU - Fields, Parks

AU - Gentile, Ann

AU - Gerber, Richard

AU - Gienger, Michael

AU - Greenseid, Joe

AU - Greiner, Annette

AU - Hadri, Bilel

AU - He, Yun

AU - Hoppe, Dennis

AU - Kaila, Urpo

AU - Kelly, Kaki

AU - Klein, Mark

AU - Kristiansen, Alex

AU - Leak, Steve

AU - Mason, Mike

AU - Pedretti, Kevin

AU - Piccinali, Jean Guillaume

AU - Repik, Jason

AU - Rogers, Jim

AU - Salminen, Susanna

AU - Showerman, Mike

AU - Whitney, Cary

AU - Williams, Jim

PY - 2018/10/29

Y1 - 2018/10/29

N2 - Monitoring of High Performance Computing (HPC) platforms is critical to successful operations, can provide insights into performance-impacting conditions, and can inform methodologies for improving science throughput. However, monitoring systems are not generally considered core capabilities in system requirements specifications nor in vendor development strategies. In this paper we present work performed at a number of large-scale HPC sites towards developing monitoring capabilities that fill current gaps in ease of problem identification and root cause discovery. We also present our collective views, based on the experiences presented, on needs and requirements for enabling development by vendors or users of effective sharable end-to-end monitoring capabilities.

AB - Monitoring of High Performance Computing (HPC) platforms is critical to successful operations, can provide insights into performance-impacting conditions, and can inform methodologies for improving science throughput. However, monitoring systems are not generally considered core capabilities in system requirements specifications nor in vendor development strategies. In this paper we present work performed at a number of large-scale HPC sites towards developing monitoring capabilities that fill current gaps in ease of problem identification and root cause discovery. We also present our collective views, based on the experiences presented, on needs and requirements for enabling development by vendors or users of effective sharable end-to-end monitoring capabilities.

KW - HPC monitoring

KW - Monitoring architecture

KW - System administration

UR - http://www.scopus.com/inward/record.url?scp=85057218940&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057218940&partnerID=8YFLogxK

U2 - 10.1109/CLUSTER.2018.00069

DO - 10.1109/CLUSTER.2018.00069

M3 - Conference contribution

AN - SCOPUS:85057218940

T3 - Proceedings - IEEE International Conference on Cluster Computing, ICCC

SP - 532

EP - 542

BT - Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018

Y2 - 10 September 2018 through 13 September 2018

ER -

Large-Scale System Monitoring Experiences and Recommendations

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Online availability

Library availability

Related links

Fingerprint

Cite this