Application-level regression testing framework using Jenkins

Reuben Budiardja, Timothy Bouvet, Galen Arnold

Research output: Contribution to journalArticle

Abstract

Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. Our goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integrate with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.

Original languageEnglish (US)
Article numbere4339
JournalConcurrency Computation
Volume30
Issue number1
DOIs
StatePublished - Jan 10 2018

Fingerprint

Supercomputers
Servers
Regression
Testing
Monitoring
Supercomputer
Authentication
Large scale systems
Automation
Server
Password
Best Practice
Authorization
Large-scale Systems
Use Case
Usability
Water
Monitor
Integrate
Software

Keywords

  • applications
  • benchmarking
  • performance
  • regression testing
  • system-monitoring

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Computer Science Applications
  • Computer Networks and Communications
  • Computational Theory and Mathematics

Cite this

Application-level regression testing framework using Jenkins. / Budiardja, Reuben; Bouvet, Timothy; Arnold, Galen.

In: Concurrency Computation, Vol. 30, No. 1, e4339, 10.01.2018.

Research output: Contribution to journalArticle

Budiardja, Reuben ; Bouvet, Timothy ; Arnold, Galen. / Application-level regression testing framework using Jenkins. In: Concurrency Computation. 2018 ; Vol. 30, No. 1.
@article{2e36c8bb6a78451b891ac7c5b4e1cfc0,
title = "Application-level regression testing framework using Jenkins",
abstract = "Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. Our goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integrate with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.",
keywords = "applications, benchmarking, performance, regression testing, system-monitoring",
author = "Reuben Budiardja and Timothy Bouvet and Galen Arnold",
year = "2018",
month = "1",
day = "10",
doi = "10.1002/cpe.4339",
language = "English (US)",
volume = "30",
journal = "Concurrency Computation",
issn = "1532-0626",
publisher = "John Wiley and Sons Ltd",
number = "1",

}

TY - JOUR

T1 - Application-level regression testing framework using Jenkins

AU - Budiardja, Reuben

AU - Bouvet, Timothy

AU - Arnold, Galen

PY - 2018/1/10

Y1 - 2018/1/10

N2 - Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. Our goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integrate with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.

AB - Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. Our goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integrate with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.

KW - applications

KW - benchmarking

KW - performance

KW - regression testing

KW - system-monitoring

UR - http://www.scopus.com/inward/record.url?scp=85037168425&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85037168425&partnerID=8YFLogxK

U2 - 10.1002/cpe.4339

DO - 10.1002/cpe.4339

M3 - Article

AN - SCOPUS:85037168425

VL - 30

JO - Concurrency Computation

JF - Concurrency Computation

SN - 1532-0626

IS - 1

M1 - e4339

ER -