An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures

Smruti Padhy, Jay Alameda, Rob Kooper, Rui Liu, Sandeep Puthanveetil Satheesan, Inna Zharnitsky, Gregory Jansen, Michael C. Dietze, Praveen Kumar, Jong Sung Lee, Richard Marciano, Luigi Marini, Barbara S Minsker, Chris Navarro, Marcus Slavenas, William C Sullivan, Kenton Guadron McHenry

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Brown Dog is an extensible data cyberinfrastructure, that provides a set of extensible and distributed data conversion and metadata extraction services to enable access and search within unstructured, un-curated and inaccessible research data across different domains of sciences and social science, which ultimately aids in supporting reproducibility of results. We envision that Brown Dog, as a data cyberinfrastructure, is an essential service in a comprehensive cyberinfrastructure which includes data services, high performance computing services and more that would enable scholarly research in a variety of disciplines that today is not yet possi-ble. Brown Dog focuses on four initial use cases, specifically, addressing the conversion and extraction needs in the research areas of ecology, civil and environmental engineering, library and information science, and use by the general public. In this paper, we describe an architecture that supports contribution of data transformation tools from users, and automatic deployment of the tools as Brown Dog services in diverse infrastructures such as cloud or high performance computing (HPC) based on user demands and load on the system. We also present results validating the performance of the initial implementation of Brown Dog.

Original languageEnglish (US)
Title of host publicationProceedings of XSEDE 2016
Subtitle of host publicationDiversity, Big Data, and Science at Scale
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450347556
DOIs
StatePublished - Jul 17 2016
EventConference on Diversity, Big Data, and Science at Scale, XSEDE 2016 - Miami, United States
Duration: Jul 17 2016Jul 21 2016

Publication series

NameACM International Conference Proceeding Series
Volume17-21-July-2016

Other

OtherConference on Diversity, Big Data, and Science at Scale, XSEDE 2016
CountryUnited States
CityMiami
Period7/17/167/21/16

Fingerprint

Environmental engineering
Information science
Social sciences
Information use
Ecology
Civil engineering
Metadata

Keywords

  • Autocuration
  • Civil and environmental engineering
  • Cloud
  • Data conversion
  • Data cyberinfrastructure
  • Digital preservation
  • Ecology
  • Elasticity
  • HPC
  • Library and information science
  • Metadata extraction

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Cite this

Padhy, S., Alameda, J., Kooper, R., Liu, R., Satheesan, S. P., Zharnitsky, I., ... McHenry, K. G. (2016). An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures. In Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale [a33] (ACM International Conference Proceeding Series; Vol. 17-21-July-2016). Association for Computing Machinery. https://doi.org/10.1145/2949550.2949647

An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures. / Padhy, Smruti; Alameda, Jay; Kooper, Rob; Liu, Rui; Satheesan, Sandeep Puthanveetil; Zharnitsky, Inna; Jansen, Gregory; Dietze, Michael C.; Kumar, Praveen; Lee, Jong Sung; Marciano, Richard; Marini, Luigi; Minsker, Barbara S; Navarro, Chris; Slavenas, Marcus; Sullivan, William C; McHenry, Kenton Guadron.

Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale. Association for Computing Machinery, 2016. a33 (ACM International Conference Proceeding Series; Vol. 17-21-July-2016).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Padhy, S, Alameda, J, Kooper, R, Liu, R, Satheesan, SP, Zharnitsky, I, Jansen, G, Dietze, MC, Kumar, P, Lee, JS, Marciano, R, Marini, L, Minsker, BS, Navarro, C, Slavenas, M, Sullivan, WC & McHenry, KG 2016, An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures. in Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale., a33, ACM International Conference Proceeding Series, vol. 17-21-July-2016, Association for Computing Machinery, Conference on Diversity, Big Data, and Science at Scale, XSEDE 2016, Miami, United States, 7/17/16. https://doi.org/10.1145/2949550.2949647
Padhy S, Alameda J, Kooper R, Liu R, Satheesan SP, Zharnitsky I et al. An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures. In Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale. Association for Computing Machinery. 2016. a33. (ACM International Conference Proceeding Series). https://doi.org/10.1145/2949550.2949647
Padhy, Smruti ; Alameda, Jay ; Kooper, Rob ; Liu, Rui ; Satheesan, Sandeep Puthanveetil ; Zharnitsky, Inna ; Jansen, Gregory ; Dietze, Michael C. ; Kumar, Praveen ; Lee, Jong Sung ; Marciano, Richard ; Marini, Luigi ; Minsker, Barbara S ; Navarro, Chris ; Slavenas, Marcus ; Sullivan, William C ; McHenry, Kenton Guadron. / An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures. Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale. Association for Computing Machinery, 2016. (ACM International Conference Proceeding Series).
@inproceedings{58a66527404e42a380456913be5f2ed4,
title = "An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures",
abstract = "Brown Dog is an extensible data cyberinfrastructure, that provides a set of extensible and distributed data conversion and metadata extraction services to enable access and search within unstructured, un-curated and inaccessible research data across different domains of sciences and social science, which ultimately aids in supporting reproducibility of results. We envision that Brown Dog, as a data cyberinfrastructure, is an essential service in a comprehensive cyberinfrastructure which includes data services, high performance computing services and more that would enable scholarly research in a variety of disciplines that today is not yet possi-ble. Brown Dog focuses on four initial use cases, specifically, addressing the conversion and extraction needs in the research areas of ecology, civil and environmental engineering, library and information science, and use by the general public. In this paper, we describe an architecture that supports contribution of data transformation tools from users, and automatic deployment of the tools as Brown Dog services in diverse infrastructures such as cloud or high performance computing (HPC) based on user demands and load on the system. We also present results validating the performance of the initial implementation of Brown Dog.",
keywords = "Autocuration, Civil and environmental engineering, Cloud, Data conversion, Data cyberinfrastructure, Digital preservation, Ecology, Elasticity, HPC, Library and information science, Metadata extraction",
author = "Smruti Padhy and Jay Alameda and Rob Kooper and Rui Liu and Satheesan, {Sandeep Puthanveetil} and Inna Zharnitsky and Gregory Jansen and Dietze, {Michael C.} and Praveen Kumar and Lee, {Jong Sung} and Richard Marciano and Luigi Marini and Minsker, {Barbara S} and Chris Navarro and Marcus Slavenas and Sullivan, {William C} and McHenry, {Kenton Guadron}",
year = "2016",
month = "7",
day = "17",
doi = "10.1145/2949550.2949647",
language = "English (US)",
series = "ACM International Conference Proceeding Series",
publisher = "Association for Computing Machinery",
booktitle = "Proceedings of XSEDE 2016",

}

TY - GEN

T1 - An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures

AU - Padhy, Smruti

AU - Alameda, Jay

AU - Kooper, Rob

AU - Liu, Rui

AU - Satheesan, Sandeep Puthanveetil

AU - Zharnitsky, Inna

AU - Jansen, Gregory

AU - Dietze, Michael C.

AU - Kumar, Praveen

AU - Lee, Jong Sung

AU - Marciano, Richard

AU - Marini, Luigi

AU - Minsker, Barbara S

AU - Navarro, Chris

AU - Slavenas, Marcus

AU - Sullivan, William C

AU - McHenry, Kenton Guadron

PY - 2016/7/17

Y1 - 2016/7/17

N2 - Brown Dog is an extensible data cyberinfrastructure, that provides a set of extensible and distributed data conversion and metadata extraction services to enable access and search within unstructured, un-curated and inaccessible research data across different domains of sciences and social science, which ultimately aids in supporting reproducibility of results. We envision that Brown Dog, as a data cyberinfrastructure, is an essential service in a comprehensive cyberinfrastructure which includes data services, high performance computing services and more that would enable scholarly research in a variety of disciplines that today is not yet possi-ble. Brown Dog focuses on four initial use cases, specifically, addressing the conversion and extraction needs in the research areas of ecology, civil and environmental engineering, library and information science, and use by the general public. In this paper, we describe an architecture that supports contribution of data transformation tools from users, and automatic deployment of the tools as Brown Dog services in diverse infrastructures such as cloud or high performance computing (HPC) based on user demands and load on the system. We also present results validating the performance of the initial implementation of Brown Dog.

AB - Brown Dog is an extensible data cyberinfrastructure, that provides a set of extensible and distributed data conversion and metadata extraction services to enable access and search within unstructured, un-curated and inaccessible research data across different domains of sciences and social science, which ultimately aids in supporting reproducibility of results. We envision that Brown Dog, as a data cyberinfrastructure, is an essential service in a comprehensive cyberinfrastructure which includes data services, high performance computing services and more that would enable scholarly research in a variety of disciplines that today is not yet possi-ble. Brown Dog focuses on four initial use cases, specifically, addressing the conversion and extraction needs in the research areas of ecology, civil and environmental engineering, library and information science, and use by the general public. In this paper, we describe an architecture that supports contribution of data transformation tools from users, and automatic deployment of the tools as Brown Dog services in diverse infrastructures such as cloud or high performance computing (HPC) based on user demands and load on the system. We also present results validating the performance of the initial implementation of Brown Dog.

KW - Autocuration

KW - Civil and environmental engineering

KW - Cloud

KW - Data conversion

KW - Data cyberinfrastructure

KW - Digital preservation

KW - Ecology

KW - Elasticity

KW - HPC

KW - Library and information science

KW - Metadata extraction

UR - http://www.scopus.com/inward/record.url?scp=84989177759&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84989177759&partnerID=8YFLogxK

U2 - 10.1145/2949550.2949647

DO - 10.1145/2949550.2949647

M3 - Conference contribution

T3 - ACM International Conference Proceeding Series

BT - Proceedings of XSEDE 2016

PB - Association for Computing Machinery

ER -