An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures

Smruti Padhy, Jay Alameda, Rob Kooper, Rui Liu, Sandeep Puthanveetil Satheesan, Inna Zharnitsky, Gregory Jansen, Michael C. Dietze, Praveen Kumar, Jong Sung Lee, Richard Marciano, Luigi Marini, Barbara S Minsker, Chris Navarro, Marcus Slavenas, William C Sullivan, Kenton Guadron McHenry

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Brown Dog is an extensible data cyberinfrastructure, that provides a set of extensible and distributed data conversion and metadata extraction services to enable access and search within unstructured, un-curated and inaccessible research data across different domains of sciences and social science, which ultimately aids in supporting reproducibility of results. We envision that Brown Dog, as a data cyberinfrastructure, is an essential service in a comprehensive cyberinfrastructure which includes data services, high performance computing services and more that would enable scholarly research in a variety of disciplines that today is not yet possi-ble. Brown Dog focuses on four initial use cases, specifically, addressing the conversion and extraction needs in the research areas of ecology, civil and environmental engineering, library and information science, and use by the general public. In this paper, we describe an architecture that supports contribution of data transformation tools from users, and automatic deployment of the tools as Brown Dog services in diverse infrastructures such as cloud or high performance computing (HPC) based on user demands and load on the system. We also present results validating the performance of the initial implementation of Brown Dog.



  • Autocuration
  • Civil and environmental engineering
  • Cloud
  • Data conversion
  • Data cyberinfrastructure
  • Digital preservation
  • Ecology
  • Elasticity
  • HPC
  • Library and information science
  • Metadata extraction

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Cite this

Padhy, S., Alameda, J., Kooper, R., Liu, R., Satheesan, S. P., Zharnitsky, I., Jansen, G., Dietze, M. C., Kumar, P., Lee, J. S., Marciano, R., Marini, L., Minsker, B. S., Navarro, C., Slavenas, M., Sullivan, W. C., & McHenry, K. G. (2016). An architecture for automatic deployment of brown dog services at scale into diverse computing infrastructures. In Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale [a33] (ACM International Conference Proceeding Series; Vol. 17-21-July-2016). Association for Computing Machinery.