Abstract
We present Brown Dog, two highly extensible services that aim to leverage any existing pieces of code, libraries, services, or standalone software (past or present) towards providing users with a simple to use and programmable means of automated aid in the curation and indexing of distributed collections of uncurated and/or unstructured data. Data collections such as these encompassing large varieties of data, in addition to large amounts of data, pose a significant challenge within modern day «Big Data» efforts. The two services, the Data Access Proxy (DAP) and the Data Tilling Service (DTS), focusing on format conversions and content based analysis/extraction respectively, wrap relevant conversion and extraction operations within arbitrary software, manages their deployment in an elastic manner, and manages job execution from behind a deliberately compact REST API. We describe both the motivation and need/scientific drivers for such services, the constituent components that allow for arbitrary software/code to be used and managed, and lastly an evaluation of the systems capabilities and scalability.
| Original language | English (US) |
|---|---|
| Title of host publication | Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015 |
| Editors | Feng Luo, Kemafor Ogan, Mohammed J. Zaki, Laura Haas, Beng Chin Ooi, Vipin Kumar, Sudarsan Rachuri, Saumyadipta Pyne, Howard Ho, Xiaohua Hu, Shipeng Yu, Morris Hui-I Hsiao, Jian Li |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 493-500 |
| Number of pages | 8 |
| ISBN (Electronic) | 9781479999255 |
| DOIs | |
| State | Published - Dec 22 2015 |
| Event | 3rd IEEE International Conference on Big Data, IEEE Big Data 2015 - Santa Clara, United States Duration: Oct 29 2015 → Nov 1 2015 |
Publication series
| Name | Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015 |
|---|
Other
| Other | 3rd IEEE International Conference on Big Data, IEEE Big Data 2015 |
|---|---|
| Country | United States |
| City | Santa Clara |
| Period | 10/29/15 → 11/1/15 |
Fingerprint
Keywords
- digital preservation
- unstructured data
- web services
ASJC Scopus subject areas
- Computer Networks and Communications
- Computer Science Applications
- Information Systems
- Software
Cite this
Brown Dog : Leveraging everything towards autocuration. / Padhy, Smruti; Jansen, Greg; Alameda, Jay; Black, Edgar; Diesendruck, Liana; Dietze, Mike; Kumar, Praveen; Kooper, Rob; Lee, Jong Sung; Liu, Rui; Marciano, Richard; Marini, Luigi; Mattson, Dave; Minsker, Barbara; Navarro, Chris; Slavenas, Marcus; Sullivan, William C; Votava, Jason; Zharnitsky, Inna; McHenry, Kenton Guadron.
Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015. ed. / Feng Luo; Kemafor Ogan; Mohammed J. Zaki; Laura Haas; Beng Chin Ooi; Vipin Kumar; Sudarsan Rachuri; Saumyadipta Pyne; Howard Ho; Xiaohua Hu; Shipeng Yu; Morris Hui-I Hsiao; Jian Li. Institute of Electrical and Electronics Engineers Inc., 2015. p. 493-500 7363791 (Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Brown Dog
T2 - Leveraging everything towards autocuration
AU - Padhy, Smruti
AU - Jansen, Greg
AU - Alameda, Jay
AU - Black, Edgar
AU - Diesendruck, Liana
AU - Dietze, Mike
AU - Kumar, Praveen
AU - Kooper, Rob
AU - Lee, Jong Sung
AU - Liu, Rui
AU - Marciano, Richard
AU - Marini, Luigi
AU - Mattson, Dave
AU - Minsker, Barbara
AU - Navarro, Chris
AU - Slavenas, Marcus
AU - Sullivan, William C
AU - Votava, Jason
AU - Zharnitsky, Inna
AU - McHenry, Kenton Guadron
PY - 2015/12/22
Y1 - 2015/12/22
N2 - We present Brown Dog, two highly extensible services that aim to leverage any existing pieces of code, libraries, services, or standalone software (past or present) towards providing users with a simple to use and programmable means of automated aid in the curation and indexing of distributed collections of uncurated and/or unstructured data. Data collections such as these encompassing large varieties of data, in addition to large amounts of data, pose a significant challenge within modern day «Big Data» efforts. The two services, the Data Access Proxy (DAP) and the Data Tilling Service (DTS), focusing on format conversions and content based analysis/extraction respectively, wrap relevant conversion and extraction operations within arbitrary software, manages their deployment in an elastic manner, and manages job execution from behind a deliberately compact REST API. We describe both the motivation and need/scientific drivers for such services, the constituent components that allow for arbitrary software/code to be used and managed, and lastly an evaluation of the systems capabilities and scalability.
AB - We present Brown Dog, two highly extensible services that aim to leverage any existing pieces of code, libraries, services, or standalone software (past or present) towards providing users with a simple to use and programmable means of automated aid in the curation and indexing of distributed collections of uncurated and/or unstructured data. Data collections such as these encompassing large varieties of data, in addition to large amounts of data, pose a significant challenge within modern day «Big Data» efforts. The two services, the Data Access Proxy (DAP) and the Data Tilling Service (DTS), focusing on format conversions and content based analysis/extraction respectively, wrap relevant conversion and extraction operations within arbitrary software, manages their deployment in an elastic manner, and manages job execution from behind a deliberately compact REST API. We describe both the motivation and need/scientific drivers for such services, the constituent components that allow for arbitrary software/code to be used and managed, and lastly an evaluation of the systems capabilities and scalability.
KW - digital preservation
KW - unstructured data
KW - web services
UR - http://www.scopus.com/inward/record.url?scp=84963716196&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84963716196&partnerID=8YFLogxK
U2 - 10.1109/BigData.2015.7363791
DO - 10.1109/BigData.2015.7363791
M3 - Conference contribution
AN - SCOPUS:84963716196
T3 - Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015
SP - 493
EP - 500
BT - Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015
A2 - Luo, Feng
A2 - Ogan, Kemafor
A2 - Zaki, Mohammed J.
A2 - Haas, Laura
A2 - Ooi, Beng Chin
A2 - Kumar, Vipin
A2 - Rachuri, Sudarsan
A2 - Pyne, Saumyadipta
A2 - Ho, Howard
A2 - Hu, Xiaohua
A2 - Yu, Shipeng
A2 - Hsiao, Morris Hui-I
A2 - Li, Jian
PB - Institute of Electrical and Electronics Engineers Inc.
ER -