Abstract
We present and define a structured digital object, called a "Tale," for the dissemination and publication of computational scientific findings in the scholarly record. The Tale emerges from the NSF funded Whole Tale project (wholetale.org) which is developing a computational environment designed to capture the entire computational pipeline associated with a scientific experiment and thereby enable computational reproducibility. A Tale allows researchers to create and package code, data and information about the workflow and computational environment necessary to support, review, and recreate the computational results reported in published research. The Tale then captures the artifacts and information needed to facilitate understanding, transparency, and execution of the Tale for review and reproducibility at the time of publication.
| Original language | English (US) |
|---|---|
| Title of host publication | P-RECS 2019 - Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems, co-located with HPDC 2019 |
| Publisher | Association for Computing Machinery, Inc |
| Pages | 17-22 |
| Number of pages | 6 |
| ISBN (Electronic) | 9781450367561 |
| DOIs | |
| State | Published - Jun 17 2019 |
| Event | 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems, P-RECS 2019, co-located with HPDC 2019 - Phoenix, United States Duration: Jun 24 2019 → … |
Publication series
| Name | P-RECS 2019 - Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems, co-located with HPDC 2019 |
|---|
Conference
| Conference | 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems, P-RECS 2019, co-located with HPDC 2019 |
|---|---|
| Country/Territory | United States |
| City | Phoenix |
| Period | 6/24/19 → … |
Keywords
- Computing environments
- Cyberinfrastructure
- Open code
- Open data
- Publishing standards
- Reproducibility
ASJC Scopus subject areas
- Computational Theory and Mathematics
- Computer Science Applications
- Software
Fingerprint
Dive into the research topics of 'Implementing computational reproducibility in the Whole Tale environment'. Together they form a unique fingerprint.Cite this
- APA
- Standard
- Harvard
- Vancouver
- Author
- BIBTEX
- RIS
Implementing computational reproducibility in the Whole Tale environment. / Chard, Kyle; Gaffney, Niall; Jones, Matthew B. et al.
P-RECS 2019 - Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems, co-located with HPDC 2019. Association for Computing Machinery, Inc, 2019. p. 17-22 (P-RECS 2019 - Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems, co-located with HPDC 2019).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Implementing computational reproducibility in the Whole Tale environment
AU - Chard, Kyle
AU - Gaffney, Niall
AU - Jones, Matthew B.
AU - Kowalik, Kacper
AU - Ludaescher, Bertram
AU - Nabrzyski, Jarek
AU - Stodden, Victoria
AU - Taylor, Ian
AU - Turk, Matthew J
AU - Willis, Craig
N1 - Funding Information: We present and define a structured digital object, called a "Tale," for the dissemination and publication of computational scientific findings in the scholarly record. The Tale emerges from the NSF funded Whole Tale project (wholetale.org) which is developing a computational environment designed to capture the entire computational pipeline associated with a scientific experiment and thereby enable computational reproducibility. A Tale allows researchers to create and package code, data and information about the workflow and computational environment necessary to support, review, and recreate the computational results reported in published research. The Tale then captures the artifacts and information needed to facilitate understanding, transparency, and execution of the Tale for review and reproducibility at the time of publication. computing environments cyberinfrastructure open code open data publishing standards reproducibility 1 Kyle Chard Author 2 Niall Gaffney Author 3 Matthew B. Jones Author 4 Kacper Kowalik Author 5 Bertram Ludäscher Author 6 Jarek Nabrzyski Author 7 Victoria Stodden Author 8 Ian Taylor Author 9 Matthew J. Turk Author 10 Craig Willis Author 1 H. Monajemi, D. L. Donoho and V. Stodden (2016). Making massive computational experiments painless. IEEE International Conference on Big Data (Big Data), Washington, DC, 2016, pp. 2368--2373. doi: 10.1109/BigData.2016.7840870 2 V. Stodden, F. Leisch, R. D. Peng (eds.) (2014). Implementing Reproducible Research. Boca Raton, FL: Chapman & Hall/CRC The R Series. 3 I. Jimenez, M. Sevilla, N. Watkins, C. Maltzahn, J. Lofstead, K. Mohror, A. Arpaci-Dusseau and R. Arpaci-Dusseau. (2017). The Popper Convention: Making Reproducible Systems Evaluation Practical. IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 1561--70. 4 I. Jimenez, M. Sevilla, N. Watkins, C. Maltzahn, J. Lofstead, K. Mohror, A. Arpaci-Dusseau and R. Arpaci-Dusseau. (2016). Standing on the Shoulders of Giants by Managing Scientific Experiments Like Software, USENIX. 41(4). 5 M. McLennan, and R. Kennell. (2010). HUBzero: A Platform for Dissemination and Collaboration in Computational Science and Engineering. Computing in Science & Engineering. 12(2). 10.1109/MCSE.2010.41 6 N. Wilkins-Diehr. (2007), Special Issue: Science Gateways-Common Community Interfaces to Grid Resources. Concurrency Computat.: Pract. Exper., 19: 743--749. doi:10.1002/cpe.1098 7 M. Dahan, J. Towns, T. Cockerill, I Foster, K. Gaither, A. Grimshaw, V. Hazlewood, S. Lathrop, D. Lifka, G. Peterson, R. Roskies, J. Scott, & N. Wilkins-Diehr. (2014). XSEDE: Accelerating scientific discovery. Computing in Science and Engineering. 16. 62--74. 10.1109/MCSE.2014.80. 8 A. Brinckman, K. Chard, N. Gaffney, M. Hategan, M. B. Jones, K.Kowalik, S. Kulasekaran, B. Ludaescher, B. D. Mecum, J. Nabrzyski, V. Stodden, I. J. Taylor, M. J. Turk, K. Turner. (2019). Computing environments for reproducibility: Capturing the "Whole Tale". Future Generation Comp. Syst. 94. pp. 854--867. 9 N. Cartwright. (1991), Replicability, Reproducibility, and Robustness: Comments on Harry Collins, History of Political Economy, 23(1). pp. 143--155. 10 J. F. Claerbout and M. Karrenbach. (1992). Electronic documents give reproducible research a new meaning. SEG Technical Program Expanded Abstracts. pp. 601--604. 10.1190/1.1822162 11 J. B. Buckheit and D. L. Donoho. (1995) WaveLab and Reproducible Research. In: Antoniadis A., Oppenheim G. (eds) Wavelets and Statistics. Lecture Notes in Statistics, vol 103. Springer, New York, NY. 12 R. D. Peng. (2011). Reproducible Research in Computational Science. Science. 334(6060). pp. 1226--1227. 10.1126/science.1213847 13 V. Stodden. (2013). Resolving Irreproducibility in Empirical and Computational Research. IMS Bull. Online. http://bit.ly/BullIMStat2013. 14 L. A. Barba. (2018). Terminologies for Reproducible Research. CoRR. abs/1802.03311. 15 R. Gentleman and D. Temple Lang. (2007). Statistical Analyses and Reproducible Research. Journal of Computational and Graphical Statistics. 16(1). pp. 1--23. 16 V. Stodden and S. Miguez. 2014. Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research. Journal of Open Research Software, 2(1), p.e21. DOI: http://doi.org/10.5334/jors.ay 17 V. Stodden, D. Bailey and J. Borwein (2013). Set the Default to 'Open'. Notices of the AMS. 18 V. Stodden, D. Bailey and J. Borwein (2013). "Setting the Default to Reproducible" in Computational Science Research. SIAM News. 19 ICERM Reproducibility Workshop Report. (2012). Setting the Default to Reproducible. https://icerm.brown.edu/topical_workshops/tw12--5-rcem/icerm_report.pdf 20 T. Christian, S. Lafferty-Hess, W. G. Jacoby, and T. Carsey. (2018). Operationalizing the Replication Standard. International Journal of Digital Curation. 13(1). DOI: https://doi.org/10.2218/ijdc.v13i1.555 21 K. Chard, M. D'Arcy, B. Heavner, I. Foster, C. Kesselman, R. Madduri, A. Rodriguez, S. Soiland-Reyes, C. Goble, K. Clark, E. W. Deutsch, I. Dinov, N. Price, A. Toga. (2016). I'll take that to go: Big data bags and minimal identifiers for exchange of large complex datasets. IEEE International Conference on Big Data. 22 C. Willis. (2019). Whole Tale Example "Water Tale". DOI: 10.5281/zenodo.2641314 . https://zenodo.org/record/2641314#.XLVCees3nOQ 23 F. Ren, L. Ward, T. Williams, K. J. Laws, C. Wolverton, J. Hattrick-Simpers and A. Mehta. (2018). Accelerated discovery of metallic glasses through iteration of machine learning and high-throughput experiments. Science. 4(4). DOI: 10.1126/sciadv.aaq1566 24 B. Blaiszik, K. Chard, J. Pruyne, R. Ananthakrishnan, S. Tuecke, and I. Foster. (2016). The Materials Data Facility: Data services to advance materials science research. JOM. 68(8). pp. 2045--2052. 25 Gentleman, Robert and Temple Lang, Duncan, (2004). "Statistical Analyses and Reproducible Research" Bioconductor Project Working Papers. Working Paper 2. https://biostats.bepress.com/bioconductor/paper2 26 V. Stodden, S. Miguez, J. Seiler. (2015). "ResearchCompendia.org: Cyberinfrastructure for Reproducibility and Collaboration in Computational Science." Computing in Science & Engineering, 17(1). 10.1109/MCSE.2015.18 27 F. Chirigati, D. Shasha and J. Freire, (2013). ReproZip: Using Provenance to Support Computational Reproducibility, 5th USENIX Workshop on the Theory and Practice of Provenance. 28 L. Oliveira, D. Wilkinson, D. Mossé, and B. Childers. (2018). Supporting Long-term Reproducible Software Execution. In Proceedings of the First International Workshop on Practical Reproducible Evaluation of Computer Systems (P-RECS'18). ACM, New York, NY, USA, Article 6, 6 pages. DOI: https://doi.org/10.1145/3214239.3214245 29 I. Jimenez, M. Sevilla, N. Watkins, C. Maltzahn, J. Lofstead, K. Mohror, A. Arpaci-Dusseau and R. Arpaci-Dusseau. (2017). The Popper Convention: Making Reproducible Systems Evaluation Practical. IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 1561--70. https://doi.org/10.1109/IPDPSW.2017.157. 30 G. Fursin, A. Lokhmotov and E. Plowman. (2016). Collective Knowledge: Towards R&D Sustainability. Proceedings of the Conference on Design, Automation and Test in Europe (DATE'16). 31 B. Grüning, J. Chilton, J. Köster, R. Dale, N. Soranzo, M. van den Beek, J. Goecks, R. Backofen, A. Nekrutenko and J. Taylor. (2018). Practical computational reproducibility in the life sciences. Cell Systems 6. 32 Z. Yuan, D. Hai Ton That, S. Kothari, G. Fils, and T. Malik. (2018). Utilizing Provenance in Reusable Research Objects. Informatics, 5(1), 14; https://doi.org/10.3390/informatics5010014 33 A. P Davison, M Mattioni, D Samarkanov, B Tele'nczuk. (2014). Sumatra: A Toolkit for Reproducible Research. In Implementing Reproducible Research, Eds: Stodden, V and Leisch, F and and Chapman, R D Peng, pp.57--79 34 L. A. M. C. Carvalho, K. Belhajjame and C. B. Medeiros, "Converting scripts into reproducible workflow research objects," 2016 IEEE 12th International Conference on e-Science (e-Science), Baltimore, MD, 2016, pp. 71--80. doi: 10.1109/eScience.2016.7870887 35 Sandve GK, Nekrutenko A, Taylor J, Hovig E (2013) Ten Simple Rules for Reproducible Computational Research. PLoS Comput Biol 9(10): e1003285. https://doi.org/10.1371/journal.pcbi.1003285 36 Jupyter et al., (2018). Binder 2.0 - Reproducible, Interactive, Sharable Environments for Science at Scale." Proceedings of the 17th Python in Science Conference. doi:10.25080/Majora-4af1f417-011 1 p17-chard.pdf ACM 2019 [-] [-] [-] [-] [-] Publisher Copyright: © 2019 Association for Computing Machinery.
PY - 2019/6/17
Y1 - 2019/6/17
N2 - We present and define a structured digital object, called a "Tale," for the dissemination and publication of computational scientific findings in the scholarly record. The Tale emerges from the NSF funded Whole Tale project (wholetale.org) which is developing a computational environment designed to capture the entire computational pipeline associated with a scientific experiment and thereby enable computational reproducibility. A Tale allows researchers to create and package code, data and information about the workflow and computational environment necessary to support, review, and recreate the computational results reported in published research. The Tale then captures the artifacts and information needed to facilitate understanding, transparency, and execution of the Tale for review and reproducibility at the time of publication.
AB - We present and define a structured digital object, called a "Tale," for the dissemination and publication of computational scientific findings in the scholarly record. The Tale emerges from the NSF funded Whole Tale project (wholetale.org) which is developing a computational environment designed to capture the entire computational pipeline associated with a scientific experiment and thereby enable computational reproducibility. A Tale allows researchers to create and package code, data and information about the workflow and computational environment necessary to support, review, and recreate the computational results reported in published research. The Tale then captures the artifacts and information needed to facilitate understanding, transparency, and execution of the Tale for review and reproducibility at the time of publication.
KW - Computing environments
KW - Cyberinfrastructure
KW - Open code
KW - Open data
KW - Publishing standards
KW - Reproducibility
UR - http://www.scopus.com/inward/record.url?scp=85069169849&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85069169849&partnerID=8YFLogxK
U2 - 10.1145/3322790.3330594
DO - 10.1145/3322790.3330594
M3 - Conference contribution
AN - SCOPUS:85069169849
T3 - P-RECS 2019 - Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems, co-located with HPDC 2019
SP - 17
EP - 22
BT - P-RECS 2019 - Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems, co-located with HPDC 2019
PB - Association for Computing Machinery, Inc
T2 - 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems, P-RECS 2019, co-located with HPDC 2019
Y2 - 24 June 2019
ER -