Deep scientific computing requires deep data

W. T.C. Kramer, A. Shoshani, D. A. Agarwal, B. R. Draney, G. Jin, G. F. Butler, J. A. Hules

Research output: Contribution to journalArticlepeer-review


Increasingly, scientific advances require the fusion of large amounts of complex data with extraordinary amounts of computational power. The problems of deep science demand deep computing and deep storage resources. In addition to teraflop-range computing engines with their own local storage, facilities must provide large data repositories of the order of 10-100 petabytes, and networking to allow the movement of multi-terabyte files in a timely and secure manner. This paper examines such problems and identifies associated challenges. The paper discusses some of the storage systems and data management methods that are needed for computing facilities to address the challenges and describes some ongoing improvements.

Original languageEnglish (US)
Pages (from-to)209-232
Number of pages24
JournalIBM Journal of Research and Development
Issue number2
StatePublished - Mar 2004
Externally publishedYes

ASJC Scopus subject areas

  • General Computer Science


Dive into the research topics of 'Deep scientific computing requires deep data'. Together they form a unique fingerprint.

Cite this