TY - GEN
T1 - The dark energy survey data management system as a data intensive science gateway
AU - Kotwani, Kailash
AU - Myers, James
AU - Baker, Bill
AU - Mohr, Joseph
AU - Daues, Greg
AU - Darnell, Tony
AU - Armstrong, Bob
AU - McLaren, Terrence
AU - Plutchak, Joel
AU - Cai, Dora
AU - Gower, Michelle
AU - Desai, Shantanu
AU - Chandra, Ankit
N1 - Publisher Copyright:
Copyright 2010 ACM.
PY - 2010/11/29
Y1 - 2010/11/29
N2 - The Dark Energy survey (DEs) collaboration is science effort to understand cosmic acceleration and cdark energy1 responsible for this phenomenon. Dark Energy survey Data Management (DEsDM) system observational astronomy processingpipeline management system that will be used to: process obtained from a survey with the newDEs field camera (DE covering 5ooo sq degree of southern sky; archive final co-added images;extractcatalogs of celestial object every image anddeliver data products to communitythrough portals and services. DEsDM has been designed as a data intensive science Gateway coupling use of shared computational resources (e.g. Teragrid) wi owned databases and file systems for storage distributed across three continents. DEsDM system over the next six years time will perform over 1omillioncPu-hours(sus) of image processing and serve over 4Petabytesof images and 14 objectsto the international DEs collaboration. when delivered for operations in 2o11, it will be one of, if not the, most scalable and powerful systems for processing telescope images, creating co added deep images, and generating detailed catalogs in existence. The project1s software components consist of a processing framework, an ensemble of astronomy codes, integratedarchive,adata-access framework infrastructure. This paper provides an overview of the scope and highlights, the architectural features developed and planned to be able to supportGateway-style management scaleintensive continuous processing and on-demand user queries for analysis.
AB - The Dark Energy survey (DEs) collaboration is science effort to understand cosmic acceleration and cdark energy1 responsible for this phenomenon. Dark Energy survey Data Management (DEsDM) system observational astronomy processingpipeline management system that will be used to: process obtained from a survey with the newDEs field camera (DE covering 5ooo sq degree of southern sky; archive final co-added images;extractcatalogs of celestial object every image anddeliver data products to communitythrough portals and services. DEsDM has been designed as a data intensive science Gateway coupling use of shared computational resources (e.g. Teragrid) wi owned databases and file systems for storage distributed across three continents. DEsDM system over the next six years time will perform over 1omillioncPu-hours(sus) of image processing and serve over 4Petabytesof images and 14 objectsto the international DEs collaboration. when delivered for operations in 2o11, it will be one of, if not the, most scalable and powerful systems for processing telescope images, creating co added deep images, and generating detailed catalogs in existence. The project1s software components consist of a processing framework, an ensemble of astronomy codes, integratedarchive,adata-access framework infrastructure. This paper provides an overview of the scope and highlights, the architectural features developed and planned to be able to supportGateway-style management scaleintensive continuous processing and on-demand user queries for analysis.
KW - Collaborative web applications
KW - Large scale systems
KW - Observational astronomy
KW - Pipeline
UR - http://www.scopus.com/inward/record.url?scp=85026768941&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85026768941&partnerID=8YFLogxK
U2 - 10.1145/1890799.1890808
DO - 10.1145/1890799.1890808
M3 - Conference contribution
AN - SCOPUS:85026768941
T3 - Proceedings of the 8th International Workshop on Middleware for Grids, Clouds and e-Science, MGC 2010 - Held at the ACM/IFIP/USENIX 11th International Middleware Conference
BT - Proceedings of the 8th International Workshop on Middleware for Grids, Clouds and e-Science, MGC 2010 - Held at the ACM/IFIP/USENIX 11th International Middleware Conference
PB - Association for Computing Machinery
T2 - 8th International Workshop on Middleware for Grids, Clouds and e-Science, MGC 2010
Y2 - 29 November 2010 through 3 December 2010
ER -