TY - GEN
T1 - Network-aware data caching and prefetching for Cloud-hosted metadata retrieval
AU - Zhang, Bing
AU - Ross, Brandon
AU - Tripathi, Sanatkumar
AU - Batra, Sonali
AU - Kosar, Tevfik
N1 - Copyright:
Copyright 2014 Elsevier B.V., All rights reserved.
PY - 2013
Y1 - 2013
N2 - With the overwhelming emergence of data-intensive applications in the Cloud, the wide-area transfer of metadata and other descriptive information about remote data is critically important for searching, indexing, and enumerating remote file system hierarchies, as well as for purposes of data transfer estimation and reservation. In this paper, we present a highly efficient network-aware caching and prefetching mechanism tailored to reduce metadata access latency and improve responsiveness in wide-area data transfers. To improve the maximum requests per second (RPS) handled by the system, we designed and implemented a network-aware prefetching service using dynamically provisioned parallel TCP streams. To improve the performance of accessing local metadata, we designed and implemented a non-blocking concurrent in-memory cache to handle unexpected bursts of requests. We have implemented the proposed mechanisms in the Directory Listing Service (DLS) system-a Cloud-hosted metadata retrieval, caching, and prefetching system, and have evaluated its performance on Amazon EC2 and XSEDE.
AB - With the overwhelming emergence of data-intensive applications in the Cloud, the wide-area transfer of metadata and other descriptive information about remote data is critically important for searching, indexing, and enumerating remote file system hierarchies, as well as for purposes of data transfer estimation and reservation. In this paper, we present a highly efficient network-aware caching and prefetching mechanism tailored to reduce metadata access latency and improve responsiveness in wide-area data transfers. To improve the maximum requests per second (RPS) handled by the system, we designed and implemented a network-aware prefetching service using dynamically provisioned parallel TCP streams. To improve the performance of accessing local metadata, we designed and implemented a non-blocking concurrent in-memory cache to handle unexpected bursts of requests. We have implemented the proposed mechanisms in the Directory Listing Service (DLS) system-a Cloud-hosted metadata retrieval, caching, and prefetching system, and have evaluated its performance on Amazon EC2 and XSEDE.
KW - Caching
KW - Metadata retrieval
KW - Prefetching
KW - Software as a service (SaaS)
KW - Wide-area transfers
UR - http://www.scopus.com/inward/record.url?scp=84892932155&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84892932155&partnerID=8YFLogxK
U2 - 10.1145/2534695.2534700
DO - 10.1145/2534695.2534700
M3 - Conference contribution
AN - SCOPUS:84892932155
SN - 9781450325226
T3 - Proc. of NDM 2013: 3rd Int. Workshop on Network-Aware Data Management - Held in Conjunction with SC 2013: The Int. Conference for High Performance Computing, Networking, Storage and Analysis
BT - Proc. of NDM 2013
T2 - 3rd International Workshop on Network-Aware Data Management, NDM 2013 - Held in Conjunction with the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013
Y2 - 17 November 2013 through 17 November 2013
ER -