Pyro: A spatial-temporal big-data storage system

Shen Li, Shaohan Hu, Raghu Ganti, Mudhakar Srivatsa, Tarek Abdelzaher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

With the rapid growth of mobile devices and applications, geo-tagged data has become a major workload for big data storage systems. In order to achieve scalability, existing solutions build an additional index layer above general purpose distributed data stores. Fulfilling the semantic level need, this approach, however, leaves a lot to be desired for execution efficiency, especially when users query for moving objects within a high resolution geometric area, which we call geometry queries. Such geometry queries translate to a much larger set of range scans, forcing the backend to handle orders of magnitude more requests. Moreover, spatial-temporal applications naturally create dynamic workload hotspots1, which pushes beyond the design scope of existing solutions. This paper presents Pyro, a spatial-temporal bigdata storage system tailored for high resolution geometry queries and dynamic hotspots. Pyro understands geometries internally, which allows range scans of a geometry query to be aggregately optimized. Moreover, Pyro employs a novel replica placement policy in the DFS layer that allows Pyro to split a region without losing data locality benefits. Our evaluations use NYC taxi trace data and an 80-server cluster. Results show that Pyro reduces the response time by 60X on 1km×1km rectangle geometries compared to the state-of-the-art solutions. Pyro further achieves 10X throughput improvement on 100m×100m rectangle geometries2.

Original languageEnglish (US)
Title of host publicationProceedings of the 2015 USENIX Annual Technical Conference, USENIX ATC 2015
PublisherUSENIX Association
Pages97-109
Number of pages13
ISBN (Electronic)9781931971225
StatePublished - Jan 1 2015
Event2015 USENIX Annual Technical Conference, USENIX ATC 2015 - Santa Clara, United States
Duration: Jul 8 2015Jul 10 2015

Publication series

NameProceedings of the 2015 USENIX Annual Technical Conference, USENIX ATC 2015

Conference

Conference2015 USENIX Annual Technical Conference, USENIX ATC 2015
CountryUnited States
CitySanta Clara
Period7/8/157/10/15

    Fingerprint

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Li, S., Hu, S., Ganti, R., Srivatsa, M., & Abdelzaher, T. (2015). Pyro: A spatial-temporal big-data storage system. In Proceedings of the 2015 USENIX Annual Technical Conference, USENIX ATC 2015 (pp. 97-109). (Proceedings of the 2015 USENIX Annual Technical Conference, USENIX ATC 2015). USENIX Association.