Airphant: Cloud-oriented Document Indexing

Supawit Chockchowwat, Chaitanya Sood, Yongjoo Park

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Modern data warehouses can scale compute nodes independently of storage. These systems persist their data on cloud storage, which is always available and cost-efficient. Ad-hoc compute nodes then fetch necessary data on-demand from cloud storage. This ability to quickly scale or shrink data systems is highly beneficial if query workloads may change over time. We apply this new architecture to search engines with a focus on optimizing their latencies in cloud environments. However, simply placing existing search engines (e.g., Apache Lucene) on top of cloud storage significantly increases their end-to-end query latencies (i.e., more than 6 seconds on average in one of our studies). This is because their indexes can incur multiple network round-trips due to their hierarchical structure (e.g., skip lists, B-trees, learned indexes). To address this issue, we develop a new statistical index, called IoU Sketch. For lookup, IoU Sketch makes multiple asynchronous network requests in parallel. While IoU Sketch may fetch more bytes than existing indexes, it significantly reduces the index lookup time because parallel requests do not block each other. Based on IoU Sketch, we built an end-to-end search engine called Airphant; we describe how Airphant builds, optimizes, and manages IoU Sketch, and ultimately supports keyword-based querying. In our experiments with four real datasets, Airphant's average end-to-end latencies are between 13 milliseconds and 300 milliseconds, up to 8.97× faster than Apache Lucence and 113.39× faster than Elasticsearch.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 IEEE 38th International Conference on Data Engineering, ICDE 2022
PublisherIEEE Computer Society
Number of pages14
ISBN (Electronic)9781665408837
StatePublished - 2022
Event38th IEEE International Conference on Data Engineering, ICDE 2022 - Virtual, Online, Malaysia
Duration: May 9 2022May 12 2022

Publication series

NameProceedings - International Conference on Data Engineering
ISSN (Print)1084-4627


Conference38th IEEE International Conference on Data Engineering, ICDE 2022
CityVirtual, Online


  • Airphant
  • Cloud storage
  • Database as a service
  • Indexing
  • Information retrieval
  • Inverted Index
  • IoU Sketch
  • Multi layer Hash Table
  • Physical database design
  • Separation of compute and storage
  • Sketch data structure

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems


Dive into the research topics of 'Airphant: Cloud-oriented Document Indexing'. Together they form a unique fingerprint.

Cite this