DeepStore: In-storage acceleration for intelligent queries

Vikram Sharma Mailthody, Zaid Qureshi, Weixin Liang, Ziyan Feng, Simon Garcia De Gonzalo, Youjie Li, Hubertus Franke, Jinjun Xiong, Jian Huang, Wen Mei Hwu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recent advancements in deep learning techniques facilitate intelligentquery support in diverse applications, such as content-based image retrieval and audio texturing. Unlike conventional key-based queries, these intelligent queries lack efficient indexing and require complex compute operations for feature matching. To achieve highperformance intelligent querying against massive datasets, modern computing systems employ GPUs in-conjunction with solid-state drives (SSDs) for fast data access and parallel data processing. However, our characterization with various intelligent-query workloads developed with deep neural networks (DNNs), shows that the storage I/O bandwidth is still the major bottleneck that contributes 56%-90% of the query execution time. To this end, we present DeepStore, an in-storage accelerator architecture for intelligent queries. It consists of (1) energy-efficient in-storage accelerators designed specifically for supporting DNNbased intelligent queries, under the resource constraints in modern SSD controllers; (2) a similarity-based in-storage query cache to exploit the temporal locality of user queries for further performance improvement; and (3) a lightweight in-storage runtime system working as the query engine, which provides a simple software abstraction to support different types of intelligent queries. DeepStore exploits SSD parallelisms with design space exploration for achieving the maximal energy efficiency for in-storage accelerators. We validate DeepStore design with an SSD simulator, and evaluate it with a variety of vision, text, and audio based intelligent queries. Compared with the state-of-the-art GPU+SSD approach, DeepStore improves the query performance by up to 17.7, and energy-efficiency by up to 78.6.

Original languageEnglish (US)
Title of host publicationMICRO 2019 - 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings
PublisherIEEE Computer Society
Pages224-238
Number of pages15
ISBN (Electronic)9781450369381
DOIs
StatePublished - Oct 12 2019
Event52nd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2019 - Columbus, United States
Duration: Oct 12 2019Oct 16 2019

Publication series

NameProceedings of the Annual International Symposium on Microarchitecture, MICRO
ISSN (Print)1072-4451

Conference

Conference52nd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2019
CountryUnited States
CityColumbus
Period10/12/1910/16/19

Fingerprint

Particle accelerators
Energy efficiency
Texturing
Image retrieval
Simulators
Engines
Bandwidth
Controllers
Graphics processing unit
Deep neural networks
Deep learning

Keywords

  • Hardware accelerators
  • In-storage computing
  • Information retrieval
  • Intelligent query
  • Solid-state drive

ASJC Scopus subject areas

  • Hardware and Architecture

Cite this

Mailthody, V. S., Qureshi, Z., Liang, W., Feng, Z., Gonzalo, S. G. D., Li, Y., ... Hwu, W. M. (2019). DeepStore: In-storage acceleration for intelligent queries. In MICRO 2019 - 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings (pp. 224-238). (Proceedings of the Annual International Symposium on Microarchitecture, MICRO). IEEE Computer Society. https://doi.org/10.1145/3352460.3358320

DeepStore : In-storage acceleration for intelligent queries. / Mailthody, Vikram Sharma; Qureshi, Zaid; Liang, Weixin; Feng, Ziyan; Gonzalo, Simon Garcia De; Li, Youjie; Franke, Hubertus; Xiong, Jinjun; Huang, Jian; Hwu, Wen Mei.

MICRO 2019 - 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings. IEEE Computer Society, 2019. p. 224-238 (Proceedings of the Annual International Symposium on Microarchitecture, MICRO).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mailthody, VS, Qureshi, Z, Liang, W, Feng, Z, Gonzalo, SGD, Li, Y, Franke, H, Xiong, J, Huang, J & Hwu, WM 2019, DeepStore: In-storage acceleration for intelligent queries. in MICRO 2019 - 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings. Proceedings of the Annual International Symposium on Microarchitecture, MICRO, IEEE Computer Society, pp. 224-238, 52nd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2019, Columbus, United States, 10/12/19. https://doi.org/10.1145/3352460.3358320
Mailthody VS, Qureshi Z, Liang W, Feng Z, Gonzalo SGD, Li Y et al. DeepStore: In-storage acceleration for intelligent queries. In MICRO 2019 - 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings. IEEE Computer Society. 2019. p. 224-238. (Proceedings of the Annual International Symposium on Microarchitecture, MICRO). https://doi.org/10.1145/3352460.3358320
Mailthody, Vikram Sharma ; Qureshi, Zaid ; Liang, Weixin ; Feng, Ziyan ; Gonzalo, Simon Garcia De ; Li, Youjie ; Franke, Hubertus ; Xiong, Jinjun ; Huang, Jian ; Hwu, Wen Mei. / DeepStore : In-storage acceleration for intelligent queries. MICRO 2019 - 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings. IEEE Computer Society, 2019. pp. 224-238 (Proceedings of the Annual International Symposium on Microarchitecture, MICRO).
@inproceedings{29929414203d449a96dd92d7d5208024,
title = "DeepStore: In-storage acceleration for intelligent queries",
abstract = "Recent advancements in deep learning techniques facilitate intelligentquery support in diverse applications, such as content-based image retrieval and audio texturing. Unlike conventional key-based queries, these intelligent queries lack efficient indexing and require complex compute operations for feature matching. To achieve highperformance intelligent querying against massive datasets, modern computing systems employ GPUs in-conjunction with solid-state drives (SSDs) for fast data access and parallel data processing. However, our characterization with various intelligent-query workloads developed with deep neural networks (DNNs), shows that the storage I/O bandwidth is still the major bottleneck that contributes 56{\%}-90{\%} of the query execution time. To this end, we present DeepStore, an in-storage accelerator architecture for intelligent queries. It consists of (1) energy-efficient in-storage accelerators designed specifically for supporting DNNbased intelligent queries, under the resource constraints in modern SSD controllers; (2) a similarity-based in-storage query cache to exploit the temporal locality of user queries for further performance improvement; and (3) a lightweight in-storage runtime system working as the query engine, which provides a simple software abstraction to support different types of intelligent queries. DeepStore exploits SSD parallelisms with design space exploration for achieving the maximal energy efficiency for in-storage accelerators. We validate DeepStore design with an SSD simulator, and evaluate it with a variety of vision, text, and audio based intelligent queries. Compared with the state-of-the-art GPU+SSD approach, DeepStore improves the query performance by up to 17.7, and energy-efficiency by up to 78.6.",
keywords = "Hardware accelerators, In-storage computing, Information retrieval, Intelligent query, Solid-state drive",
author = "Mailthody, {Vikram Sharma} and Zaid Qureshi and Weixin Liang and Ziyan Feng and Gonzalo, {Simon Garcia De} and Youjie Li and Hubertus Franke and Jinjun Xiong and Jian Huang and Hwu, {Wen Mei}",
year = "2019",
month = "10",
day = "12",
doi = "10.1145/3352460.3358320",
language = "English (US)",
series = "Proceedings of the Annual International Symposium on Microarchitecture, MICRO",
publisher = "IEEE Computer Society",
pages = "224--238",
booktitle = "MICRO 2019 - 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings",

}

TY - GEN

T1 - DeepStore

T2 - In-storage acceleration for intelligent queries

AU - Mailthody, Vikram Sharma

AU - Qureshi, Zaid

AU - Liang, Weixin

AU - Feng, Ziyan

AU - Gonzalo, Simon Garcia De

AU - Li, Youjie

AU - Franke, Hubertus

AU - Xiong, Jinjun

AU - Huang, Jian

AU - Hwu, Wen Mei

PY - 2019/10/12

Y1 - 2019/10/12

N2 - Recent advancements in deep learning techniques facilitate intelligentquery support in diverse applications, such as content-based image retrieval and audio texturing. Unlike conventional key-based queries, these intelligent queries lack efficient indexing and require complex compute operations for feature matching. To achieve highperformance intelligent querying against massive datasets, modern computing systems employ GPUs in-conjunction with solid-state drives (SSDs) for fast data access and parallel data processing. However, our characterization with various intelligent-query workloads developed with deep neural networks (DNNs), shows that the storage I/O bandwidth is still the major bottleneck that contributes 56%-90% of the query execution time. To this end, we present DeepStore, an in-storage accelerator architecture for intelligent queries. It consists of (1) energy-efficient in-storage accelerators designed specifically for supporting DNNbased intelligent queries, under the resource constraints in modern SSD controllers; (2) a similarity-based in-storage query cache to exploit the temporal locality of user queries for further performance improvement; and (3) a lightweight in-storage runtime system working as the query engine, which provides a simple software abstraction to support different types of intelligent queries. DeepStore exploits SSD parallelisms with design space exploration for achieving the maximal energy efficiency for in-storage accelerators. We validate DeepStore design with an SSD simulator, and evaluate it with a variety of vision, text, and audio based intelligent queries. Compared with the state-of-the-art GPU+SSD approach, DeepStore improves the query performance by up to 17.7, and energy-efficiency by up to 78.6.

AB - Recent advancements in deep learning techniques facilitate intelligentquery support in diverse applications, such as content-based image retrieval and audio texturing. Unlike conventional key-based queries, these intelligent queries lack efficient indexing and require complex compute operations for feature matching. To achieve highperformance intelligent querying against massive datasets, modern computing systems employ GPUs in-conjunction with solid-state drives (SSDs) for fast data access and parallel data processing. However, our characterization with various intelligent-query workloads developed with deep neural networks (DNNs), shows that the storage I/O bandwidth is still the major bottleneck that contributes 56%-90% of the query execution time. To this end, we present DeepStore, an in-storage accelerator architecture for intelligent queries. It consists of (1) energy-efficient in-storage accelerators designed specifically for supporting DNNbased intelligent queries, under the resource constraints in modern SSD controllers; (2) a similarity-based in-storage query cache to exploit the temporal locality of user queries for further performance improvement; and (3) a lightweight in-storage runtime system working as the query engine, which provides a simple software abstraction to support different types of intelligent queries. DeepStore exploits SSD parallelisms with design space exploration for achieving the maximal energy efficiency for in-storage accelerators. We validate DeepStore design with an SSD simulator, and evaluate it with a variety of vision, text, and audio based intelligent queries. Compared with the state-of-the-art GPU+SSD approach, DeepStore improves the query performance by up to 17.7, and energy-efficiency by up to 78.6.

KW - Hardware accelerators

KW - In-storage computing

KW - Information retrieval

KW - Intelligent query

KW - Solid-state drive

UR - http://www.scopus.com/inward/record.url?scp=85074449092&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074449092&partnerID=8YFLogxK

U2 - 10.1145/3352460.3358320

DO - 10.1145/3352460.3358320

M3 - Conference contribution

AN - SCOPUS:85074449092

T3 - Proceedings of the Annual International Symposium on Microarchitecture, MICRO

SP - 224

EP - 238

BT - MICRO 2019 - 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Proceedings

PB - IEEE Computer Society

ER -