From Sketching to Natural Language: Expressive Visual Querying for Accelerating Insight

Tarique Siddiqui, Paul Luh, Zesheng Wang, Karrie Karahalios, Aditya G Parameswaran

Research output: Contribution to journalArticlepeer-review


Data visualization is the primary means by which data analysts explore patterns, trends, and insights in their data. Unfortunately, existing visual analytics tools offer limited expressiveness and scalability when it comes to searching for visualizations over large datasets, making visual data exploration labor-intensive and timeconsuming. We first discuss our prior work on Zenvisage that helps accelerate exploratory data analysis via an interactive interface and an expressive visualization query language, but offers limited flexibility when the pattern of interest is under-specified and approximate. Motivated from our findings from Zenvisage, we develop ShapeSearch, an efficient and flexible pattern-searching tool that enables the search for desired patterns via multiple mechanisms: sketch, natural-language, and visual regular expressions. ShapeSearch leverages a novel shape querying algebra that can express a large class of shape queries and supports query-aware and perceptually-aware optimizations to execute shape queries within interactive response times. To further improve the usability and performance of both Zenvisage and ShapeSearch, we discuss a number of open research problems.

Original languageEnglish (US)
Pages (from-to)51-58
Number of pages8
JournalSIGMOD Record
Issue number1
StatePublished - Mar 2021

ASJC Scopus subject areas

  • Software
  • Information Systems


Dive into the research topics of 'From Sketching to Natural Language: Expressive Visual Querying for Accelerating Insight'. Together they form a unique fingerprint.

Cite this