ViX: Analysis-driven Compiler for Efficient Low-Precision Variational Inference

Ashitabh Misra, Jacob Laurel, Sasa Misailovic

Research output: Chapter in Book/Report/Conference proceedingConference contribution


As large quantities of stochastic data are processed onboard tiny edge devices, these systems must constantly make decisions under uncertainty. This challenge necessitates principled embedded compiler support for time- and energy-efficient probabilistic inference. However, compiling probabilistic inference to run on the edge is significantly understudied, and the existing research is limited to computationally expensive MCMC algorithms. Hence, these works cannot leverage faster variational inference algorithms which can better scale to larger data sizes that are representative of realistic workloads in the edge setting. However, naively writing code for differentiable inference on resource-constrained edge devices is challenging due to the need for expensive floating point computations. Even when using reduced precision, a developer still faces the challenge of choosing the right quantization scheme, as gradients can be notoriously unstable in the face of low-precision. To address these challenges, we propose ViX which is the first compiler for low-precision probabilistic programming with variational inference. ViX generates optimized variational inference code in reduced precision by automatically exploiting Bayesian domain knowledge and analytical mathematical properties to ensure that low-precision gradients can still be effectively used. ViX can scale inference to much larger data-sets than previous compilers for resource-constrained probabilistic programming while attaining both high accuracy and significant speedup. Our evaluation of ViX across 7 benchmarks shows that ViX-generated code is up to 8.15× faster than performing the same variational inference in 32-bit floating point and also up to 22.67× faster than performing the variational inference in 64-bit double precision, all with minimal accuracy loss. Further, on a subset of our benchmarks, ViX can scale inference to data sizes between 16-80× larger than the existing state-of-the-art tool Statheros.

Original languageEnglish (US)
Title of host publication2023 Design, Automation and Test in Europe Conference and Exhibition, DATE 2023 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9783981926378
StatePublished - 2023
Externally publishedYes
Event2023 Design, Automation and Test in Europe Conference and Exhibition, DATE 2023 - Antwerp, Belgium
Duration: Apr 17 2023Apr 19 2023

Publication series

NameProceedings -Design, Automation and Test in Europe, DATE
ISSN (Print)1530-1591


Conference2023 Design, Automation and Test in Europe Conference and Exhibition, DATE 2023

ASJC Scopus subject areas

  • General Engineering


Dive into the research topics of 'ViX: Analysis-driven Compiler for Efficient Low-Precision Variational Inference'. Together they form a unique fingerprint.

Cite this