A 0.44-J/dec, 39.9-s/dec, Recurrent Attention In-Memory Processor for Keyword Spotting

Hassan Dbouk, Sujan K. Gonugondla, Charbel Sakr, Naresh R. Shanbhag

Research output: Contribution to journalArticlepeer-review


This article presents a deep learning-based classifier IC for keyword spotting (KWS) in 65-nm CMOS designed using an algorithm-hardware co-design approach. First, a recurrent attention model (RAM) algorithm for the KWS task (the KeyRAM algorithm) is proposed. The KeyRAM algorithm enables accuracy versus energy scalability via a confidence-based computation (CC) scheme, leading to a 2.5 reduction in computational complexity compared to state-of-the-art (SOTA) neural networks, and is well-suited for in-memory computing (IMC) since the bulk (89%) of its computations are 4-b matrix-vector multiplies. The KeyRAM IC comprises a multi-bit multi-bank IMC architecture with a digital co-processor. A sparsity-aware summation scheme is proposed to alleviate the challenge faced by IMCs when summing sparse activations. The digital co-processor employs diagonal major weight storage to compute without any stalls. This combination of the IMC and digital processors enables a balanced tradeoff between energy efficiency and high accuracy computation. The resultant KWS IC achieves SOTA decision latency of 39.9 s with a decision energy < 0.5 J /dec which translates to more than 24 savings in the energy-delay product (EDP) of decisions over existing KWS ICs.

Original languageEnglish (US)
Article number9239367
Pages (from-to)2234-2244
Number of pages11
JournalIEEE Journal of Solid-State Circuits
Issue number7
StatePublished - Jul 2021


  • In-memory computing (IMC)
  • keyword spotting (KWS)
  • machine learning
  • recurrent attention networks

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this