TY - GEN
T1 - Morph
T2 - 51st Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2018
AU - Hegde, Kartik
AU - Agrawal, Rohit
AU - Yao, Yulun
AU - Fletcher, Christopher W.
N1 - Funding Information:
This work was partially supported by NSF award CCF-1725734 and a DARPA SDH contract. †These two authors contributed equally
Publisher Copyright:
© 2018 IEEE.
PY - 2018/12/12
Y1 - 2018/12/12
N2 - The past several years have seen both an explosion in the use of Convolutional Neural Networks (CNNs) and accelerators to make CNN inference practical. In the architecture community, the lion share of effort has targeted CNN inference for image recognition. The closely related problem of video recognition has received far less attention as an accelerator target. This is surprising, as video recognition is more computationally intensive than image recognition, and video traffic is predicted to be the majority of internet traffic in the coming years. This paper fills the gap between algorithmic and hardware advances for video recognition by providing a design space exploration and flexible architecture for accelerating 3D Convolutional Neural Networks (3D CNNs)-the core kernel in modern video understanding. When compared to (2D) CNNs used for image recognition, efficiently accelerating 3D CNNs poses a significant engineering challenge due to their large (and variable over time) memory footprint and higher dimensionality. To address these challenges, we design a novel accelerator called 'Morph,' that can adaptively support different spatial and temporal tiling strategies depending on the needs of each layer of each target 3D CNN. We codesign a software infrastructure alongside the Morph hardware to find good-fit parameters to control the hardware. Evaluated on state-of-The-Art 3D CNNs, Morph achieves up to 2.7× (1.9× average) reduction in energy consumption and improves performance/watt up to 4.4× (3× average) compared to a baseline 3D CNN accelerator, with an area overhead of 2%. Morph further achieves a 11.6× average energy reduction on 3D CNNs when compared to Eyeriss, a popular 2D CNN accelerator, while reducing efficiency compared to Eyeriss on a 2D CNN by 71%.
AB - The past several years have seen both an explosion in the use of Convolutional Neural Networks (CNNs) and accelerators to make CNN inference practical. In the architecture community, the lion share of effort has targeted CNN inference for image recognition. The closely related problem of video recognition has received far less attention as an accelerator target. This is surprising, as video recognition is more computationally intensive than image recognition, and video traffic is predicted to be the majority of internet traffic in the coming years. This paper fills the gap between algorithmic and hardware advances for video recognition by providing a design space exploration and flexible architecture for accelerating 3D Convolutional Neural Networks (3D CNNs)-the core kernel in modern video understanding. When compared to (2D) CNNs used for image recognition, efficiently accelerating 3D CNNs poses a significant engineering challenge due to their large (and variable over time) memory footprint and higher dimensionality. To address these challenges, we design a novel accelerator called 'Morph,' that can adaptively support different spatial and temporal tiling strategies depending on the needs of each layer of each target 3D CNN. We codesign a software infrastructure alongside the Morph hardware to find good-fit parameters to control the hardware. Evaluated on state-of-The-Art 3D CNNs, Morph achieves up to 2.7× (1.9× average) reduction in energy consumption and improves performance/watt up to 4.4× (3× average) compared to a baseline 3D CNN accelerator, with an area overhead of 2%. Morph further achieves a 11.6× average energy reduction on 3D CNNs when compared to Eyeriss, a popular 2D CNN accelerator, while reducing efficiency compared to Eyeriss on a 2D CNN by 71%.
KW - 3D Convolutional Neural Networks
KW - Dataflow
KW - Hardware acceleration
KW - Hardware/Software codesign
KW - Video recognition
UR - http://www.scopus.com/inward/record.url?scp=85060031231&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85060031231&partnerID=8YFLogxK
U2 - 10.1109/MICRO.2018.00080
DO - 10.1109/MICRO.2018.00080
M3 - Conference contribution
AN - SCOPUS:85060031231
T3 - Proceedings of the Annual International Symposium on Microarchitecture, MICRO
SP - 933
EP - 946
BT - Proceedings - 51st Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2018
PB - IEEE Computer Society
Y2 - 20 October 2018 through 24 October 2018
ER -