Video-rate stereo matching using Markov random field TRW-S inference on a hybrid CPU+FPGA computing platform

Jungwook Choi, Rob A. Rutenbar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We demonstrate a video-rate stereo matching system implemented on a hybrid CPU+FPGA platform (Convey HC-1). Emerging applications such as 3D gesture recognition and automotive navigation demand fast and high quality stereo vision. We describe a custom hardware-accelerated Markov Random Field inference system for this task. Starting from a core architecture for streaming tree-reweighted message passing (TRW-S) inference, we describe the end-to-end system engineering needed to move from this single frame message update to full stereo video. We partition the stereo matching procedure across the CPU and the FPGAs, and apply both function-level pipelining and frame-level parallelism to achieve the required speed. Experimental results show that our system achieves a speed of 12 frames per second for challenging video stereo matching tasks. We note that this appears to be the first implementation of TRW-S inference at video rates, and that our system is also significantly faster than several recent GPU implementations of similar stereo inference methods based on belief propagation (BP).

Original languageEnglish (US)
Title of host publicationFPGA 2013 - Proceedings of the 2013 ACM/SIGDA International Symposium on Field Programmable Gate Arrays
Pages63-71
Number of pages9
DOIs
StatePublished - Mar 7 2013
Event2013 ACM/SIGDA International Symposium on Field Programmable Gate Arrays, FPGA 2013 - Monterey, CA, United States
Duration: Feb 11 2013Feb 13 2013

Publication series

NameACM/SIGDA International Symposium on Field Programmable Gate Arrays - FPGA

Other

Other2013 ACM/SIGDA International Symposium on Field Programmable Gate Arrays, FPGA 2013
CountryUnited States
CityMonterey, CA
Period2/11/132/13/13

Keywords

  • convey hybrid-core computing platform
  • markov random field energy minimization methods
  • tree-reweighted message passing
  • video-rate stereo matching

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Video-rate stereo matching using Markov random field TRW-S inference on a hybrid CPU+FPGA computing platform'. Together they form a unique fingerprint.

Cite this