TY - JOUR
T1 - Can We Learn Heuristics for Graphical Model Inference Using Reinforcement Learning?
AU - Messaoud, Safa
AU - Kumar, Maghav
AU - Schwing, Alexander G.
N1 - Funding Information:
We study how to solve higher order CRF inference for semantic segmentation with reinforcement learning. The approach is able to deal with potentials that are too expensive to optimize using conventional techniques and outperforms traditional approaches while being more efficient. Hence, the proposed approach offers more flexibility for energy functions while scaling linearly with the number of nodes and the potential order. To answer our question: can we learn heuristics for graphical model inference? We think we can but we also want to note that a lot of manual work is required to find suitable features and graph structures. For this reason we think more research is needed to truly automate learning of heuristics for graphical model inference. We hope the research community will join us in this quest. Acknowledgements: This work is supported in part by NSF under Grant No. 1718221 and MRI #1725729, UIUC, Sam-sung, 3M, and Cisco Systems Inc. (Gift Award CG 1377144). We thank Cisco for access to the Arcetri cluster and Iou-Jen Liu for initial discussions.
PY - 2020
Y1 - 2020
N2 - Combinatorial optimization is frequently used in computer vision. For instance, in applications like semantic segmentation, human pose estimation and action recognition, programs are formulated for solving inference in Conditional Random Fields (CRFs) to produce a structured output that is consistent with visual features of the image. However, solving inference in CRFs is in general intractable, and approximation methods are computationally demanding and limited to unary, pairwise and hand-crafted forms of higher order potentials. In this paper, we show that we can learn program heuristics, i.e., policies, for solving inference in higher order CRFs for the task of semantic segmentation, using reinforcement learning. Our method solves inference tasks efficiently without imposing any constraints on the form of the potentials. We show compelling results on the Pascal VOC and MOTS datasets.
AB - Combinatorial optimization is frequently used in computer vision. For instance, in applications like semantic segmentation, human pose estimation and action recognition, programs are formulated for solving inference in Conditional Random Fields (CRFs) to produce a structured output that is consistent with visual features of the image. However, solving inference in CRFs is in general intractable, and approximation methods are computationally demanding and limited to unary, pairwise and hand-crafted forms of higher order potentials. In this paper, we show that we can learn program heuristics, i.e., policies, for solving inference in higher order CRFs for the task of semantic segmentation, using reinforcement learning. Our method solves inference tasks efficiently without imposing any constraints on the form of the potentials. We show compelling results on the Pascal VOC and MOTS datasets.
UR - http://www.scopus.com/inward/record.url?scp=85094820641&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85094820641&partnerID=8YFLogxK
U2 - 10.1109/CVPR42600.2020.00761
DO - 10.1109/CVPR42600.2020.00761
M3 - Conference article
AN - SCOPUS:85094820641
SP - 7586
EP - 7596
JO - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
JF - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
SN - 1063-6919
M1 - 9156920
T2 - 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020
Y2 - 14 June 2020 through 19 June 2020
ER -