MaskRNN: Instance level video object segmentation

Yuan Ting Hu; Jia Bin Huang; Alexander G. Schwing

MaskRNN: Instance level video object segmentation

Yuan Ting Hu, Jia Bin Huang, Alexander G. Schwing

Research output: Contribution to journal › Conference article

Abstract

Instance level video object segmentation is an important technique for video editing and compression. To capture the temporal coherence, in this paper, we develop MaskRNN, a recurrent neural net approach which fuses in each frame the output of two deep nets for each object instance - a binary segmentation net providing a mask and a localization net providing a bounding box. Due to the recurrent component and the localization component, our method is able to take advantage of long-term temporal structures of the video data as well as rejecting outliers. We validate the proposed algorithm on three challenging benchmark datasets, the DAVIS-2016 dataset, the DAVIS-2017 dataset, and the Segtrack v2 dataset, achieving state-of-the-art performance on all of them.

Original language	English (US)
Pages (from-to)	325-334
Number of pages	10
Journal	Advances in Neural Information Processing Systems
Volume	2017-December
State	Published - Jan 1 2017
Event	31st Annual Conference on Neural Information Processing Systems, NIPS 2017 - Long Beach, United States Duration: Dec 4 2017 → Dec 9 2017

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems
Signal Processing

Access to Document

Fingerprint Dive into the research topics of 'MaskRNN: Instance level video object segmentation'. Together they form a unique fingerprint.

Cite this

Hu, Y. T., Huang, J. B., & Schwing, A. G. (2017). MaskRNN: Instance level video object segmentation. Advances in Neural Information Processing Systems, 2017-December, 325-334.

@article{bec137afe4eb4756b3018e071cf8edfb,

title = "MaskRNN: Instance level video object segmentation",

abstract = "Instance level video object segmentation is an important technique for video editing and compression. To capture the temporal coherence, in this paper, we develop MaskRNN, a recurrent neural net approach which fuses in each frame the output of two deep nets for each object instance - a binary segmentation net providing a mask and a localization net providing a bounding box. Due to the recurrent component and the localization component, our method is able to take advantage of long-term temporal structures of the video data as well as rejecting outliers. We validate the proposed algorithm on three challenging benchmark datasets, the DAVIS-2016 dataset, the DAVIS-2017 dataset, and the Segtrack v2 dataset, achieving state-of-the-art performance on all of them.",

author = "Hu, {Yuan Ting} and Huang, {Jia Bin} and Schwing, {Alexander G.}",

year = "2017",

month = jan,

day = "1",

language = "English (US)",

volume = "2017-December",

pages = "325--334",

journal = "Advances in Neural Information Processing Systems",

issn = "1049-5258",

note = "31st Annual Conference on Neural Information Processing Systems, NIPS 2017 ; Conference date: 04-12-2017 Through 09-12-2017",

}

TY - JOUR

T1 - MaskRNN

T2 - 31st Annual Conference on Neural Information Processing Systems, NIPS 2017

AU - Hu, Yuan Ting

AU - Huang, Jia Bin

AU - Schwing, Alexander G.

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Instance level video object segmentation is an important technique for video editing and compression. To capture the temporal coherence, in this paper, we develop MaskRNN, a recurrent neural net approach which fuses in each frame the output of two deep nets for each object instance - a binary segmentation net providing a mask and a localization net providing a bounding box. Due to the recurrent component and the localization component, our method is able to take advantage of long-term temporal structures of the video data as well as rejecting outliers. We validate the proposed algorithm on three challenging benchmark datasets, the DAVIS-2016 dataset, the DAVIS-2017 dataset, and the Segtrack v2 dataset, achieving state-of-the-art performance on all of them.

AB - Instance level video object segmentation is an important technique for video editing and compression. To capture the temporal coherence, in this paper, we develop MaskRNN, a recurrent neural net approach which fuses in each frame the output of two deep nets for each object instance - a binary segmentation net providing a mask and a localization net providing a bounding box. Due to the recurrent component and the localization component, our method is able to take advantage of long-term temporal structures of the video data as well as rejecting outliers. We validate the proposed algorithm on three challenging benchmark datasets, the DAVIS-2016 dataset, the DAVIS-2017 dataset, and the Segtrack v2 dataset, achieving state-of-the-art performance on all of them.

UR - http://www.scopus.com/inward/record.url?scp=85047000277&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047000277&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85047000277

VL - 2017-December

SP - 325

EP - 334

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

Y2 - 4 December 2017 through 9 December 2017

ER -