A Novel Framework for 3D-2D Vertebra Matching

Hanchao Yu, Yang Fu, Haichao Yu, Yunchao Wei, Xinchao Wang, Jianbo Jiao, Matthew Bramlet, T Kesavadas, Honghui Shi, Zhangyang Wang, Bihan Wen, Thomas S Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

3D-2D medical image matching is a crucial task in image-guided surgery, image-guided radiation therapy and minimally invasive surgery. The task relies on identifying the correspondence between a 2D reference image and the 2D projection of 3D target image. In this paper, we propose a novel image matching framework between 3D CT projection and 2D X-ray image, tailored for vertebra images. The main idea is to learn a vertebra detector by means of deep neural network. The detected vertebra is represented by a bounding box in the 3D CT projection. Next, the bounding box annotated by the doctor on the X-ray image is matched to the corresponding box in the 3D projection. We evaluate our proposed method on our own-collected 3D-2D registration dataset. The experimental results show that our framework outperforms the state-of-the-art neural network-based keypoint matching methods.

Original languageEnglish (US)
Title of host publicationProceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages121-126
Number of pages6
ISBN (Electronic)9781728111988
DOIs
StatePublished - Apr 22 2019
Event2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 - San Jose, United States
Duration: Mar 28 2019Mar 30 2019

Publication series

NameProceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

Conference

Conference2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
CountryUnited States
CitySan Jose
Period3/28/193/30/19

Fingerprint

Image matching
Surgery
X rays
Radiotherapy
Detectors
Neural networks
Deep neural networks

Keywords

  • 3D 2D registration
  • Hough transform
  • object detection

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Signal Processing
  • Media Technology

Cite this

Yu, H., Fu, Y., Yu, H., Wei, Y., Wang, X., Jiao, J., ... Huang, T. S. (2019). A Novel Framework for 3D-2D Vertebra Matching. In Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 (pp. 121-126). [8695312] (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MIPR.2019.00029

A Novel Framework for 3D-2D Vertebra Matching. / Yu, Hanchao; Fu, Yang; Yu, Haichao; Wei, Yunchao; Wang, Xinchao; Jiao, Jianbo; Bramlet, Matthew; Kesavadas, T; Shi, Honghui; Wang, Zhangyang; Wen, Bihan; Huang, Thomas S.

Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 121-126 8695312 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yu, H, Fu, Y, Yu, H, Wei, Y, Wang, X, Jiao, J, Bramlet, M, Kesavadas, T, Shi, H, Wang, Z, Wen, B & Huang, TS 2019, A Novel Framework for 3D-2D Vertebra Matching. in Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019., 8695312, Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019, Institute of Electrical and Electronics Engineers Inc., pp. 121-126, 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019, San Jose, United States, 3/28/19. https://doi.org/10.1109/MIPR.2019.00029
Yu H, Fu Y, Yu H, Wei Y, Wang X, Jiao J et al. A Novel Framework for 3D-2D Vertebra Matching. In Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 121-126. 8695312. (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019). https://doi.org/10.1109/MIPR.2019.00029
Yu, Hanchao ; Fu, Yang ; Yu, Haichao ; Wei, Yunchao ; Wang, Xinchao ; Jiao, Jianbo ; Bramlet, Matthew ; Kesavadas, T ; Shi, Honghui ; Wang, Zhangyang ; Wen, Bihan ; Huang, Thomas S. / A Novel Framework for 3D-2D Vertebra Matching. Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 121-126 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019).
@inproceedings{77c3fdaa51c54d609693cd0619e8f350,
title = "A Novel Framework for 3D-2D Vertebra Matching",
abstract = "3D-2D medical image matching is a crucial task in image-guided surgery, image-guided radiation therapy and minimally invasive surgery. The task relies on identifying the correspondence between a 2D reference image and the 2D projection of 3D target image. In this paper, we propose a novel image matching framework between 3D CT projection and 2D X-ray image, tailored for vertebra images. The main idea is to learn a vertebra detector by means of deep neural network. The detected vertebra is represented by a bounding box in the 3D CT projection. Next, the bounding box annotated by the doctor on the X-ray image is matched to the corresponding box in the 3D projection. We evaluate our proposed method on our own-collected 3D-2D registration dataset. The experimental results show that our framework outperforms the state-of-the-art neural network-based keypoint matching methods.",
keywords = "3D 2D registration, Hough transform, object detection",
author = "Hanchao Yu and Yang Fu and Haichao Yu and Yunchao Wei and Xinchao Wang and Jianbo Jiao and Matthew Bramlet and T Kesavadas and Honghui Shi and Zhangyang Wang and Bihan Wen and Huang, {Thomas S}",
year = "2019",
month = "4",
day = "22",
doi = "10.1109/MIPR.2019.00029",
language = "English (US)",
series = "Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "121--126",
booktitle = "Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019",
address = "United States",

}

TY - GEN

T1 - A Novel Framework for 3D-2D Vertebra Matching

AU - Yu, Hanchao

AU - Fu, Yang

AU - Yu, Haichao

AU - Wei, Yunchao

AU - Wang, Xinchao

AU - Jiao, Jianbo

AU - Bramlet, Matthew

AU - Kesavadas, T

AU - Shi, Honghui

AU - Wang, Zhangyang

AU - Wen, Bihan

AU - Huang, Thomas S

PY - 2019/4/22

Y1 - 2019/4/22

N2 - 3D-2D medical image matching is a crucial task in image-guided surgery, image-guided radiation therapy and minimally invasive surgery. The task relies on identifying the correspondence between a 2D reference image and the 2D projection of 3D target image. In this paper, we propose a novel image matching framework between 3D CT projection and 2D X-ray image, tailored for vertebra images. The main idea is to learn a vertebra detector by means of deep neural network. The detected vertebra is represented by a bounding box in the 3D CT projection. Next, the bounding box annotated by the doctor on the X-ray image is matched to the corresponding box in the 3D projection. We evaluate our proposed method on our own-collected 3D-2D registration dataset. The experimental results show that our framework outperforms the state-of-the-art neural network-based keypoint matching methods.

AB - 3D-2D medical image matching is a crucial task in image-guided surgery, image-guided radiation therapy and minimally invasive surgery. The task relies on identifying the correspondence between a 2D reference image and the 2D projection of 3D target image. In this paper, we propose a novel image matching framework between 3D CT projection and 2D X-ray image, tailored for vertebra images. The main idea is to learn a vertebra detector by means of deep neural network. The detected vertebra is represented by a bounding box in the 3D CT projection. Next, the bounding box annotated by the doctor on the X-ray image is matched to the corresponding box in the 3D projection. We evaluate our proposed method on our own-collected 3D-2D registration dataset. The experimental results show that our framework outperforms the state-of-the-art neural network-based keypoint matching methods.

KW - 3D 2D registration

KW - Hough transform

KW - object detection

UR - http://www.scopus.com/inward/record.url?scp=85065605641&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065605641&partnerID=8YFLogxK

U2 - 10.1109/MIPR.2019.00029

DO - 10.1109/MIPR.2019.00029

M3 - Conference contribution

T3 - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

SP - 121

EP - 126

BT - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -