A 3D convolutional neural network method for land cover classification using LiDAR and multi-temporal Landsat imagery

Zewei Xu, Kaiyu Guan, Nathan Casler, Bin Peng, Shaowen Wang

Research output: Contribution to journalArticle

Abstract

Terrestrial landscape has complex three-dimensional (3D) features that are difficult to extract using traditional methods based on 2D representations. These methods often relegate such features to raster or metric-based (two-dimensional) representations based on Digital Surface Models (DSM) or Digital Elevation Models (DEM), and thus are not suitable for resolving morphological and intensity features for fine-scale land cover mapping. Small-footprint LiDAR provides an ideal way for capturing these 3D features. This research develops a novel method of integrating airborne LiDAR derived features and multi-temporal Landsat images to classify land cover types. We tested our approach in Williamson County, Illinois, which has diverse and mixed landscape features. Specifically, our method applied a 3D convolutional neural network (CNN) approach to extract features from LiDAR point clouds by (1) creating an occupancy grid, an intensity grid at 1-meter resolution, and then (2) normalizing and incorporating data into the 3D CNN. The extracted features (e.g., morphological and intensity features) from the 3D CNN were finally combined with multi-temporal spectral data to enhance the performance of land cover classification based on a Support Vector Machine classifier. Visual interpretation from both hyper-resolution photos and point clouds was used for training and preparation of testing data. The classification results show that our method outperforms a traditional method by 2.65% (from 81.52% to 84.17%) when solely using LiDAR and 2.19% (from 90.20% to 92.57%) when combining all available imageries. We demonstrate that our method can effectively extract LiDAR features and improve fine-scale land cover mapping through fusion of complementary types of remote sensing data.

Original languageEnglish (US)
Pages (from-to)423-434
Number of pages12
JournalISPRS Journal of Photogrammetry and Remote Sensing
Volume144
DOIs
StatePublished - Oct 1 2018

Fingerprint

imagery
Landsat
land cover
Neural networks
grids
digital elevation models
normalizing
Support vector machines
Remote sensing
Classifiers
footprints
Fusion reactions
classifiers
remote sensing
education
fusion
Testing
preparation
raster
method

Keywords

  • Big data analysis
  • Convolutional neural network
  • Land cover classification
  • LiDAR
  • Multi-temporal Landsat imagery

ASJC Scopus subject areas

  • Atomic and Molecular Physics, and Optics
  • Engineering (miscellaneous)
  • Computer Science Applications
  • Computers in Earth Sciences

Cite this

A 3D convolutional neural network method for land cover classification using LiDAR and multi-temporal Landsat imagery. / Xu, Zewei; Guan, Kaiyu; Casler, Nathan; Peng, Bin; Wang, Shaowen.

In: ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 144, 01.10.2018, p. 423-434.

Research output: Contribution to journalArticle

@article{3c3041d51e51462c828e66686e3a97cf,
title = "A 3D convolutional neural network method for land cover classification using LiDAR and multi-temporal Landsat imagery",
abstract = "Terrestrial landscape has complex three-dimensional (3D) features that are difficult to extract using traditional methods based on 2D representations. These methods often relegate such features to raster or metric-based (two-dimensional) representations based on Digital Surface Models (DSM) or Digital Elevation Models (DEM), and thus are not suitable for resolving morphological and intensity features for fine-scale land cover mapping. Small-footprint LiDAR provides an ideal way for capturing these 3D features. This research develops a novel method of integrating airborne LiDAR derived features and multi-temporal Landsat images to classify land cover types. We tested our approach in Williamson County, Illinois, which has diverse and mixed landscape features. Specifically, our method applied a 3D convolutional neural network (CNN) approach to extract features from LiDAR point clouds by (1) creating an occupancy grid, an intensity grid at 1-meter resolution, and then (2) normalizing and incorporating data into the 3D CNN. The extracted features (e.g., morphological and intensity features) from the 3D CNN were finally combined with multi-temporal spectral data to enhance the performance of land cover classification based on a Support Vector Machine classifier. Visual interpretation from both hyper-resolution photos and point clouds was used for training and preparation of testing data. The classification results show that our method outperforms a traditional method by 2.65{\%} (from 81.52{\%} to 84.17{\%}) when solely using LiDAR and 2.19{\%} (from 90.20{\%} to 92.57{\%}) when combining all available imageries. We demonstrate that our method can effectively extract LiDAR features and improve fine-scale land cover mapping through fusion of complementary types of remote sensing data.",
keywords = "Big data analysis, Convolutional neural network, Land cover classification, LiDAR, Multi-temporal Landsat imagery",
author = "Zewei Xu and Kaiyu Guan and Nathan Casler and Bin Peng and Shaowen Wang",
year = "2018",
month = "10",
day = "1",
doi = "10.1016/j.isprsjprs.2018.08.005",
language = "English (US)",
volume = "144",
pages = "423--434",
journal = "ISPRS Journal of Photogrammetry and Remote Sensing",
issn = "0924-2716",
publisher = "Elsevier",

}

TY - JOUR

T1 - A 3D convolutional neural network method for land cover classification using LiDAR and multi-temporal Landsat imagery

AU - Xu, Zewei

AU - Guan, Kaiyu

AU - Casler, Nathan

AU - Peng, Bin

AU - Wang, Shaowen

PY - 2018/10/1

Y1 - 2018/10/1

N2 - Terrestrial landscape has complex three-dimensional (3D) features that are difficult to extract using traditional methods based on 2D representations. These methods often relegate such features to raster or metric-based (two-dimensional) representations based on Digital Surface Models (DSM) or Digital Elevation Models (DEM), and thus are not suitable for resolving morphological and intensity features for fine-scale land cover mapping. Small-footprint LiDAR provides an ideal way for capturing these 3D features. This research develops a novel method of integrating airborne LiDAR derived features and multi-temporal Landsat images to classify land cover types. We tested our approach in Williamson County, Illinois, which has diverse and mixed landscape features. Specifically, our method applied a 3D convolutional neural network (CNN) approach to extract features from LiDAR point clouds by (1) creating an occupancy grid, an intensity grid at 1-meter resolution, and then (2) normalizing and incorporating data into the 3D CNN. The extracted features (e.g., morphological and intensity features) from the 3D CNN were finally combined with multi-temporal spectral data to enhance the performance of land cover classification based on a Support Vector Machine classifier. Visual interpretation from both hyper-resolution photos and point clouds was used for training and preparation of testing data. The classification results show that our method outperforms a traditional method by 2.65% (from 81.52% to 84.17%) when solely using LiDAR and 2.19% (from 90.20% to 92.57%) when combining all available imageries. We demonstrate that our method can effectively extract LiDAR features and improve fine-scale land cover mapping through fusion of complementary types of remote sensing data.

AB - Terrestrial landscape has complex three-dimensional (3D) features that are difficult to extract using traditional methods based on 2D representations. These methods often relegate such features to raster or metric-based (two-dimensional) representations based on Digital Surface Models (DSM) or Digital Elevation Models (DEM), and thus are not suitable for resolving morphological and intensity features for fine-scale land cover mapping. Small-footprint LiDAR provides an ideal way for capturing these 3D features. This research develops a novel method of integrating airborne LiDAR derived features and multi-temporal Landsat images to classify land cover types. We tested our approach in Williamson County, Illinois, which has diverse and mixed landscape features. Specifically, our method applied a 3D convolutional neural network (CNN) approach to extract features from LiDAR point clouds by (1) creating an occupancy grid, an intensity grid at 1-meter resolution, and then (2) normalizing and incorporating data into the 3D CNN. The extracted features (e.g., morphological and intensity features) from the 3D CNN were finally combined with multi-temporal spectral data to enhance the performance of land cover classification based on a Support Vector Machine classifier. Visual interpretation from both hyper-resolution photos and point clouds was used for training and preparation of testing data. The classification results show that our method outperforms a traditional method by 2.65% (from 81.52% to 84.17%) when solely using LiDAR and 2.19% (from 90.20% to 92.57%) when combining all available imageries. We demonstrate that our method can effectively extract LiDAR features and improve fine-scale land cover mapping through fusion of complementary types of remote sensing data.

KW - Big data analysis

KW - Convolutional neural network

KW - Land cover classification

KW - LiDAR

KW - Multi-temporal Landsat imagery

UR - http://www.scopus.com/inward/record.url?scp=85051758921&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85051758921&partnerID=8YFLogxK

U2 - 10.1016/j.isprsjprs.2018.08.005

DO - 10.1016/j.isprsjprs.2018.08.005

M3 - Article

VL - 144

SP - 423

EP - 434

JO - ISPRS Journal of Photogrammetry and Remote Sensing

JF - ISPRS Journal of Photogrammetry and Remote Sensing

SN - 0924-2716

ER -