CarStream: An industrial system of big data processing for Internet-of-Vehicles

Mingming Zhang, Tianyu Wo, Tao Xie, Xuelian Lin, Yaxiao Liu

Research output: Contribution to journalConference article

Abstract

As the Internet-of-Vehicles (IoV) technology becomes an increasingly important trend for future transportation, designing large-scale IoV systems has become a critical task that aims to process big data uploaded by eet vehicles and to provide data-driven services. The IoV data, especially high-frequency vehicle statuses (e.g., location, engine parameters), are characterized as large volume with a low density of value and low data quality. Such characteristics pose challenges for developing real-time applications based on such data. In this paper, we address the challenges in designing a scalable IoV system by describing CarStream, an industrial system of big data processing for chaufieured car services. Connected with over 30,000 vehicles, CarStream collects and processes multiple types of driving data including vehicle status, driver activity, and passenger-trip information. Multiple services are provided based on the collected data. CarStream has been deployed and maintained for three years in industrial usage, collecting over 40 terabytes of driving data. This paper shares our experiences on designing CarStream based on large-scale driving-data streams, and the lessons learned from the process of addressing the challenges in designing and maintaining CarStream.

Original languageEnglish (US)
Pages (from-to)1766-1777
Number of pages12
JournalProceedings of the VLDB Endowment
Volume10
Issue number12
DOIs
StatePublished - Aug 1 2017
Event43rd International Conference on Very Large Data Bases, VLDB 2017 - Munich, Germany
Duration: Aug 28 2017Sep 1 2017

Fingerprint

Internet
Big data
Railroad cars
Engines

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Cite this

CarStream : An industrial system of big data processing for Internet-of-Vehicles. / Zhang, Mingming; Wo, Tianyu; Xie, Tao; Lin, Xuelian; Liu, Yaxiao.

In: Proceedings of the VLDB Endowment, Vol. 10, No. 12, 01.08.2017, p. 1766-1777.

Research output: Contribution to journalConference article

Zhang, Mingming ; Wo, Tianyu ; Xie, Tao ; Lin, Xuelian ; Liu, Yaxiao. / CarStream : An industrial system of big data processing for Internet-of-Vehicles. In: Proceedings of the VLDB Endowment. 2017 ; Vol. 10, No. 12. pp. 1766-1777.
@article{056f42a1d9d84985b7820c0fef05ed25,
title = "CarStream: An industrial system of big data processing for Internet-of-Vehicles",
abstract = "As the Internet-of-Vehicles (IoV) technology becomes an increasingly important trend for future transportation, designing large-scale IoV systems has become a critical task that aims to process big data uploaded by eet vehicles and to provide data-driven services. The IoV data, especially high-frequency vehicle statuses (e.g., location, engine parameters), are characterized as large volume with a low density of value and low data quality. Such characteristics pose challenges for developing real-time applications based on such data. In this paper, we address the challenges in designing a scalable IoV system by describing CarStream, an industrial system of big data processing for chaufieured car services. Connected with over 30,000 vehicles, CarStream collects and processes multiple types of driving data including vehicle status, driver activity, and passenger-trip information. Multiple services are provided based on the collected data. CarStream has been deployed and maintained for three years in industrial usage, collecting over 40 terabytes of driving data. This paper shares our experiences on designing CarStream based on large-scale driving-data streams, and the lessons learned from the process of addressing the challenges in designing and maintaining CarStream.",
author = "Mingming Zhang and Tianyu Wo and Tao Xie and Xuelian Lin and Yaxiao Liu",
year = "2017",
month = "8",
day = "1",
doi = "10.14778/3137765.3137781",
language = "English (US)",
volume = "10",
pages = "1766--1777",
journal = "Proceedings of the VLDB Endowment",
issn = "2150-8097",
publisher = "Very Large Data Base Endowment Inc.",
number = "12",

}

TY - JOUR

T1 - CarStream

T2 - An industrial system of big data processing for Internet-of-Vehicles

AU - Zhang, Mingming

AU - Wo, Tianyu

AU - Xie, Tao

AU - Lin, Xuelian

AU - Liu, Yaxiao

PY - 2017/8/1

Y1 - 2017/8/1

N2 - As the Internet-of-Vehicles (IoV) technology becomes an increasingly important trend for future transportation, designing large-scale IoV systems has become a critical task that aims to process big data uploaded by eet vehicles and to provide data-driven services. The IoV data, especially high-frequency vehicle statuses (e.g., location, engine parameters), are characterized as large volume with a low density of value and low data quality. Such characteristics pose challenges for developing real-time applications based on such data. In this paper, we address the challenges in designing a scalable IoV system by describing CarStream, an industrial system of big data processing for chaufieured car services. Connected with over 30,000 vehicles, CarStream collects and processes multiple types of driving data including vehicle status, driver activity, and passenger-trip information. Multiple services are provided based on the collected data. CarStream has been deployed and maintained for three years in industrial usage, collecting over 40 terabytes of driving data. This paper shares our experiences on designing CarStream based on large-scale driving-data streams, and the lessons learned from the process of addressing the challenges in designing and maintaining CarStream.

AB - As the Internet-of-Vehicles (IoV) technology becomes an increasingly important trend for future transportation, designing large-scale IoV systems has become a critical task that aims to process big data uploaded by eet vehicles and to provide data-driven services. The IoV data, especially high-frequency vehicle statuses (e.g., location, engine parameters), are characterized as large volume with a low density of value and low data quality. Such characteristics pose challenges for developing real-time applications based on such data. In this paper, we address the challenges in designing a scalable IoV system by describing CarStream, an industrial system of big data processing for chaufieured car services. Connected with over 30,000 vehicles, CarStream collects and processes multiple types of driving data including vehicle status, driver activity, and passenger-trip information. Multiple services are provided based on the collected data. CarStream has been deployed and maintained for three years in industrial usage, collecting over 40 terabytes of driving data. This paper shares our experiences on designing CarStream based on large-scale driving-data streams, and the lessons learned from the process of addressing the challenges in designing and maintaining CarStream.

UR - http://www.scopus.com/inward/record.url?scp=85036660962&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85036660962&partnerID=8YFLogxK

U2 - 10.14778/3137765.3137781

DO - 10.14778/3137765.3137781

M3 - Conference article

AN - SCOPUS:85036660962

VL - 10

SP - 1766

EP - 1777

JO - Proceedings of the VLDB Endowment

JF - Proceedings of the VLDB Endowment

SN - 2150-8097

IS - 12

ER -