UrbanFlow

Large-scale framework to integrate social media and authoritative landuse maps

Kiumars Soltani, Aiman Soliman, Anand Padmanabhan, Shaowen Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Everyday massive amounts of geo-tagged information are generated around urban environment using micro-blogging services and content sharing platforms. These new Big Geospatial Data sources provide an opportunity to understand people activities and their interaction with the urban environment. In this regard, it is crucial to integrate geo-tagged micro-data with more authoritative sources such as official landuse maps. This integration would benefit the urban research community by combining real time information about people activities and their spatial interaction with the synoptic view of physical infrastructure as depicted in official landuse maps. However, the scientific effort for integrating heterogeneous data sources is hindered by the lack of scalable Geospatial synthesis capabilities to accommodate the massive volume and fast update of microdata. We developed UrbanFlow, a platform to integrate geolocated Twitter data with detailed landuse map (parcel level) to detect and analyze individual human mobility patterns. The platform provides scientists with a set of tools to extract key locations of each Twitter user, assess the extraction quality and uncertainty, and analyze city neighbors' connectivity based on detected users' frequent visitation patterns. These capabilities are built on a novel scalable solution for the point in/nearest polygon algorithm, implemented on Hadoop to harness the power of distributed systems to combine massive point data and large number of polygon in scale-out fashion. Our results showed that we are able to effectively process large data stream of Twitter data (2.42 billion tweets) and synthesize that with highly detailed landuse map (468,641 parcels for the city of Chicago).

Original languageEnglish (US)
Title of host publicationProceedings of XSEDE 2016
Subtitle of host publicationDiversity, Big Data, and Science at Scale
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450347556
DOIs
StatePublished - Jul 17 2016
EventConference on Diversity, Big Data, and Science at Scale, XSEDE 2016 - Miami, United States
Duration: Jul 17 2016Jul 21 2016

Publication series

NameACM International Conference Proceeding Series
Volume17-21-July-2016

Other

OtherConference on Diversity, Big Data, and Science at Scale, XSEDE 2016
CountryUnited States
CityMiami
Period7/17/167/21/16

Fingerprint

Uncertainty
Big data

Keywords

  • Cybergis
  • Data synthesis
  • Hadoop
  • Interactive visualization
  • Point in polygon
  • Social media
  • Urban mobility

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Soltani, K., Soliman, A., Padmanabhan, A., & Wang, S. (2016). UrbanFlow: Large-scale framework to integrate social media and authoritative landuse maps. In Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale [a2] (ACM International Conference Proceeding Series; Vol. 17-21-July-2016). Association for Computing Machinery. https://doi.org/10.1145/2949550.2949578

UrbanFlow : Large-scale framework to integrate social media and authoritative landuse maps. / Soltani, Kiumars; Soliman, Aiman; Padmanabhan, Anand; Wang, Shaowen.

Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale. Association for Computing Machinery, 2016. a2 (ACM International Conference Proceeding Series; Vol. 17-21-July-2016).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Soltani, K, Soliman, A, Padmanabhan, A & Wang, S 2016, UrbanFlow: Large-scale framework to integrate social media and authoritative landuse maps. in Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale., a2, ACM International Conference Proceeding Series, vol. 17-21-July-2016, Association for Computing Machinery, Conference on Diversity, Big Data, and Science at Scale, XSEDE 2016, Miami, United States, 7/17/16. https://doi.org/10.1145/2949550.2949578
Soltani K, Soliman A, Padmanabhan A, Wang S. UrbanFlow: Large-scale framework to integrate social media and authoritative landuse maps. In Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale. Association for Computing Machinery. 2016. a2. (ACM International Conference Proceeding Series). https://doi.org/10.1145/2949550.2949578
Soltani, Kiumars ; Soliman, Aiman ; Padmanabhan, Anand ; Wang, Shaowen. / UrbanFlow : Large-scale framework to integrate social media and authoritative landuse maps. Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale. Association for Computing Machinery, 2016. (ACM International Conference Proceeding Series).
@inproceedings{e4cb3e96cb404d20b34acd688a9ecf37,
title = "UrbanFlow: Large-scale framework to integrate social media and authoritative landuse maps",
abstract = "Everyday massive amounts of geo-tagged information are generated around urban environment using micro-blogging services and content sharing platforms. These new Big Geospatial Data sources provide an opportunity to understand people activities and their interaction with the urban environment. In this regard, it is crucial to integrate geo-tagged micro-data with more authoritative sources such as official landuse maps. This integration would benefit the urban research community by combining real time information about people activities and their spatial interaction with the synoptic view of physical infrastructure as depicted in official landuse maps. However, the scientific effort for integrating heterogeneous data sources is hindered by the lack of scalable Geospatial synthesis capabilities to accommodate the massive volume and fast update of microdata. We developed UrbanFlow, a platform to integrate geolocated Twitter data with detailed landuse map (parcel level) to detect and analyze individual human mobility patterns. The platform provides scientists with a set of tools to extract key locations of each Twitter user, assess the extraction quality and uncertainty, and analyze city neighbors' connectivity based on detected users' frequent visitation patterns. These capabilities are built on a novel scalable solution for the point in/nearest polygon algorithm, implemented on Hadoop to harness the power of distributed systems to combine massive point data and large number of polygon in scale-out fashion. Our results showed that we are able to effectively process large data stream of Twitter data (2.42 billion tweets) and synthesize that with highly detailed landuse map (468,641 parcels for the city of Chicago).",
keywords = "Cybergis, Data synthesis, Hadoop, Interactive visualization, Point in polygon, Social media, Urban mobility",
author = "Kiumars Soltani and Aiman Soliman and Anand Padmanabhan and Shaowen Wang",
year = "2016",
month = "7",
day = "17",
doi = "10.1145/2949550.2949578",
language = "English (US)",
series = "ACM International Conference Proceeding Series",
publisher = "Association for Computing Machinery",
booktitle = "Proceedings of XSEDE 2016",

}

TY - GEN

T1 - UrbanFlow

T2 - Large-scale framework to integrate social media and authoritative landuse maps

AU - Soltani, Kiumars

AU - Soliman, Aiman

AU - Padmanabhan, Anand

AU - Wang, Shaowen

PY - 2016/7/17

Y1 - 2016/7/17

N2 - Everyday massive amounts of geo-tagged information are generated around urban environment using micro-blogging services and content sharing platforms. These new Big Geospatial Data sources provide an opportunity to understand people activities and their interaction with the urban environment. In this regard, it is crucial to integrate geo-tagged micro-data with more authoritative sources such as official landuse maps. This integration would benefit the urban research community by combining real time information about people activities and their spatial interaction with the synoptic view of physical infrastructure as depicted in official landuse maps. However, the scientific effort for integrating heterogeneous data sources is hindered by the lack of scalable Geospatial synthesis capabilities to accommodate the massive volume and fast update of microdata. We developed UrbanFlow, a platform to integrate geolocated Twitter data with detailed landuse map (parcel level) to detect and analyze individual human mobility patterns. The platform provides scientists with a set of tools to extract key locations of each Twitter user, assess the extraction quality and uncertainty, and analyze city neighbors' connectivity based on detected users' frequent visitation patterns. These capabilities are built on a novel scalable solution for the point in/nearest polygon algorithm, implemented on Hadoop to harness the power of distributed systems to combine massive point data and large number of polygon in scale-out fashion. Our results showed that we are able to effectively process large data stream of Twitter data (2.42 billion tweets) and synthesize that with highly detailed landuse map (468,641 parcels for the city of Chicago).

AB - Everyday massive amounts of geo-tagged information are generated around urban environment using micro-blogging services and content sharing platforms. These new Big Geospatial Data sources provide an opportunity to understand people activities and their interaction with the urban environment. In this regard, it is crucial to integrate geo-tagged micro-data with more authoritative sources such as official landuse maps. This integration would benefit the urban research community by combining real time information about people activities and their spatial interaction with the synoptic view of physical infrastructure as depicted in official landuse maps. However, the scientific effort for integrating heterogeneous data sources is hindered by the lack of scalable Geospatial synthesis capabilities to accommodate the massive volume and fast update of microdata. We developed UrbanFlow, a platform to integrate geolocated Twitter data with detailed landuse map (parcel level) to detect and analyze individual human mobility patterns. The platform provides scientists with a set of tools to extract key locations of each Twitter user, assess the extraction quality and uncertainty, and analyze city neighbors' connectivity based on detected users' frequent visitation patterns. These capabilities are built on a novel scalable solution for the point in/nearest polygon algorithm, implemented on Hadoop to harness the power of distributed systems to combine massive point data and large number of polygon in scale-out fashion. Our results showed that we are able to effectively process large data stream of Twitter data (2.42 billion tweets) and synthesize that with highly detailed landuse map (468,641 parcels for the city of Chicago).

KW - Cybergis

KW - Data synthesis

KW - Hadoop

KW - Interactive visualization

KW - Point in polygon

KW - Social media

KW - Urban mobility

UR - http://www.scopus.com/inward/record.url?scp=84989195580&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84989195580&partnerID=8YFLogxK

U2 - 10.1145/2949550.2949578

DO - 10.1145/2949550.2949578

M3 - Conference contribution

T3 - ACM International Conference Proceeding Series

BT - Proceedings of XSEDE 2016

PB - Association for Computing Machinery

ER -