Building a scalable distributed online media processing environment

Shadi A. Noghabi, R H Campbell, Indranil Gupta

Research output: Contribution to journalConference article

Abstract

Media has become dominant in all aspects of human lives, from critical applications such as medical, military, and security (e.g. surveillance cameras) to entertainment applications such as social media and media sharing websites. Billions of massive media objects (e.g., videos, photos, documents, etc.) are generated every second with high diversity among them (in terms of sizes and formats). These objects have to be stored and retrieved reliably, with low latency and in a scalable while efficient fashion. Additionally, various types of processing are done on media objects, from simple compressions and format conversion, to more complex machine learning algorithms detecting certain patterns and objects. Existing large-scale storage and processing systems face several challenges when handling media objects. My research focuses on building an unified storage and processing environment tailored specifically for media objects, while maintaining high efficiency and scalability. I have built a scalable, load-balanced, efficient storage system optimized for media objects based on their unique access patterns. Currently, I am working on developing an efficient media processing system and integrating these two systems into one framework.

Original languageEnglish (US)
JournalCEUR Workshop Proceedings
Volume1671
StatePublished - Jan 1 2016
Event2016 VLDB PhD Workshop, VLDB-PhD 2016 - New Delhi, India
Duration: Sep 9 2016 → …

Fingerprint

Processing
Learning algorithms
Learning systems
Scalability
Websites
Cameras

Keywords

  • Distributed storage
  • Media processing
  • Online processing

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Building a scalable distributed online media processing environment. / Noghabi, Shadi A.; Campbell, R H; Gupta, Indranil.

In: CEUR Workshop Proceedings, Vol. 1671, 01.01.2016.

Research output: Contribution to journalConference article

@article{25f61bf15d464a5dbe04046152724a96,
title = "Building a scalable distributed online media processing environment",
abstract = "Media has become dominant in all aspects of human lives, from critical applications such as medical, military, and security (e.g. surveillance cameras) to entertainment applications such as social media and media sharing websites. Billions of massive media objects (e.g., videos, photos, documents, etc.) are generated every second with high diversity among them (in terms of sizes and formats). These objects have to be stored and retrieved reliably, with low latency and in a scalable while efficient fashion. Additionally, various types of processing are done on media objects, from simple compressions and format conversion, to more complex machine learning algorithms detecting certain patterns and objects. Existing large-scale storage and processing systems face several challenges when handling media objects. My research focuses on building an unified storage and processing environment tailored specifically for media objects, while maintaining high efficiency and scalability. I have built a scalable, load-balanced, efficient storage system optimized for media objects based on their unique access patterns. Currently, I am working on developing an efficient media processing system and integrating these two systems into one framework.",
keywords = "Distributed storage, Media processing, Online processing",
author = "Noghabi, {Shadi A.} and Campbell, {R H} and Indranil Gupta",
year = "2016",
month = "1",
day = "1",
language = "English (US)",
volume = "1671",
journal = "CEUR Workshop Proceedings",
issn = "1613-0073",
publisher = "CEUR-WS",

}

TY - JOUR

T1 - Building a scalable distributed online media processing environment

AU - Noghabi, Shadi A.

AU - Campbell, R H

AU - Gupta, Indranil

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Media has become dominant in all aspects of human lives, from critical applications such as medical, military, and security (e.g. surveillance cameras) to entertainment applications such as social media and media sharing websites. Billions of massive media objects (e.g., videos, photos, documents, etc.) are generated every second with high diversity among them (in terms of sizes and formats). These objects have to be stored and retrieved reliably, with low latency and in a scalable while efficient fashion. Additionally, various types of processing are done on media objects, from simple compressions and format conversion, to more complex machine learning algorithms detecting certain patterns and objects. Existing large-scale storage and processing systems face several challenges when handling media objects. My research focuses on building an unified storage and processing environment tailored specifically for media objects, while maintaining high efficiency and scalability. I have built a scalable, load-balanced, efficient storage system optimized for media objects based on their unique access patterns. Currently, I am working on developing an efficient media processing system and integrating these two systems into one framework.

AB - Media has become dominant in all aspects of human lives, from critical applications such as medical, military, and security (e.g. surveillance cameras) to entertainment applications such as social media and media sharing websites. Billions of massive media objects (e.g., videos, photos, documents, etc.) are generated every second with high diversity among them (in terms of sizes and formats). These objects have to be stored and retrieved reliably, with low latency and in a scalable while efficient fashion. Additionally, various types of processing are done on media objects, from simple compressions and format conversion, to more complex machine learning algorithms detecting certain patterns and objects. Existing large-scale storage and processing systems face several challenges when handling media objects. My research focuses on building an unified storage and processing environment tailored specifically for media objects, while maintaining high efficiency and scalability. I have built a scalable, load-balanced, efficient storage system optimized for media objects based on their unique access patterns. Currently, I am working on developing an efficient media processing system and integrating these two systems into one framework.

KW - Distributed storage

KW - Media processing

KW - Online processing

UR - http://www.scopus.com/inward/record.url?scp=84988967374&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84988967374&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:84988967374

VL - 1671

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

SN - 1613-0073

ER -