Scaling data-plane logging in large scale networks

Ahsan Arefin, Ahmed Khurshid, Matthew Caesar, Klara Nahrstedt

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Understanding and troubleshooting wide area networks (such as military backbone networks and ISP networks) are challenging tasks due to their large, distributed, and highly dynamic nature. Building a system that can record and replay fine-grained behaviors of such networks would simplify this problem by allowing operators to recreate the sequence and precise ordering of events (e.g., packet-level forwarding decisions, route changes, failures) taking place in their networks. However, doing this at large scales seems intractable due to the vast amount of information that would need to be logged. In this paper, we propose a scalable and reliable framework to monitor fine-grained data-plane behavior within a large network. We give a feasible architecture for a distributed logging facility, a tree-based data structure for log compression and show how this logged information helps network operators to detect and debug anomalous behavior of the network. Experimental results obtained through trace-driven simulations and Click software router experiments show that our design is lightweight in terms of processing time, memory requirement and control overhead, yet still achieves over 99% precision in capturing network events.

Original languageEnglish (US)
Title of host publication2010 Military Communications Conference, MILCOM 2010
Pages1308-1314
Number of pages7
DOIs
StatePublished - Dec 1 2011
Event2011 IEEE Military Communications Conference, MILCOM 2011 - Baltimore, MD, United States
Duration: Nov 7 2011Nov 10 2011

Publication series

NameProceedings - IEEE Military Communications Conference MILCOM

Other

Other2011 IEEE Military Communications Conference, MILCOM 2011
CountryUnited States
CityBaltimore, MD
Period11/7/1111/10/11

Fingerprint

Wide area networks
Routers
Data structures
Data storage equipment
Processing
Experiments

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Arefin, A., Khurshid, A., Caesar, M., & Nahrstedt, K. (2011). Scaling data-plane logging in large scale networks. In 2010 Military Communications Conference, MILCOM 2010 (pp. 1308-1314). [6127483] (Proceedings - IEEE Military Communications Conference MILCOM). https://doi.org/10.1109/MILCOM.2011.6127483

Scaling data-plane logging in large scale networks. / Arefin, Ahsan; Khurshid, Ahmed; Caesar, Matthew; Nahrstedt, Klara.

2010 Military Communications Conference, MILCOM 2010. 2011. p. 1308-1314 6127483 (Proceedings - IEEE Military Communications Conference MILCOM).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Arefin, A, Khurshid, A, Caesar, M & Nahrstedt, K 2011, Scaling data-plane logging in large scale networks. in 2010 Military Communications Conference, MILCOM 2010., 6127483, Proceedings - IEEE Military Communications Conference MILCOM, pp. 1308-1314, 2011 IEEE Military Communications Conference, MILCOM 2011, Baltimore, MD, United States, 11/7/11. https://doi.org/10.1109/MILCOM.2011.6127483
Arefin A, Khurshid A, Caesar M, Nahrstedt K. Scaling data-plane logging in large scale networks. In 2010 Military Communications Conference, MILCOM 2010. 2011. p. 1308-1314. 6127483. (Proceedings - IEEE Military Communications Conference MILCOM). https://doi.org/10.1109/MILCOM.2011.6127483
Arefin, Ahsan ; Khurshid, Ahmed ; Caesar, Matthew ; Nahrstedt, Klara. / Scaling data-plane logging in large scale networks. 2010 Military Communications Conference, MILCOM 2010. 2011. pp. 1308-1314 (Proceedings - IEEE Military Communications Conference MILCOM).
@inproceedings{f1b113bf0b3145c2a97661a871c5f006,
title = "Scaling data-plane logging in large scale networks",
abstract = "Understanding and troubleshooting wide area networks (such as military backbone networks and ISP networks) are challenging tasks due to their large, distributed, and highly dynamic nature. Building a system that can record and replay fine-grained behaviors of such networks would simplify this problem by allowing operators to recreate the sequence and precise ordering of events (e.g., packet-level forwarding decisions, route changes, failures) taking place in their networks. However, doing this at large scales seems intractable due to the vast amount of information that would need to be logged. In this paper, we propose a scalable and reliable framework to monitor fine-grained data-plane behavior within a large network. We give a feasible architecture for a distributed logging facility, a tree-based data structure for log compression and show how this logged information helps network operators to detect and debug anomalous behavior of the network. Experimental results obtained through trace-driven simulations and Click software router experiments show that our design is lightweight in terms of processing time, memory requirement and control overhead, yet still achieves over 99{\%} precision in capturing network events.",
author = "Ahsan Arefin and Ahmed Khurshid and Matthew Caesar and Klara Nahrstedt",
year = "2011",
month = "12",
day = "1",
doi = "10.1109/MILCOM.2011.6127483",
language = "English (US)",
isbn = "9781467300810",
series = "Proceedings - IEEE Military Communications Conference MILCOM",
pages = "1308--1314",
booktitle = "2010 Military Communications Conference, MILCOM 2010",

}

TY - GEN

T1 - Scaling data-plane logging in large scale networks

AU - Arefin, Ahsan

AU - Khurshid, Ahmed

AU - Caesar, Matthew

AU - Nahrstedt, Klara

PY - 2011/12/1

Y1 - 2011/12/1

N2 - Understanding and troubleshooting wide area networks (such as military backbone networks and ISP networks) are challenging tasks due to their large, distributed, and highly dynamic nature. Building a system that can record and replay fine-grained behaviors of such networks would simplify this problem by allowing operators to recreate the sequence and precise ordering of events (e.g., packet-level forwarding decisions, route changes, failures) taking place in their networks. However, doing this at large scales seems intractable due to the vast amount of information that would need to be logged. In this paper, we propose a scalable and reliable framework to monitor fine-grained data-plane behavior within a large network. We give a feasible architecture for a distributed logging facility, a tree-based data structure for log compression and show how this logged information helps network operators to detect and debug anomalous behavior of the network. Experimental results obtained through trace-driven simulations and Click software router experiments show that our design is lightweight in terms of processing time, memory requirement and control overhead, yet still achieves over 99% precision in capturing network events.

AB - Understanding and troubleshooting wide area networks (such as military backbone networks and ISP networks) are challenging tasks due to their large, distributed, and highly dynamic nature. Building a system that can record and replay fine-grained behaviors of such networks would simplify this problem by allowing operators to recreate the sequence and precise ordering of events (e.g., packet-level forwarding decisions, route changes, failures) taking place in their networks. However, doing this at large scales seems intractable due to the vast amount of information that would need to be logged. In this paper, we propose a scalable and reliable framework to monitor fine-grained data-plane behavior within a large network. We give a feasible architecture for a distributed logging facility, a tree-based data structure for log compression and show how this logged information helps network operators to detect and debug anomalous behavior of the network. Experimental results obtained through trace-driven simulations and Click software router experiments show that our design is lightweight in terms of processing time, memory requirement and control overhead, yet still achieves over 99% precision in capturing network events.

UR - http://www.scopus.com/inward/record.url?scp=84856954443&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84856954443&partnerID=8YFLogxK

U2 - 10.1109/MILCOM.2011.6127483

DO - 10.1109/MILCOM.2011.6127483

M3 - Conference contribution

AN - SCOPUS:84856954443

SN - 9781467300810

T3 - Proceedings - IEEE Military Communications Conference MILCOM

SP - 1308

EP - 1314

BT - 2010 Military Communications Conference, MILCOM 2010

ER -