Data reduction for the scalable automated analysis of distributed darknet traffic

Michael Bailey, Evan Cooke, Farnam Jahanian, Niels Provos, Karl Rosaen, David Watson

Research output: Contribution to conferencePaper

Abstract

Threats to the privacy of users and to the availability of Internet infrastructure are evolving at a tremendous rate. To characterize these emerging threats, researchers must effectively balance monitoring the large number of hosts needed to quickly build confidence in new attacks, while still preserving the detail required to differentiate these attacks. One class of techniques that attempts to achieve this balance involves hybrid systems that combine the scalable monitoring of unused address blocks (or darknets) with forensic honeypots (or honeyfarms). In this paper we examine the properties of individual and distributed dark-nets to determine the effectiveness of building scalable hybrid systems. We show that individual darknets are dominated by a small number of sources repeating the same actions. This enables source-based techniques to be effective at reducing the number of connections to be evaluated by over 90%. We demonstrate that the dominance of locally targeted attack behavior and the limited life of random scanning hosts result in few of these sources being repeated across darknets. To achieve reductions beyond source-based approaches, we look to source-distribution based methods and expand them to include notions of local and global behavior. We show that this approach is effective at reducing the number of events by deploying it in 30 production networks during early 2005. Each of the identified events during this period represented a major globally-scoped attack including the WINS vulnerability scanning, Veritas Backup Agent vulnerability scanning, and the MySQL Worm.

Original languageEnglish (US)
Pages239-252
Number of pages14
StatePublished - Dec 1 2005
Event5th ACM SIGCOMM Conference on Internet Measurement, IMC 2005 - Berkeley, CA, United States
Duration: Oct 19 2005Oct 21 2005

Other

Other5th ACM SIGCOMM Conference on Internet Measurement, IMC 2005
CountryUnited States
CityBerkeley, CA
Period10/19/0510/21/05

Fingerprint

Data reduction
Scanning
Hybrid systems
Monitoring
Availability
Internet

ASJC Scopus subject areas

  • Software
  • Computer Networks and Communications

Cite this

Bailey, M., Cooke, E., Jahanian, F., Provos, N., Rosaen, K., & Watson, D. (2005). Data reduction for the scalable automated analysis of distributed darknet traffic. 239-252. Paper presented at 5th ACM SIGCOMM Conference on Internet Measurement, IMC 2005, Berkeley, CA, United States.

Data reduction for the scalable automated analysis of distributed darknet traffic. / Bailey, Michael; Cooke, Evan; Jahanian, Farnam; Provos, Niels; Rosaen, Karl; Watson, David.

2005. 239-252 Paper presented at 5th ACM SIGCOMM Conference on Internet Measurement, IMC 2005, Berkeley, CA, United States.

Research output: Contribution to conferencePaper

Bailey, M, Cooke, E, Jahanian, F, Provos, N, Rosaen, K & Watson, D 2005, 'Data reduction for the scalable automated analysis of distributed darknet traffic', Paper presented at 5th ACM SIGCOMM Conference on Internet Measurement, IMC 2005, Berkeley, CA, United States, 10/19/05 - 10/21/05 pp. 239-252.
Bailey M, Cooke E, Jahanian F, Provos N, Rosaen K, Watson D. Data reduction for the scalable automated analysis of distributed darknet traffic. 2005. Paper presented at 5th ACM SIGCOMM Conference on Internet Measurement, IMC 2005, Berkeley, CA, United States.
Bailey, Michael ; Cooke, Evan ; Jahanian, Farnam ; Provos, Niels ; Rosaen, Karl ; Watson, David. / Data reduction for the scalable automated analysis of distributed darknet traffic. Paper presented at 5th ACM SIGCOMM Conference on Internet Measurement, IMC 2005, Berkeley, CA, United States.14 p.
@conference{7869b7ec936247eebaae3bb85df6cb99,
title = "Data reduction for the scalable automated analysis of distributed darknet traffic",
abstract = "Threats to the privacy of users and to the availability of Internet infrastructure are evolving at a tremendous rate. To characterize these emerging threats, researchers must effectively balance monitoring the large number of hosts needed to quickly build confidence in new attacks, while still preserving the detail required to differentiate these attacks. One class of techniques that attempts to achieve this balance involves hybrid systems that combine the scalable monitoring of unused address blocks (or darknets) with forensic honeypots (or honeyfarms). In this paper we examine the properties of individual and distributed dark-nets to determine the effectiveness of building scalable hybrid systems. We show that individual darknets are dominated by a small number of sources repeating the same actions. This enables source-based techniques to be effective at reducing the number of connections to be evaluated by over 90{\%}. We demonstrate that the dominance of locally targeted attack behavior and the limited life of random scanning hosts result in few of these sources being repeated across darknets. To achieve reductions beyond source-based approaches, we look to source-distribution based methods and expand them to include notions of local and global behavior. We show that this approach is effective at reducing the number of events by deploying it in 30 production networks during early 2005. Each of the identified events during this period represented a major globally-scoped attack including the WINS vulnerability scanning, Veritas Backup Agent vulnerability scanning, and the MySQL Worm.",
author = "Michael Bailey and Evan Cooke and Farnam Jahanian and Niels Provos and Karl Rosaen and David Watson",
year = "2005",
month = "12",
day = "1",
language = "English (US)",
pages = "239--252",
note = "5th ACM SIGCOMM Conference on Internet Measurement, IMC 2005 ; Conference date: 19-10-2005 Through 21-10-2005",

}

TY - CONF

T1 - Data reduction for the scalable automated analysis of distributed darknet traffic

AU - Bailey, Michael

AU - Cooke, Evan

AU - Jahanian, Farnam

AU - Provos, Niels

AU - Rosaen, Karl

AU - Watson, David

PY - 2005/12/1

Y1 - 2005/12/1

N2 - Threats to the privacy of users and to the availability of Internet infrastructure are evolving at a tremendous rate. To characterize these emerging threats, researchers must effectively balance monitoring the large number of hosts needed to quickly build confidence in new attacks, while still preserving the detail required to differentiate these attacks. One class of techniques that attempts to achieve this balance involves hybrid systems that combine the scalable monitoring of unused address blocks (or darknets) with forensic honeypots (or honeyfarms). In this paper we examine the properties of individual and distributed dark-nets to determine the effectiveness of building scalable hybrid systems. We show that individual darknets are dominated by a small number of sources repeating the same actions. This enables source-based techniques to be effective at reducing the number of connections to be evaluated by over 90%. We demonstrate that the dominance of locally targeted attack behavior and the limited life of random scanning hosts result in few of these sources being repeated across darknets. To achieve reductions beyond source-based approaches, we look to source-distribution based methods and expand them to include notions of local and global behavior. We show that this approach is effective at reducing the number of events by deploying it in 30 production networks during early 2005. Each of the identified events during this period represented a major globally-scoped attack including the WINS vulnerability scanning, Veritas Backup Agent vulnerability scanning, and the MySQL Worm.

AB - Threats to the privacy of users and to the availability of Internet infrastructure are evolving at a tremendous rate. To characterize these emerging threats, researchers must effectively balance monitoring the large number of hosts needed to quickly build confidence in new attacks, while still preserving the detail required to differentiate these attacks. One class of techniques that attempts to achieve this balance involves hybrid systems that combine the scalable monitoring of unused address blocks (or darknets) with forensic honeypots (or honeyfarms). In this paper we examine the properties of individual and distributed dark-nets to determine the effectiveness of building scalable hybrid systems. We show that individual darknets are dominated by a small number of sources repeating the same actions. This enables source-based techniques to be effective at reducing the number of connections to be evaluated by over 90%. We demonstrate that the dominance of locally targeted attack behavior and the limited life of random scanning hosts result in few of these sources being repeated across darknets. To achieve reductions beyond source-based approaches, we look to source-distribution based methods and expand them to include notions of local and global behavior. We show that this approach is effective at reducing the number of events by deploying it in 30 production networks during early 2005. Each of the identified events during this period represented a major globally-scoped attack including the WINS vulnerability scanning, Veritas Backup Agent vulnerability scanning, and the MySQL Worm.

UR - http://www.scopus.com/inward/record.url?scp=84878677794&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84878677794&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:84878677794

SP - 239

EP - 252

ER -