Knowledge discovery from big data for intrusion detection using LDA

Jingwei Huang; Zbigniew T Kalbarczyk; David Malcolm Nicol

doi:10.1109/BigData.Congress.2014.111

Knowledge discovery from big data for intrusion detection using LDA

Jingwei Huang, Zbigniew T Kalbarczyk, David Malcolm Nicol

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper explores a hybrid approach of intrusion detection through knowledge discovery from big data using Latent Dirichlet Allocation (LDA). We identify the 'hidden' patterns of operations conducted by both normal users and malicious users from a large volume of network/systems logs, by mapping this problem to the topic modeling problem and leveraging the well established LDA models and learning algorithms. This new approach potentially completes the strength of signature-based and anomaly-based methods.

Original language	English (US)
Title of host publication	Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014
Editors	Peter Chen, Peter Chen, Hemant Jain
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	760-761
Number of pages	2
ISBN (Electronic)	9781479950577
DOIs	https://doi.org/10.1109/BigData.Congress.2014.111
State	Published - Sep 22 2014
Event	3rd IEEE International Congress on Big Data, BigData Congress 2014 - Anchorage, United States Duration: Jun 27 2014 → Jul 2 2014

Publication series

Name	Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014

Other

Other	3rd IEEE International Congress on Big Data, BigData Congress 2014
Country	United States
City	Anchorage
Period	6/27/14 → 7/2/14

Keywords

LDA
big data
data mining
intrusion detection

ASJC Scopus subject areas

Computer Science Applications

Access to Document

10.1109/BigData.Congress.2014.111

Fingerprint Dive into the research topics of 'Knowledge discovery from big data for intrusion detection using LDA'. Together they form a unique fingerprint.

Cite this

Huang, J., Kalbarczyk, Z. T., & Nicol, D. M. (2014). Knowledge discovery from big data for intrusion detection using LDA. In P. Chen, P. Chen, & H. Jain (Eds.), Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014 (pp. 760-761). [6906855] (Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.Congress.2014.111

Knowledge discovery from big data for intrusion detection using LDA. / Huang, Jingwei; Kalbarczyk, Zbigniew T ; Nicol, David Malcolm.

Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014. ed. / Peter Chen; Peter Chen; Hemant Jain. Institute of Electrical and Electronics Engineers Inc., 2014. p. 760-761 6906855 (Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Huang, J, Kalbarczyk, ZT & Nicol, DM 2014, Knowledge discovery from big data for intrusion detection using LDA. in P Chen, P Chen & H Jain (eds), Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014., 6906855, Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014, Institute of Electrical and Electronics Engineers Inc., pp. 760-761, 3rd IEEE International Congress on Big Data, BigData Congress 2014, Anchorage, United States, 6/27/14. https://doi.org/10.1109/BigData.Congress.2014.111

Huang J, Kalbarczyk ZT , Nicol DM. Knowledge discovery from big data for intrusion detection using LDA. In Chen P, Chen P, Jain H, editors, Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014. Institute of Electrical and Electronics Engineers Inc. 2014. p. 760-761. 6906855. (Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014). https://doi.org/10.1109/BigData.Congress.2014.111

Huang, Jingwei ; Kalbarczyk, Zbigniew T ; Nicol, David Malcolm. / Knowledge discovery from big data for intrusion detection using LDA. Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014. editor / Peter Chen ; Peter Chen ; Hemant Jain. Institute of Electrical and Electronics Engineers Inc., 2014. pp. 760-761 (Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014).

@inproceedings{a3d1b067c53042b0abefa4616791a3ec,

title = "Knowledge discovery from big data for intrusion detection using LDA",

abstract = "This paper explores a hybrid approach of intrusion detection through knowledge discovery from big data using Latent Dirichlet Allocation (LDA). We identify the 'hidden' patterns of operations conducted by both normal users and malicious users from a large volume of network/systems logs, by mapping this problem to the topic modeling problem and leveraging the well established LDA models and learning algorithms. This new approach potentially completes the strength of signature-based and anomaly-based methods.",

keywords = "LDA, big data, data mining, intrusion detection",

author = "Jingwei Huang and Kalbarczyk, {Zbigniew T} and Nicol, {David Malcolm}",

year = "2014",

month = sep,

day = "22",

doi = "10.1109/BigData.Congress.2014.111",

language = "English (US)",

series = "Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "760--761",

editor = "Peter Chen and Peter Chen and Hemant Jain",

booktitle = "Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014",

address = "United States",

note = "3rd IEEE International Congress on Big Data, BigData Congress 2014 ; Conference date: 27-06-2014 Through 02-07-2014",

}

TY - GEN

T1 - Knowledge discovery from big data for intrusion detection using LDA

AU - Huang, Jingwei

AU - Kalbarczyk, Zbigniew T

AU - Nicol, David Malcolm

PY - 2014/9/22

Y1 - 2014/9/22

N2 - This paper explores a hybrid approach of intrusion detection through knowledge discovery from big data using Latent Dirichlet Allocation (LDA). We identify the 'hidden' patterns of operations conducted by both normal users and malicious users from a large volume of network/systems logs, by mapping this problem to the topic modeling problem and leveraging the well established LDA models and learning algorithms. This new approach potentially completes the strength of signature-based and anomaly-based methods.

AB - This paper explores a hybrid approach of intrusion detection through knowledge discovery from big data using Latent Dirichlet Allocation (LDA). We identify the 'hidden' patterns of operations conducted by both normal users and malicious users from a large volume of network/systems logs, by mapping this problem to the topic modeling problem and leveraging the well established LDA models and learning algorithms. This new approach potentially completes the strength of signature-based and anomaly-based methods.

KW - LDA

KW - big data

KW - data mining

KW - intrusion detection

UR - http://www.scopus.com/inward/record.url?scp=84923865799&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84923865799&partnerID=8YFLogxK

U2 - 10.1109/BigData.Congress.2014.111

DO - 10.1109/BigData.Congress.2014.111

M3 - Conference contribution

AN - SCOPUS:84923865799

T3 - Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014

SP - 760

EP - 761

BT - Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014

A2 - Chen, Peter

A2 - Jain, Hemant

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 3rd IEEE International Congress on Big Data, BigData Congress 2014

Y2 - 27 June 2014 through 2 July 2014

ER -