Title

Intelligent big data summarization for rare anomaly detection

Document Type

Journal Article

Publication Title

IEEE Access

Publisher

Institute of Electrical and Electronics Engineers

School

ECU Security Research Institute / School of Science

RAS ID

28853

Comments

Originally published as: Ahmed, M. (2019). Intelligent big data summarization for rare anomaly detection. IEEE Access, 7, 68669 - 68677. Original publication available here

Abstract

Identifying interesting patterns from a huge amount of data is a challenging task across a wide range of application domain. Especially, for cyber security being able to identify rare types of network activities or anomalies from network traffic data (a.k.a. Big Data!) is an important but time-consuming data analysis task having moderate computing resources. Existing research has shown that it is possible to detect rare anomalies from the summarized version of big data. Therefore, summarization is an effective preprocessing function before applying anomaly detection techniques. This aim of this paper is to improve and quantify the scalability and accuracy of the anomaly detection techniques by using summarization. Hence, we propose a sampling-based summarization technique (SUCh: Summarization Using Chernoff-Bound) which is computationally effective than the existing techniques and also performs better in identifying rare anomalies from twelve benchmark network traffic datasets. The experimental results show that, instead of using original dataset, a summary of the data yields better performance in terms of true positive and false positive rates, when used for anomaly detection with less time required.

DOI

10.1109/ACCESS.2019.2918364

Share

 
COinS