Intelligent big data summarization for rare anomaly detection
Document Type
Journal Article
Publication Title
IEEE Access
Publisher
Institute of Electrical and Electronics Engineers
School
ECU Security Research Institute / School of Science
RAS ID
28853
Abstract
Identifying interesting patterns from a huge amount of data is a challenging task across a wide range of application domain. Especially, for cyber security being able to identify rare types of network activities or anomalies from network traffic data (a.k.a. Big Data!) is an important but time-consuming data analysis task having moderate computing resources. Existing research has shown that it is possible to detect rare anomalies from the summarized version of big data. Therefore, summarization is an effective preprocessing function before applying anomaly detection techniques. This aim of this paper is to improve and quantify the scalability and accuracy of the anomaly detection techniques by using summarization. Hence, we propose a sampling-based summarization technique (SUCh: Summarization Using Chernoff-Bound) which is computationally effective than the existing techniques and also performs better in identifying rare anomalies from twelve benchmark network traffic datasets. The experimental results show that, instead of using original dataset, a summary of the data yields better performance in terms of true positive and false positive rates, when used for anomaly detection with less time required.
DOI
10.1109/ACCESS.2019.2918364
Access Rights
subscription content
Comments
Ahmed, M. (2019). Intelligent big data summarization for rare anomaly detection. IEEE Access, 7, 68669 - 68677. Available here