DistilXIDS: Efficient, lightweight and explainable transformer-based language model for real-time network intrusion detection
Abstract
The increasing complexity of network environments and the prevalence of sophisticated cyberattacks underscore the urgent need for advanced, efficient, and lightweight Intrusion Detection Systems (IDS). In response to this demand, we introduce DistilXIDS, a transformer-based model designed specifically for intrusion detection. DistilXIDS is built on DistilBERT, a distilled variant of Bidirectional Encoder Representations from Transformers (BERT), and is fine-tuned to identify Distributed Denial of Service (DDoS) attacks. Extensive experimentation on three widely recognized cybersecurity benchmark datasets, CIC-IDS2017, CIC-DDoS2019, and UNSW-NB15, highlights DistilXIDS's outstanding classification performance. The model achieves accuracy, precision, recall, and F1-scores exceeding 99 %, with AUC-ROC scores surpassing 0.99. Importantly, it maintains low rates of false positives and false negatives, demonstrating its reliability for practical applications. In addition to classification capabilities, DistilXIDS features comprehensive real-time performance evaluations, assessing its deployability in high-throughput environments. The model exhibits low inference latency and high throughput alongside efficient GPU utilization. Scalability analyses reveal that optimal performance is achieved at moderate batch sizes, reinforcing the model's suitability for real-time intrusion detection in operational settings. To investigate the impact of class imbalance, we first evaluated a baseline model on unbalanced data. This was followed by class-weighted fine-tuning and the application of the Synthetic Minority Oversampling Technique (SMOTE), both of which significantly enhanced recall for underrepresented attack samples. Furthermore, explainability is embedded within DistilXIDS through the use of SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME). This integration facilitates feature attribution and interpretability, thereby fostering analyst trust and operational transparency. In summary, these findings illustrate the adaptability of transformer-based architectures to structured network telemetry, paving the way for robust, interpretable, and efficient IDS solutions. DistilXIDS effectively addresses the balance among computational efficiency, real-time performance, interpretability, and predictive accuracy in cybersecurity applications.
Document Type
Journal Article
Date of Publication
3-1-2026
Volume
668
Publication Title
Neurocomputing
Publisher
Elsevier
School
Centre for Securing Digital Futures
Copyright
subscription content
Comments
Ajayan, A., Kirubavathi, G., & Sarker, I. H. (2025). DistilXIDS: Efficient, lightweight and explainable transformer-based language model for real-time network intrusion detection. Neurocomputing, 668, 132398. https://doi.org/10.1016/j.neucom.2025.132398