Author Identifier
Matthew Gaber: https://orcid.org/0000-0003-1684-1392
Date of Award
2025
Document Type
Thesis
Publisher
Edith Cowan University
Degree Name
Doctor of Philosophy (Integrated)
School
School of Science
First Supervisor
Mohiuddin Ahmed
Second Supervisor
Helge Janicke
Abstract
Cyber attacks are constantly evolving and becoming more frequent, where a combination of technological advancements, financial motivation, advanced evasion techniques and targeted attacks contribute to increasingly sophisticated malware. Consequently, the field of Artificial Intelligence (AI) for malware detection is a highly active area of research, but the practical implementation of AI models in production environments is advancing at a slower pace. The performance of an AI model in accurately classifying novel malware is fundamentally contingent upon the quality of the features utilized during the training process. Therefore, an effective analysis tool must be capable of forcing malware to reveal its malicious intent and subsequently extracting authentic features. Further, the development of highly accurate models requires extensive and diverse datasets comprising both malicious and benign software. This research used a quantitative design and applied experimental methodology to develop a zero day malware detection framework that incorporated five critical aspects: malware and software repositories, malware sophistication, analysis tools and techniques, feature engineering, and Deep Learning (DL). Through investigation of the individual components and subsequent integration, this research developed Peekaboo, a state-of-the-art Dynamic Binary Instrumentation (DBI) tool for genuine feature extraction, as well as Pulse and Alpha, AI based zero day malware detection frameworks that demonstrate exceptional performance, all of which significantly advance the field of cybersecurity. Perfect classification, that is 100% accuracy, precision, recall, and F1, was achieved for worms, ransomware, and APTs, clearly distinguishing them from benign software. High performance was also maintained for botnets, tools, spyware, and trojans, with accuracies above 96% and F1-scores between 97.64 and 99.01, demonstrating robust detection even for complex and utility like zero day malware. These innovations provide cutting-edge capabilities for countering malware threats, enabling rapid detection of truly new and sophisticated malware while contributing valuable knowledge and insights to the field.
DOI
10.25958/dnm2-c081
Access Note
Access to this thesis is embargoed until 1st July 2030
Recommended Citation
Gaber, M. (2025). Zero-day malware detection: Leveraging dynamic binary instrumentation and transformer models for effective real-world malware classification. Edith Cowan University. https://doi.org/10.25958/dnm2-c081