Date of Award


Degree Type


Degree Name

Bachelor of Science (Security) Honours


School of Science

First Advisor

Mike Johnstone


Cyber security is fast becoming a strategic priority across both governments and private organisations. With technology abundantly available, and the unbridled growth in the size and complexity of information systems, cyber criminals have a multitude of targets. Therefore, cyber security assessments are becoming common practice as concerns about information security grow. Penetration testing is one strategy used to mitigate the risk of cyber-attack. Penetration testers attempt to compromise systems using the same tools and techniques as malicious attackers thus, aim to identify vulnerabilities before an attack occurs. Penetration testing can be complex depending on the scope and domain area under investigation, for this reason it is often managed similarly to that of a project necessitating the implementation of some framework or methodology. Fortunately, there are an array of penetration testing methodologies and frameworks available to facilitate such projects, however, determining what is a framework and what is methodology within this context can lend itself to uncertainty. Furthermore, little exists in relation to mature frameworks whereby quality can be measured. This research defines the concept of “methodology” and “framework” within a penetration testing context. In addition, the research presents a gap analysis of the theoretical vs. the practical classification of nine penetration testing frameworks and/or methodologies and subsequently selects two frameworks to undergo quality evaluation using a realworld case study. Quality characteristics were derived from a review of four quality models, thus building the foundation for a proposed penetration testing quality model. The penetration testing quality model is a modified version of an ISO quality model whereby the two chosen frameworks underwent quality evaluation.

Defining methodologies and frameworks for the purposes of penetration testing was achieved. A suitable definition was formed by way of analysing properties of each category respectively, thus a Framework vs. Methodology Characteristics matrix is presented. Extending upon the nomenclature resolution, a gap analysis was performed to determine if a framework is actually a framework, i.e., it has a sound underlying ontology. In contrast, many “frameworks” appear to be simply collections of tools or techniques. In addition, two frameworks OWASP’s Testing Guide and Information System Security Assessment Framework (ISSAF), were employed to perform penetration tests based on a real-world case study to facilitate quality evaluation based on a proposed quality model. The research suggests there are various ways in which quality for penetration testing frameworks can be measured; therefore concluded that quality evaluation is possible.


Shanley, A., & Johnstone, M. N. (2015). Selection of penetration testing methodologies: A comparison and evaluation. 13th Australian Information Security Management Conference, held from the 30 November – 2 December, 2015 (pp. 65-72), Edith Cowan University Joondalup Campus, Perth, Western Australia.