Using IT to assess IT: Towards greater authenticity in summative performance assessment

Christopher Paul Newhouse, Edith Cowan University

This article was originally published as: Newhouse, C. P. (2011). Using IT to assess IT: Towards greater authenticity in summative performance assessment. Computers and Education, 56(2), 388-402. Original article available here

Abstract

An applied Information Technology (IT) course that is assessed using pen and paper may sound incongruous but it is symptomatic of the state of high-stakes assessment in jurisdictions such as Western Australia. Whereas technology has permeated most aspects of modern life, including schooling, and more has been demanded of education systems in terms of outcomes and participation, methods of summativeassessment have changed little and are seriously out of alignment with curriculum, pedagogy and the needs of individuals and society. This paper reports on an analysis of some of the data from a component of a study into the feasibility of using digital technologies to achieve greaterauthenticity in summativeperformanceassessment in the Applied Information Technology (AIT) course in Western Australian secondary schools. In the first phase of the study a sample of 115 students completed a digital portfolio and a computer-based exam that were both externally assessed using online tools and by two methods of marking, with the results analysed using Rasch modelling software. A traditional analytical method and a comparative pairs method of marking were investigated. The study found that both the digital portfolio and computer-based exam were implemented without significant technical difficulty and were well accepted by the students and teachers. The work output in digital form was readily accessed from an online repository by external markers using a standard web browser. The two methods of marking provided highly reliable scores, with those from the comparative pairs method being the more reliable. A number of questions of validity and manageability were raised and the strengths and weaknesses of the two forms of assessment revealed. It was concluded that it was feasible to implement either form of assessment for high-stakes purposes, with a resulting improvement in alignment and authenticity.

 

Link to publisher version (DOI)

10.1016/j.compedu.2010.08.023