Document Type

Journal Article

Publication Title

Disability and Rehabilitation

Publisher

Taylor and Francis

School

School of Medical and Health Sciences

RAS ID

31822

Comments

Rohde, A., McCracken, M., Worrall, L., Farrell, A., O’Halloran, R., Godecke, E., . . . Doi, S. A. (2022). Inter-rater reliability, intra-rater reliability and internal consistency of the Brisbane Evidence-Based Language Test. Disability and Rehabilitation, 44(4), 637-645. https://doi.org/10.1080/09638288.2020.1776774

Abstract

Purpose: To examine the inter-rater reliability, intra-rater reliability, internal consistency and practice effects associated with a new test, the Brisbane Evidence-Based Language Test. Methods: Reliability estimates were obtained in a repeated-measures design through analysis of clinician video ratings of stroke participants completing the Brisbane Evidence-Based Language Test. Inter-rater reliability was determined by comparing 15 independent clinicians’ scores of 15 randomly selected videos. Intra-rater reliability was determined by comparing two clinicians’ scores of 35 videos when re-scored after a two-week interval. Results: Intraclass correlation coefficient (ICC) analysis demonstrated almost perfect inter-rater reliability (0.995; 95% confidence interval: 0.990–0.998), intra-rater reliability (0.994; 95% confidence interval: 0.989–0.997) and internal consistency (Cronbach’s α = 0.940 (95% confidence interval: 0.920–1.0)). Almost perfect correlations (0.998; 95% confidence interval: 0.995–0.999) between face-to-face and video ratings were obtained. Conclusion: The Brisbane Evidence-Based Language Test demonstrates almost perfect inter-rater reliability, intra-rater reliability and internal consistency. High correlation coefficients and narrow confidence intervals demonstrated minimal practice effects with scoring or influence of years of clinical experience on test scores. Almost perfect correlations between face-to-face and video scoring methods indicate these reliability estimates have direct application to everyday practice. The test is available from brisbanetest.org. Implications for Rehabilitation The Brisbane Evidence-Based Language Test is a new measure for the assessment of acquired language disorders. The Brisbane Evidence-Based Language Test demonstrated almost perfect inter-rater reliability, intra-rater reliability and internal consistency. High reliability estimates and narrow confidence intervals indicated that test ratings vary minimally when administered by clinicians of different experience levels, or different levels of familiarity with the new measure. The test is a reliable measure of language performance for use in clinical practice and research.

DOI

10.1080/09638288.2020.1776774

Creative Commons License

Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.

Share

 
COinS