Evaluating the validity of the online multiliteracy assessment tool

Authors

Kellie Buckley-Walker, University of WollongongFollow
Jim S. Tognolini, University of SydneyFollow
Lori Lockyer, University of Technology sydneyFollow
Ian M. Brown, University of WollongongFollow
Peter Caputi, University of WollongongFollow

RIS ID

117820

Publication Details

Buckley-Walker, K., Tognolini, J., Lockyer, L., Brown, I. & Caputi, P. (2017). Evaluating the validity of the online multiliteracy assessment tool. Australian Journal of Education, 61 (3), 305-327.

Abstract

This study aims to assess the validity of the Online Multiliteracy Assessment for students in Years 5 and 6. The Online Multiliteracy Assessment measures students' abilities in making and creating meaning, using a variety of different modes of communication, such as text, audio and video. The study involved selecting two groups of students: the first group (n=19) was used in two pilot studies of the items and the second (n=299) was used in a field trial validating the functioning of the items and assessing the quality of the scale. The results indicated that the Online Multiliteracy Assessment has acceptable test-retest reliability; however, the fit to the Rasch model was less than ideal. Further investigation identified two important areas for improvement. First, the items assessing the higher order skills of synthesising, communicating and creating need to be more cognitively demanding. Second, some items need to be modified in order to improve their functionality.