Year

2017

Degree Name

Doctor of Philosophy

Department

School of Mathematics and Applied Statistics

Abstract

This thesis explores methods to address non-response in surveys of Australian school students. These surveys are subject to non-response, resulting in losses in precision and the possibility of non-response bias. Non-response bias is potentially a major source of survey error and protection from this risk is an important consideration in the design of a survey and in the estimation of results. Following a review of the literature of the methods most commonly used to adjust for non-response in surveys, including weighting adjustments, post-stratification, propensity estimation, the use of a generalised regression estimator and regression-based imputation methods, the thesis examines patterns of non-response observed in Australia’s participation in a major international comparative survey, the Trends in International Mathematics and Science (TIMSS) survey conducted in 2011.

The thesis examines factors related to non-response in TIMSS as well as factors that explain mathematics achievement, one of the major outcomes of this survey. A major focus of the research is whether the incorporation of data collected through Australia’s annual census testing of students - the ‘NAPLAN’ assessments – might be used for improving the management of non-response for TIMSS or other such surveys. An additional source of data it will explore for this purpose is MySchool, a web-based resource developed by the Australian Government to assist the parents and other community members to monitor and compare schools. To date these data sources have not been made available for the management of non-response.

The thesis explores non-response at both school and student levels, reflecting the most commonly used sample designs used in educational surveys and examines and compares adjustment methods across a range of scenarios involving differing rates and complexities of non-response. For non-response at the school level, the widely used practice of substitution of non-responding schools performs well under the scenarios investigated, as do regression based adjustment methods such as the use of a generalised regression estimator and the regression based imputation. For the investigation of student-level non-response, a simulated sub-population of Australian students is constructed using TIMSS survey data.

The investigations in the thesis demonstrate that patterns of non-response are related to important background variables such as the prior performance profile of schools and students and socio-economic background. They also demonstrate that the adjustment methods currently used to address non-response are unlikely to remove non-response bias and may also result in losses of precision. Adjustment methods that make use of NAPLAN and My School data, particularly generalised regression estimation and regression-based imputation methods would likely provide strong protection against non-response bias whilst maintaining good precision on the major survey estimates.

Download

Share

COinS
 

Unless otherwise indicated, the views expressed in this thesis are those of the author and do not necessarily represent the views of the University of Wollongong.