Doctor of Philosophy
School of Mathematics and Applied Statistics
Baharun, Norhayati, Improving students' learning of statistics: the impact of web-based learning support on student outcomes, Doctor of Philosophy thesis, School of Mathematics and Applied Statistics, University of Wollongong, 2012. http://ro.uow.edu.au/theses/3618
This thesis investigated the provision of learning supports within an e-learning environment in terms of its impact on student learning outcomes in mathematics and statistics. A review of literature on relevant learning theories was used by the researcher as a way of describing and explaining the educational practices within an online learning environment and her involvement through three case studies. The two primary case studies involved two introductory level statistics subjects. These subjects were taught and co-ordinated by the same lecturer with similar assessment practices and high quality resources. A heavy emphasis was placed on the completion of tasks, the structure of which drew on a variety of theoretical perspectives (behaviourism, cognitivism, constructivism, social constructivism, connectivism, and experiential learning). The third case study involved adapting and trialling learning designs in two mathematics subjects. Mixed methodology was used to gather evaluation data in the case studies. This included survey responses, e-learning forum analysis and student tracking statistics, and document analysis in order to triangulate the outcomes at each of the four stages of Alexander and Herdberg‟s (1994) model for evaluating innovation (Design, Development, Implementation, and Institutionalisation).
The first case study addressed students' concerns about the study of statistics. The initial iteration involved 86 postgraduate students enrolled in GHMD983 August/Spring 2008 at the University of Wollongong. The analysis of responses to prompts given to students by the lecturer in the forum revealed that 58% of the students were anxious studying statistics due to their lack of mathematical skills and unpleasant past experiences when taking similar subjects. A suite of video resources on major topics was developed to support student learning and was made available via the subject's e-learning site. At the end of the session, survey responses revealed that 98% of the students favoured the use of video resources to assist them to learn and understand statistics, and that these reduced their anxieties and helped them learn and understand statistics better.
Based on the results of the first implementation in the first case study demonstrating the use of video resources potentially assisting students‟ learning of statistics and easing their anxieties, the second case study was initiated with 89 undergraduate students enrolled in the first year undergraduate subject, STAT131, March/Autumn 2009. The suite of video resources was extended to include major topic areas for the first year introductory statistics subject. These resources were also provided as a learning support within the subject‟s e-learning site. At the end of the session, survey responses revealed that only 39% of the students regarded the video resources as useful for their learning. This finding contrasted with the results from postgraduates in the first case study on the worth of videos for providing learning support. However, unlike the postgraduates, the undergraduates commented that they were not aware of the existence of the video resources in the e-learning site. In contrast to the postgraduate subject, the videos were placed in a by-type resource folder rather than weekly folders.
The second iteration in the first case study in August/Spring 2009 addressed issues related to learning designs: how can the video resources be embedded more effectively within the e-learning site so as to support students‟ learning of statistics? The researcher reviewed the literature to identify an appropriate learning design framework and representation to more effectively embed the support resources within an online learning environment. On the basis of Oliver and Herrington‟s (2001) framework, a Learning Design Visual Sequence (LDVS) representation (developed from the Learning Designs Project available at http://www.learning designs.uow.edu.au/) was adapted for this study. This led to the implementation of learning design maps within weekly folders embedded in the e-learning system. As a result, learning design maps (in DOC files) were implemented in the subject with the maps placed in the subject‟s e-learning homepage within weekly folders, and the students were provided with by-type resource folders. Outcomes from the surveys conducted at the end of the session revealed that the video resources were still considered to be of great importance for student learning by 96% of the students, but there were still some technical issues to resolve.
In the third implementation of SHS940 August/Spring 2010, resources were provided in the e-learning site via two designs: the learning design maps (in PDF files) within the weekly folders, and by-type resource folders. Data from the e-learning student tracking in week 12 suggested that students preferred to use both designs rather than just having the maps within weekly folders. Sixty one per cent of the students had used both the by-type resource folders and the learning design maps within weekly folders to access resources in the e-learning system but none of them used only the learning design maps within weekly folders to access resources. In August/Spring 2011, both the weekly folders without the maps (but contained all the components that the maps suggested, i.e. resources, tasks, and supports) and the by-type resource folders were provided to students in the e-learning site. There were changes in the assessment system with the inclusion of a draft and redraft approach to the first two assignments in addition to a test retest approach of the remaining three assessments. The final outcomes demonstrated that the proportion of students attaining top grades (High Distinction and Distinction) in 2008 (67%) was significantly higher (Z = 3.63, p < 0.001) than in 2009 (37%). There was a significant increase in the (83%) between 2010 and 2011 (Z = 4.72, p < 0.001) compared to the overall top grades rate (54%) between 2008 and 2009. On average, the proportion of students who failed in 2010 and 2011 overall top grades rate was approximately 2% compared to 4% both in 2008 and 2009. From this implementation, it appeared that the provision of both designs: (i) weekly folders associated with resources, tasks, and supports, and (ii) by-type resource folders, along with the draft and redraft of assignments for students to sufficiently support their learning of the subject, led to the achievement of high grades with few failures. More importantly, students reported far less anxiety than when the study was initiated.
The second implementation in the second case study in March/Autumn 2010 also addressed the issue of subject design specifically on how the resources can be displayed and delivered effectively within the e-learning system so that they would be beneficial for all students. In this study, there was a focus on reducing high failure rates. Design improvements included the use of maps in HTML files within weekly folders in addition to by-type resource folders. The CAOS test (Comprehensive Assessment of Outcomes in Statistics from https://app.gen.umn.edu/artist/) was administered in the subject at the beginning (pre-test) and the end of the session (post-test). This test was used as an external measure to assess the students‟ basic statistical literacy and reasoning in STAT131.
While students welcomed the video resources and new designs, failure rates remained high. As a consequence, in March/Autumn 2011 a Headstart program was introduced in STAT131. This program allowed students to access the first module of work (including the first assessment) in the e-learning site four weeks prior to the start of the formal session. The use of videos was integral to the design of the Headstart program. In regard to the impact of the Headstart program introduced in the subject, a one-way ANOVA and Scheffe post-hoc tests revealed a statistically significant difference (F2, 192 = 5.301, p = 0.006) in the mean of final marks between the students who engaged with the program (72 marks) compared to those who did not (63 marks). An analysis of differences between two proportions showed that the proportion of students who perceived the video resources as useful for learning increased significantly (Z = 1.91, p = 0.028) from 39% in 2009 to almost 60% in 2011. An analysis of student outcomes showed that students with access to the Headstart program including assessment involving draft and redraft of assignment, the improved learning design maps within weekly folders, and the by-type resource folders performed better in their assessment tasks compared to students in previous years without access to the Headstart program, the draft and redraft of assignment, and the improved learning design in the e-learning sites. A test for differences in proportions also showed that the failure rate fell significantly (Z = 1.99, p = 0.023) in 2011 (13%) compared to the overall failure rate between 2005 and 2010 (19%).
The third case study examined the effectiveness of the subject design and the applicability of learning design maps in a different context involving prospective primary and early childhood teachers throughout three consecutive teaching sessions. These students were enrolled in a pair of mathematics subjects, MATH131 and MATH132, at the main campus of the University of Wollongong and at four remote campuses. Both subjects investigated high quality resources as confirmed with a change evaluation and measure of four dimensions of high quality learning (student engagement, acknowledgement of the learning context, challenging students, and providing practice) and used a continuous assessment system including online practice quizzes considered to be one of the most important resources for learning throughout the three teaching sessions. Several assessments were regarded as being extremely useful for students and these included assignments (87%), quizzes (90%), and mid-session exam (82%). The failure rate was consistently under 10% since March/Autumn 2009. The subject design evaluated encompassed the delivery of online resources via the fortnightly learning design maps and by-type resource folders, as well as the assessment system. An analysis of data from the e-learning student tracking statistics showed that students vary in their preference for accessing resources with 30% of the students in MATH132 using the fortnightly learning design maps, while 57% using the by-type resource folders to access resources provided in the e-learning site. In MATH131, 21% of the students used the fortnightly learning design maps, and 51% used the by-type resource folders. This suggested that students preferred to use the by-type resource folders rather than the fortnightly learning design maps to access resources in the e-learning system. However, a test for differences in two proportions showed that the proportion of access to resources in the e-learning system via the fortnightly learning design maps in MATH131 (47%) was significantly higher (Z = 9.43, p < 0.0001) than in MATH132 (27%).
The final stage of institutionalisation demonstrated that the university staffs have adapted the approach of the learning design maps in teaching and learning in other disciplines (SCIE911, Fundamentals of Science Communication). Two further mathematics subjects with high failure rates are preparing to implement Headstart programs. This thesis concluded with a view to the future technological learning supports in statistics education and made suggestions for improvement. The exploration of these three case studies has reinforced the identified needs for an embedded learning support system to be built upon high quality learning resources, a redemptive assessment system, and good video and print support resources. Several approaches to assessment including the test and retest, draft and redraft of assignment, minor and major assessments, and practice quizzes appeared useful for supporting students‟ learning and for creating an online learning support system wherein students at risk are identified for further learning support purposes. Learning designs need to vary with students‟ learning context, so that students can optimally locate resources. Students have strong preferences in how they wish to access resources. The recommendations from this study would be to include both by-type resource folders and weekly (topic) folders aligning the primary high quality resources, tasks, and supports. A learning support system may not be all that is required, with constant gathering of evidence to direct change being necessary. In this study, evidence pointed to the adoption of an innovative Headstart program which allowed more students to successfully complete their studies.