Doctor of Philosophy
School of Mathematics and Applied Statistics - Faculty of Informatics
Morris, Maureen M, Evaluating university teaching and learning in an outcome-based model: replanting Bloom, PhD thesis, School of Mathematics and Applied Statistics, University of Wollongong, 2008. http://ro.uow.edu.au/theses/784
This thesis was inspired by an experienced teacher’s desire to enhance student learning through implementation of a teaching/learning framework focused on promotion of higher order cognition. Two case studies document the construction, implementation and evaluation of learning frameworks for two disparate undergraduate university subjects.
Structurally, the thesis falls into three component parts. In the first, the researcher has reviewed the literature for an appropriate methodology, grounded her understanding of student learning through examination of relevant learning theories, canvassed suitable pedagogical strategies before construction of the teaching/learning frameworks, and devised an evaluation framework. In the second part, the two case studies have been described and in the final part, the threads of evidence have been drawn into the conclusion.
Action research afforded an appropriate methodology for the study. It offered facility for a spiral of implementation, review and re-implementation. Bound as a practitioner by the pragmatic perspective of what works, the researcher engaged multiple methodologies (grounded research encompassing elements of phenomenology and ethnography) in both case studies. She adopted a mixed method approach, with evidence derived from assessment data, survey responses, her annotated journal and comments from collaborating teachers and students.
The researcher’s primary intent was to construct aligned teaching/learning frameworks that promoted contextualised thinking for students in the two disciplines. Judgment of the effectiveness of the resulting frameworks in enhancing student learning required a strict evaluative regimen.
Key issues percolated through the thinking of the researcher/teacher:
• life-long learning;
• meta-cognition and deeper learning; and
• marking of assessment that recognises achievement of learning objectives, offers students task related feedback and does not merely represent an aggregation of marks for ranking of students along a curve.
Therefore, strategies were included that fostered independent learning and promoted productive collaboration, while marking criteria formed the focus for aligning marking with the objectives.
The primary case study examined teaching and learning in a foundation course in statistics at the University of Wollongong in Australia. The intent was to foster statistical thinking in students. Experienced in the field, the teacher assumed an active role as a participant researcher. In consultation with discipline experts and innovative teachers, the researcher/teacher observed the existing environment for a single session (N=159). Learning objectives were then rigorously scrutinised and behaviourally reframed; objectives were specified for learning and assessment tasks; and marking criteria devised to scaffold student responses, check assessment for objective achievement and provide detailed and task related feedback. Thus the objectives formed the agents of constructive alignment.
Implementation of the selected strategies was tracked over the subsequent four sessions (cohorts ranging in size from 152 to 192 students). Evidence of student learning and the effectiveness of the framework was derived from:
• assessment marks and grades;
• deconstruction of assessment tasks and responses using the revised Bloom’s Taxonomy (Anderson and Krathwohl, 2001);
• student survey responses;
• teacher and marker survey responses;
• the researcher’s journal, annotated by collaborative teachers; and
• peer discussions.
Results have highlighted increases in mean marks in summative assessment accompanied by shifts to higher order cognitive demand in assessment tasks across the implementations. Furthermore, strong correlations between proportions of students reporting confidence in topic learning and exam performance have lent credence to the teacher’s claim that students know what they know and know what they do not know.
The aim of the second case study was the design and implementation of an aligned curriculum for a subject focused on promoting critical and evaluative thinking in undergraduate accounting students. Although not the teacher/researcher’s field of expertise, intense consultations with the subject designers produced behaviourally framed objectives and a teaching/learning framework that targeted the desired skills. This case study consisted of a single implementation (N=223). Results were not conclusive, but examination of the detail has provided fresh insight into the potential value of peer evaluation and student portfolios to address the desired thinking.
Comparison of the two case studies has highlighted the marked similarity between the teacher’s expectations of statistical thinking, which underpins the University of Wollongong subject, and critical and evaluative thinking, which underpins the University of Western Sydney subject. ‘Structure’ has been identified as essential to successful implementation of the frameworks targeting discipline thinking. The structure of the desired thinking needs not only to be modelled but also to be recognised by students before it is effectively assimilated.
The researcher’s journey has required reflective practice that includes both telescopic and microscopic review of her thinking, her habits and the action and reaction occurring within her classroom. The evaluation of student learning undertaken in this thesis has formalised the teacher’s informal and intuitive response to the ostensibly absurd behaviours that take place as her students learn. Her deconstruction and interpretation of the apparent incongruities has at once affirmed past practice and inspired its renewal.
02Whole.pdf (2528 kB)