Systematic reviews as new assessment practice for teacher educators

Abstract

One of the consequences of the latest Swedish teacher education reform from 2011 is that a student teacher is required to write two student theses during her studies. These student theses should together represent 30 ECTS. One common way to implement this requirement is that the student writes two theses, where the first one is a systematic review of earlier research and the second one a more traditional empirical thesis. This article investigates if this new type of student thesis – that is, systematic reviews written by students as the first part of the requirements for teacher qualification, also involves a new kind of assessment practice for the examiners at the teacher education programmes. In the student thesis the student is supposed to demonstrate skills concerning several formal learning aims of the teacher education to obtain their qualifications. These include to follow the development of knowledge within a field and solve problems. The student thesis is not only important for the individual student to obtain qualifications, but has also been used to evaluate the teacher education programmes. Despite its significance for students and schools of education, very little research has been conducted on practices in the assessment of systematic reviews. Earlier research related to assessment of student theses demonstrates the difficulties with agreement in assessment practice. The results from earlier studies regarding the importance of the examiners’ background present a divided picture. While some research point to the examiners’ own subject or experience as important when judging theses, other studies do not.
The purpose with this study was to examine if a systematic review, as a new phenomenon at the teacher education programmes, has entailed a new assessment practice for the examiners at the universities. Supervisors and examiners at six universities with teacher education have been interviewed and have also answered a survey concerning these systematic reviews. The interviews were conducted through a combination of comparative judgement and repertory grid technique. The interviews were conducted after the respondents had read 5-8 student theses. In the interviews, criteria which were pointed out as decisive for the quality of a thesis by the respondents were discussed. The survey was a Q-sort survey where the respondents ranked different assessment criteria. A total of 45 criteria from the interviews were ranked from 0-10 by the respondents with respect to how important they are when judging a student thesis.
The results demonstrate a width in the assessment repertoire among the respondents. Examiners in the study do not see notable differences between a systematic review and a more traditional student thesis. The reasoning concerning criteria was mostly made on a relatively general level, which could be applied both to systematic reviews and empirical theses. Only 27 per cent of the respondents in the survey set a criterion which was more important for a systematic review. Those differences which were mentioned related to the structure of the thesis, the method and how the relation to research is described. Results suggest that examiners’ assessment criteria is a potentially important area for educational development within schools of education, and would also deserve attention in wider national and international discussions.