Investigating the mysteries of marking: the implications of staff marking practices for academic development

Bloxham, Susan, Boyd, Pete ORCID: https://orcid.org/0000-0002-2234-3595 , Ashworth, Mary and Orr, Susan (2009) Investigating the mysteries of marking: the implications of staff marking practices for academic development. In: 17th Improving Student Learning Symposium: Improving Student Learning - for the 21st Century Learner, 7-9 September 2009, Imperial College, London. (Unpublished)

Preview

PDF - Supplemental Material Available under License CC BY-NC Download (47kB) | Preview

Abstract

When grading student assignments university lecturers act as ‘gatekeepers’ for academic standards (Smith and Coombe, 2006). Despite its significance very little is known about the practice of marking and academic development literature has paid little attention to how we prepare and support staff for this activity. Previous research has examined the disjunction between lecturer assessment behaviours and their pedagogical beliefs (Orrell 2003), inconsistency in marking (Baume et al., 2004; Norton, 2004; Price, 2005; Read et al. 2005), the decision-making practices that obtain through moderation (Swann & Ecclestone, 1999; Orr 2007), staff attitudes to assessment criteria (Ecclestone, 2001), the interplay of subjectivity and objectivity in marking decisions (Shay 2005) and the tacit nature of assessment standards in higher education (O’Donovan, Price & Rust, 2008). However, with the exception of Orrell, studies have examined marking practices second hand, reporting on lecturer attitudes and examining the outcomes of their marking.

Consequently, the nature of marking judgments is under-researched and this study is designed to contribute to this area by investigating tutors ‘thinking-in-assessment’ first hand (Orrell 2007). It considers how, if at all, they draw on the artefacts of assessment in making their judgements. These artefacts include learning outcomes, assessment criteria, grade descriptors, feedback sheets and concepts related to quality in academic writing such as ‘critical analysis’. The study uses think aloud protocols, asking assessors to verbalise their thinking as they grade and write feedback on assignments. It asks the question, how do lecturers make judgements about student work and what is the role of artefacts within that process? The investigation uses a socio-cultural theoretical framework, recognising the assessment work of lecturers as a socially situated activity (Delandshere, 2001).

A sample of twelve lecturers from a range of subject disciplines in two UK Universities participated in the study. They were asked to think aloud as they graded two written assignments. The think aloud activity was followed by a short semi-structured interview that gathered some information on their experience of grading student work and on the process of marking the two specific assignments including the use of artefacts. Both think aloud activity and interview were recorded, transcribed and analysed using a qualitative thematic approach (Ritchie & Lewis, 2003). An initial coding framework was utilized which was amended during the process as new themes emerged. There was an ethical risk because the lecturers were marking real student assignments. Steps were taken to minimize the danger that thinking aloud might influence marking in such a way that students would be disadvantaged. Preliminary findings from the study indicate substantial differences in marking practices and use of artefacts, the widespread use of norm referencing despite the espoused use of criterion referencing, and considerable deliberations and iterative ‘self-negotiations’ regarding selecting the appropriate mark. The implications of the research for staff development will be presented and the paper will include discussion of the influence of the ‘think aloud’ protocol on marking decisions and the impact of anonymous marking on grading practices.

Item Type:	Conference or Workshop Item (Paper)
Departments:	Academic Departments > Institute of Education (IOE) > Non-Initial Teacher Education (Non-ITE)
Depositing User:	Insight Administrator
Date Deposited:	27 Oct 2010 15:15
Last Modified:	12 Jan 2024 08:46
URI:	https://insight.cumbria.ac.uk/id/eprint/341

Downloads