Research and analysis

Marker effects and examination reliability

This study looked at how three different analysis methods could help us to understand 'rater' effects on exam reliability.

• From: Ofqual
• Published: 21 January 2013

Applies to England

Documents

Marker effects and examination reliability: A comparative exploration from the perspectives of generalizability theory, Rasch modelling and multilevel modelling

Ref: Ofqual/13/5261

PDF, 1.89 MB, 87 pages

This file may not be suitable for users of assistive technology.

Request an accessible format.

If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email [email protected]. Please tell us what format you need. It will help us if you say what assistive technology you use.

Details

This study looked at how three different analysis methods could help us to understand rater effects on exam reliability. The techniques we looked at were:

• generalizability theory (G-theory)
• item response theory (IRT): in particular the Many-Facets Partial Credit Rasch Model (MFRM)
• multilevel modelling (MLM)

We used data from AS component papers in geography and psychology for 2009, 2010 and 2011 from Edexcel.

Published 21 January 2013