Research and analysis

Against the odds: achieving greater progress for secondary students facing socio-economic disadvantage

Published 30 June 2021

About the Commission

The Social Mobility Commission is an independent advisory non-departmental public body established under the Life Chances Act 2010 as modified by the Welfare Reform and Work Act 2016. It has a duty to assess progress in improving social mobility in the UK and to promote social mobility in England.

The Commission board comprises:

• Steven Cooper, Chair of Experian UK and CEO of Aldermore Bank Plc – interim Co-Chair
• Sandra Wallace, Partner and Joint Managing Director at law firm DLA Piper – interim Co-Chair
• Saeed Atcha MBE, Chief Executive of the charity Youth Leads UK
• Alastair Da Costa, Chair of Capital City College Group
• Pippa Dunn, Co-Founder of Broody, helping entrepreneurs and start ups
• Sam Friedman, Associate Professor in Sociology at the London School of Economics
• Harvey Matthewson, Aviation Activity Officer at Aerobility, a charity for disabled people
• Jessica Oghenegweke, Director of Association of Apprentices and student at Brunel University
• Farrah Storr, Editor-in-Chief of ELLE UK
• Jody Walker, Director of retail company TJX Europe
• Liz Williams MBE, CEO of FutureDotNow, a coalition focused on accelerating digital skills
• Sammy Wright, Vice-Principal of Southmoor Academy in Sunderland

About the authors

This report is by Dr Sally Riordan, Professor Michael Jopling and Dr Sean Starr from the Education Observatory at the University of Wolverhampton.

The Education Observatory undertakes research in all areas of education and allied areas of social policy to secure social justice and regional transformation in the West Midlands. It aims to undertake high quality research and consultancy, which informs debate and has an impact on practice and policy. The Observatory encourage collaborations among researchers, practitioners, policy-makers and other local, regional and national stakeholders.

Acknowledgements

The authors would like to thank all the teachers, support staff, senior leaders and students who gave their time to take part in this research by sharing their views in surveys, interviews and focus groups. We would also like to thank Ali Jaffer, Sammy Wright, Erin Hawkins, Sasha Morgan and members of our technical advisory group for their insightful comments and feedback. This work could not have been completed without a team of researchers who conducted interviews across 32 schools in England. We especially thank Andy Aston for leading this work, as well as Brendan Bartram, Andy Cramp, Debbie Hayes and Lydia Lewis for their very useful contributions and Matt Johnson and Amy Welham for their invaluable project management skills. Finally, we thank Catherine Bassindale for her skilful editing and finishing of this report.

For further information

Any enquiries regarding this publication should be sent to:

Social Mobility Commission
Sanctuary Buildings
20 Great Smith Street
London
SW1P 3BT

Email: [email protected]

Licensing

This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.

Foreword

It has never been more important for all pupils, whatever their background, to be able to make the progress they deserve at school. And yet, in my job as Vice-Principal of a large secondary school, I constantly see how otherwise able pupils are held back by their circumstances of birth.

It shouldn’t be this way. While the pupil premium has become a vital part of school funding as part of schools’ strategies to support social mobility, it is also true that most of the successes seen so far are in the primary sector. In this report we provide one of the most thorough investigations to date on the characteristics of secondary schools able to start closing the progress gap and achieve better outcomes for disadvantaged pupils.

The blunt truth is that despite schools’ best efforts, this is staggeringly rare. Our research looked closely at the approaches which made a difference, through a survey of 285 schools across England, in-depth fieldwork, and analysis of national Progress 8 data.

We found that the pupil premium picture across schools is extremely volatile. Nationally the gap has widened every year since 2016, but there is little evidence of a universally applicable approach to tackling this. Instead, we found that schools’ individual context was crucial, as was the role of staff culture in the effective use of the pupil premium. Schools varied significantly in their context and resources, so we produced a taxonomy of schools to better understand what works for each broad type.

To this end, we have produced the pupil premium primer. This is a resource for schools that we hope will allow them to compare themselves to similar schools, learn about other schools’ pupil premium journey and approaches, and take a longer-term approach to tackling disadvantage. We recognise that the causes of educational inequality stretch far beyond the classroom, and at the core teachers need to do the best for each child in their care. Yet in light of the pandemic there has never been a more pressing time to revisit what disadvantage means and how best to address it.

The scale of the challenge is huge. Pupil premium is one of the only tools we have right now to get resources to those schools that need it most. This research shows clearly that the model of schools utilising pupil premium funding in whatever way they see fit, is by far the best strategy to deal with the individual needs of children in vastly differing circumstances. But if we are to have any hope of real change we need to refine the measure, recognising different disadvantages, and extend it into the 16 to 19 sector. Anything less is acquiescing to the continuing widening of the gap – and to the deep unfairness at the core of our system.

Sammy Wright,

Commissioner and Vice-Principal of Southmoor Academy in Sunderland

Executive summary

In 2011, the UK government introduced new funds to tackle socio-economic disadvantage in schools in England: the pupil premium. Ten years later, students from the lowest income homes, and children in care, still do not progress as well as their peers in most secondary schools. This ‘Against the odds’ study investigates the characteristics and strategies of schools that have bucked this trend. Having conducted fieldwork in 32 English secondary schools, it is the largest study of its kind.

As the coronavirus pandemic threatens to widen the progress gap between pupil premium students and non-pupil premium students it has never been more urgent to ask what teachers, school leaders, researchers and policy makers can do to tackle the negative educational consequences of poverty. ‘Against the odds’ has shown that, for most schools, the pupil premium has become a vital component of their budgets.

The pupil premium initiative has so far successfully focused financial resources on the education of some of the most vulnerable children in our schools. It has also changed school culture and policy making by making staff more aware of the challenges faced by some of their students. It is now time to build on the experiences of the last ten years to improve how schools implement the pupil premium.

There are many positive stories to be heard. We have found that schools with the most successful outcomes for pupil premium students are likely to share certain characteristics regarding their context and staff culture. However, many of these characteristics are accidents of history or location and not easy to replicate. Approaches believed to be working in some schools are perceived to fail elsewhere, but the impact of pupil premium at the system level remains difficult to identify.

Schools face immense challenges and the reasons why the progress gap continues to widen, despite their best efforts, are complex. There has been insufficient longitudinal evaluation of the pupil premium since it was introduced and this is necessary now. The findings from ‘Against the odds’ suggest that, having made promising and positive steps to support students from lower-income homes, a more variegated, coordinated and collaborative approach is now required nationally.

We have created a pupil premium primer to help schools take a longer-term approach to tackling socio-economic disadvantage. This is a complementary resource to the Education Endowment Foundation’s toolkit, providing detailed case studies of what it is like to implement a strategy on the ground. The primer’s resources enable schools to compare their contexts to other schools, learn about the journeys of similar schools, explore the attitudes of their staff, and consider the impact of policies on students’ wellbeing. They do not provide a simple solution, but enable school leaders to consider their pupil premium strategy more broadly, knowing that deeper, more prolonged effort is necessary to bring about lasting change.

Key findings

The pupil premium landscape is volatile

• the disadvantage gap initially decreased after the introduction of the pupil premium, but then stagnated and began increasing (even prior to the pandemic). Since the introduction of Progress 8 in 2016, the progress gap between students allocated pupil premium funds and their peers has increased each year (measured at a school level). This is despite the fact that most schools implement a wide range of interventions to tackle socio-economic disadvantage and have done so for many years. Schools where pupil premium students make exceptional progress struggle to sustain this over time
• the economic impact of coronavirus is likely to increase poverty and the disadvantage gap. This looks set to drive up the number of pupil premium students and sharpen the focus on its effective use
• schools are increasingly likely to spend the pupil premium on initiatives aimed at boosting students’ cultural capital. Teachers believe that quality teaching and personalisation are the most effective ways to support students facing socio-economic disadvantage, but these beliefs are not always translated into practice
• most school leaders are aware of the Education Endowment Foundation’s (EEF) teaching and learning toolkit and use it occasionally. However, they would welcome further detailed guidance, including examples of how to implement interventions in practice and of the contexts in which they have been effective, as well as more external support in tackling socio-economic disadvantage

Context matters

• where pupil premium students show exceptional progress, we estimate that around three-quarters of the schools have contextual advantages. These ‘hidden resources’ are not easy to identify from standard school data. They include historically strong reputations, high levels of parental engagement, and active alumni networks. Our estimates suggest that 15% to 20% of mainstream state secondary schools in England fall into this category
• schools with hidden resources are more likely to take a straightforward approach to socio-economic disadvantage: one person knows all pupil premium students, resources are tailored to students’ needs, and a simple and accessible pastoral system is in place
• some schools face extreme contextual challenges, which are associated with lower levels of progress among pupil premium students. Examples of these challenges include high levels of student mobility (students transferring from one secondary school to another) and large numbers who have experienced trauma or are in long-term care
• high levels of student mobility are concentrated in a minority of schools where it is a significant barrier to addressing socio-economic disadvantage
• a school’s absence rate is the strongest predictive factor of the progress made by its pupil premium students, but in most schools, it is only a minor focus of pupil premium policy
• teachers find it difficult to judge their school’s context in relation to others and some school leaders feel isolated in the challenges they face. This makes it difficult for them to implement and evaluate approaches to support students facing socio-economic disadvantage. The research did not indicate that contextualised measures would help this and it is not generally called for by schools

Staff culture matters

• some schools have found inventive ways to improve everyday school life and wellbeing for students facing socio-economic disadvantage by addressing their particular needs and skilfully deploying key staff, including support staff
• vertical tutoring and mixed attainment classes at Key Stage 3 (for pupils aged 11 to 14) are associated with higher rates of progress for pupil premium students
• a school’s staff culture regarding the pupil premium is an important factor in tackling socio-economic disadvantage. In some schools, as many as a quarter of staff have reservations about the pupil premium, believing it to be discriminatory or ineffective
• schools commonly implement pupil premium policies without considering the potential negative impact on students. However, students often have important insights into how effective or appropriate approaches are likely to be. Mechanisms for listening to students in schools are generally not well-developed
• there is no single approach to reduce the progress gap that can be implemented in all schools, or even the majority of them
• schools need support in how to use the research evidence and implement it in practice

Recommendations for national policy makers

• the Department for Education should recognise that there is no single, well-evidenced approach to socio-economic disadvantage that can be implemented to reduce the progress gap
• the Department for Education and Ofsted should review how progress data is presented and used; put measures in place to try to prevent data from being interpreted too crudely without acknowledging schools’ contexts and hidden resources; and recognise that some schools need to refocus pupil premium on a small number of critical issues
• the pupil premium criteria should be reviewed to consider whether it can support schools facing high levels of student mobility or absence. For example, schools could receive an additional pupil premium, paid termly, for students transferring from other secondary schools (provided that permanent exclusions do not rise significantly)
• most importantly, the evidence suggests that the pupil premium criteria should be adapted to recognise and provide additional funds for students facing persistent disadvantage
• pupil premium lead teachers should be recognised by a national award, and supported by local networks, to facilitate collaboration, professional learning and knowledge sharing regarding the educational impacts of socio-economic disadvantage
• extra support should be made available to schools to help them implement research evidence in practice
• large-scale, longitudinal research, with schools as active research partners, should be conducted to investigate the importance of a school’s organisation and culture. This could assess the impact of vertical tutoring and setting arrangements on the wellbeing and achievements of students facing socio-economic disadvantage

Recommendations for school leaders

• school leaders should not just consider what pupil premium approaches to implement, but how to implement them effectively in their school context
• school leaders should regularly explore staff attitudes towards the pupil premium, how well their school responds to the needs and circumstances of its students, and the impact of their policies on students’ wellbeing
• when schools have had little success at reducing the progress gap, they should refocus on a small number of critical, context-specific issues, such as improving the experience of transferring students or reducing student absence

1. Introduction

Secondary schools in England have long faced the challenge of reducing the gap between the educational achievements of young people from lower-income homes and their peers. In 2020 and 2021, the stakes have been raised. By keeping young people out of schools and increasing economic uncertainty, the coronavirus pandemic has threatened to further widen the education gap. This makes it even more pressing to address how best to support students facing socio-economic disadvantage.

The purpose of this ‘Against the odds’ study is to investigate the characteristics and strategies of schools where students from the most disadvantaged backgrounds achieve similar or better rates of progress than their peers. The study has used the pupil premium designation as a measure of socio-economic disadvantage and the Progress 8 score as a measure of their academic progress through secondary school.

The Progress 8 measure was chosen because it has not previously received as much attention by researchers as the attainment gap. It accounts for the attainment of students at the end of primary school, and therefore highlights the impact of approaches to the pupil premium at secondary school.

We distinguish in this report between a strategy, an approach and an intervention. A strategy is a plan of action with a long-term aim. An approach is a way of addressing socio-economic disadvantage, such as focusing on cultural capital or the quality of teaching across the school. An intervention is an activity intended to achieve a short-term outcome, such as providing a group of students with after-school tuition.

Background to the pupil premium

The pupil premium is an annual sum, awarded to a school for each student who satisfies one of the following criteria:

• is (or ever has been) in care
• is receiving free school meals (or has done so in the last six years)
• is from a military family

In the 2019 to 2020 academic year, students with pupil premium allocations made up 28% of the student population of mainstream secondary schools. On average, each school received £220,000 to support 230 students. 136 schools received sums of more than £500,000.

We have used the pupil premium award to identify students who face socio-economic disadvantage because of the absence of better socio-economic data at a school level. Studies have suggested that it is a crude marker of disadvantage because it excludes some of the students facing the most severe disadvantage, it records others who are not, and it does not distinguish the most severely disadvantaged.^{[footnote 1]}

‘Pupil premium student’

We use ‘pupil premium student’ instead of ‘disadvantaged student’ at the request of pupil premium students themselves: they describe the latter term as “demotivating” and “not right” but suggest that ‘pupil premium student’ is “a good phrase”

Background to Progress 8

We used a school’s Progress 8 scores to measure the progress made by students between the ages of 11 and 16. The score is calculated for each student by comparing their GCSE results with the results of peers who achieved a similar level of attainment at the end of primary school. The Department for Education publishes each school’s average Progress 8 score, both for pupil premium and non-pupil premium students. ^{[footnote 2]}

‘Progress gap’

In this report, ‘progress gap’ refers to the difference between a school’s Progress 8 score for pupil premium students, and the school’s Progress 8 score for non-pupil premium students. A positive progress gap means that pupil premium students have made more progress than their non-pupil premium peers. Unless otherwise specified, this report is concerned with negative progress gaps (where the pupil premium students make less progress)

Study summary

In this ‘Against the odds’ study, we conducted a national survey of secondary schools, analysed national progress data, and conducted fieldwork in 32 schools. We used a mixed methods approach to compare the characteristics and strategies of schools including those where pupil premium students make exceptional progress.

This resulting report highlights pupil premium strategies and interventions in schools across England. It offers insights into the many different journeys that schools are on as they attempt to reduce the negative educational impacts of socio-economic disadvantage. We have analysed national data and presented the voices of students, classroom teachers, support staff and senior leaders.

The picture emerging is a complicated and contradictory one. As many schools know only too well, there is no simple fix to eliminate the progress gap. The gap is not a single issue and schools can face very different challenges in reducing it. This report attempts to identify and discuss some of the subtleties of educational disadvantage, presenting the evidence of how school context, staff culture and school organisation are relevant to a school’s approach to reducing the gap.

The findings from this report have been used to create a pupil premium primer. These resources are designed to support longer-term thinking about the pupil premium.

2. Methodology

‘Against the odds’ uses a mixed methods approach to investigate the progress of pupil premium students. It comprises the following strands:

1. Literature review.

2. Online survey of school leaders.

3. Analysis of national Progress 8 data.

4. Fieldwork in schools.

Literature review

The study begins with a literature review of academic papers, government briefings and reports by education researchers. Of the literature published in the last ten years, 94 studies were judged to be relevant to tackling socio-economic disadvantage in English secondary schools. Nine studies published before 2009 were also included. The findings have been summarised for teachers and school leaders and are available in our pupil premium primer.

Online survey of school leaders

All mainstream secondary schools were invited, via email or telephone, to take part in an online survey regarding their school’s approaches to disadvantage, attendance, transition from primary school and school organisation. 10% of schools responded, giving our survey a total of 360 teachers from 285 schools. Most respondents were school leaders, but 80 classroom and support staff also completed the survey.

Survey coverage

Analysis showed that the schools which responded (see Figure 1) were a representative sample of mainstream secondary schools in England with respect to region, academic outcomes, proportion of students eligible for free school meals, size, and Progress 8 scores.

Figure 1. Locations of surveyed schools

Location	Number of schools surveyed
East Midlands	22
East of England	49
London	37
North East	8
North West	38
South East	41
South West	41
West Midlands	39
Yorkshire and the Humber	10

Survey analysis

Our survey included both open and closed questions. Closed question responses were analysed for correlations with the school’s progress gap.

Thematic analysis was applied to the open questions and schools were categorised according to the approaches they take to student attendance and the pupil premium. These categories were verified in 10% of cases (where we conducted fieldwork). Statistical analysis was then applied to identify correlations between particular approaches and the progress gap.

The results of the survey analysis were used to identify themes to focus the fieldwork on, for example vertical tutoring, pupil premium spending, mixed attainment classes, student mobility and student attendance.

Analysis of national Progress 8 data

We collated data from a range of sources:

• school characteristics published by the Department for Education in 2018
• school league table data published by the Department for Education from 2016 to 2019
• index of multiple deprivation data published by the Office of National Statistics
• school inspection gradings and dates, published by Ofsted
• vertical tutoring data from our online survey and a further internet search

We used multilinear regression analysis to model the Progress 8 scores of pupil premium students and the progress gap (defined above). ^{[footnote 3]}

The national data was also used to compare trends in Progress 8 scores over time for schools with different characteristics. For more information please see the technical appendices in this report.

Fieldwork in schools

Selection of schools

From the schools that responded to the survey, we attempted to identify those where pupil premium students had made exceptional progress. Because our data analysis demonstrated that a positive progress gap by itself is not a reliable indicator of the following year’s Progress 8 score, we sought schools with further indications of exceptional progress.

We categorised a school as having achieved exceptional sustained progress for pupil premium students if one of the following applied:

• the school had positive progress gaps for at least two years in a row
• the school had one positive progress gap (in the previous three years) and the Progress 8 score for pupil premium students was positive (above the national average for all students) for at least two years in a row.

We categorised a school as having achieved exceptional improved progress for pupil premium students if one of the following applied:

• the school had a positive progress gap (in the previous two years) and had improved the Progress 8 score for pupil premium students (totalling at least 0.3) for two consecutive years
• the school’s previous Progress 8 score for pupil premium students was positive (above the national average for all students) and the school had improved the Progress 8 score for pupil premium students for two consecutive years

We identified 32 schools that satisfied these criteria. Only three schools were unwilling to participate in the study. Nine grammar schools and two studio schools were excluded to ensure a breadth of school types.

Fifteen schools (eight sustainers and seven improvers) were selected to cover a range of standard school characteristics, as summarised in Table 1.

A further 15 schools were invited to take part in the study. They did not satisfy the criteria for exceptional progress but otherwise shared similar contexts, according to standard characteristic data. Their progress for pupil premium students ranged from very poor to good.

In addition to these 30 mainstream secondary schools, we interviewed the leaders of two special schools about their use of the pupil premium and the challenges they face in reducing the progress gap.

Table 1. Characteristics of the fieldwork schools

	Number of fieldwork schools with exceptional progress	Number of fieldwork schools without exceptional progress	Total number of fieldwork schools (out of 30)
Rural	2	2	4
Coastal	1	2	3
Very small: less than 800 students	3	3	6
Very large: more than 1,400 students	1	2	3
No sixth form	5	6	11
Part of a multi-academy trust	4	5	9
Academy sponsored	2	1	3
Academy converter	5	8	13
Studio school	1	1	2
Grammar school	2	1	3
Single-sex admission	3	1	4
Very high proportion of students who speak English as a second language: more than 30%	6	2	8
Very high proportion eligible for free school meals: more than 25%	3	1	4
Vertical tutoring	2	4	6
Mixed attainment classes (no setting or streaming) in Year 7	3	3	6

Research approach

In each fieldwork school, we initially conducted interviews or focus groups with members of the senior leadership team. In a further 18 schools we returned to conduct a range of interviews, focus groups and surveys with teachers, support staff and students, depending on the nature of the school’s pupil premium approach.

Figure 2. Interview and focus group participants in fieldwork

Focus Group	Number of participants
Senior Leaders	70
Classroom Teachers	84
Support Staff	13
Students	106

Fieldwork analysis

Interview transcripts were coded thematically. This enabled us to identify fine differences in the schools’ approaches and attitudes to socio-economic disadvantage and then to group schools according to these fine differences. For example, we distinguished schools that funded small group interventions according to whether the interventions were for pupil premium students only and whether students were required to attend or not.

Because our analysis was ultimately conducted at a school level, but the interviews were not, we sometimes had to make a judgement regarding how reflective a particular attitude or approach was of the school as a whole. We developed criteria in order to guide some of the more difficult decisions. For example, a school was identified as having a strong focus on cultural capital if it was highlighted by at least one member of staff and had been implemented throughout the school.

We were then able to compare the approaches and attitudes of schools with exceptional sustained progress, schools with exceptional improved progress, and schools without exceptional progress. Because of the large size of the sample, we were able to back this qualitative work up with quantitative comparisons.

Schools were grouped together where staff had demonstrated similar attitudes and strategies for the pupil premium. This was developed into a typology of schools illustrated in Figure 3. We were then able to demonstrate strong associations between a school’s choice of pupil premium approach and its context. This enriched the contextual comparisons we could make with national data, because more detailed contextual data about our fieldwork schools was available.

Figure 3. School typology^{[footnote 4]}

The fieldwork schools were plotted according to their hidden resources and extreme contextual challenges. Schools in each of the four categories took similar approaches to the pupil premium.

The impact of COVID-19

This research was interrupted by the partial closure of schools in England in March 2020. Most interviews and focus groups were conducted prior to this and the findings do not, on the whole, reflect the impact of the pandemic on schools.

However, in the period March to September 2020, we surveyed more than 200 staff in six of our fieldwork schools. It was clear from this work that the coronavirus pandemic had increased schools’ focus on students with pupil premium funding. We also found striking changes to teachers’ attitudes towards pupil premium spending. Prior to the pandemic, only 5% of school leaders (and 6% of staff) reported that pupil premium was best spent on personal resources for students. Staff were most likely to mention spending on textbooks and stationery when discussing these needs. However, during the pandemic lockdown, 70% of staff told us that providing resources was a “very effective” way to support pupil premium students and the most commonly reported need was now IT equipment, most particularly laptops with internet access. Staff did not consider this to be a temporary need during lockdown, but a long-term change in their priorities. In the words of the teachers themselves:

We could do more to be aware of IT capacity in pupil premium student households.” (Middle leader)

I would like them all to be given greater access to IT.” (Classroom teacher)

Limitations of the methodology

Ultimately, this study tackles a difficult and complex issue relating to education quality.

Many factors limited the study, the most important of which are listed here:

1. ‘Against the odds’ provides a snapshot of the status of the pupil premium in secondary schools. It is not a longitudinal study. This means that it is impossible to verify that any particular approach was responsible for exceptional progress. We can only observe that some approaches are associated with higher levels of progress. This challenge has been further complicated by the fact that almost all schools are taking a multi-layered approach to tackle socio-economic disadvantage, and because it is not truly possible to differentiate a ‘pupil premium approach’ from many other aspects and policies of a school.

2. Correlations discovered in statistical analysis do not show causal relationships and should not be interpreted as causal without further evidence. In many cases, the variables considered are only markers or signs of deeper underlying causal mechanisms.

3. Because the variables used in the statistical analysis are highly correlated with each other, it has not been possible to tease out how much each contributes to the variance of schools’ Progress 8 scores. Some factors have been found to be unstable under multilinear modelling, most notably the percentage of students eligible for free school meals.

4. The Education Endowment Foundation (EEF) currently recommends prioritising the quality of teaching as the first focus of a school’s pupil premium policy. We were unable to assess the quality of teaching in schools and thus to control for this variable. It is therefore possible that we have incorrectly attributed exceptional progress to factors other than the quality of teaching. We did attempt to use Ofsted’s rating of teaching as a proxy indicator to overcome this issue, but this was too unreliable, not least because many schools in the study had not been inspected recently.

5. When conducting analysis of the fieldwork data, researchers had to make decisions about how to categorise schools. Analysis of this kind always has a subjective component, which is shaped by the researcher’s prior experiences and impressions of the school.

6. The release of 2019 progress data by the Department for Education revealed that the results of four schools had declined (two schools that had previously been identified as making exceptional sustained progress, and two schools with exceptional improved progress). This meant they no longer strictly satisfied our criteria for exceptional progress. At that late stage of the project, we were unable to account for this and the schools were removed from the analysis. However, this did not solve the deeper issue of the volatility of progress data and the need for longitudinal studies.

3. The volatile landscape of pupil premium strategy

‘Against the odds’ has revealed that in recent years there have been major shifts in schools’ strategies and attitudes towards the pupil premium. Many schools have moved further in their pupil premium journey, with both positive and negative consequences. Teachers report that secondary schools are more focused on socio-economic disadvantage than they used to be.

There has also been a shift in attitudes around how the pupil premium should be spent, but not all these attitudes are reflected in practice. As a result, there is a growing despondency in some schools about the effectiveness of pupil premium approaches and many school leaders would welcome more detailed guidance and outside support on tackling socio-economic disadvantage.

An increased focus on socio-economic disadvantage in secondary schools

This fieldwork research demonstrates that most schools have made serious, sustained efforts to improve the academic outcomes of pupil premium students, from the perspective of both school leaders and classroom teachers. This contrasts with previous studies, in which researchers found that a significant number of schools had not directly taken up the challenge of tackling socio-economic disadvantage or had done so only recently. ^{[footnote 5]}

Greater focus on pupil premium from school leadership

All schools participating in our fieldwork had appointed a member of staff to take whole-school responsibility for the pupil premium. In most schools this position had been in place for more than three years and was held by a member of the senior leadership team. Only a minority of schools did not follow this pattern: one had created a pupil premium post only 18 months prior to the study; another admitted that pupil premium strategy had until recently merely been a tick-box exercise. Almost all senior leaders and teachers taking part in our survey reported a strong focus on pupil premium students in their school.

Almost all classroom teachers surveyed reported that they are required to provide pupil premium students with additional attention or support ^{[footnote 6]}

Greater focus on pupil premium in the classroom

School leadership teams’ focus on pupil premium has filtered down to the classroom. Almost all teachers have been asked to provide pupil premium students with additional attention during daily teaching activities. One teacher called this the ‘classic classroom approach’ to pupil premium (see Box 1). It was also described as the ‘simple’, or ‘usual’ approach. As one headteacher described it:

So staff do simple things… every staff member should have pupil premium students marked off on their teaching plan and seating plan, so they know where they are. They should mark their books first, so this pupil premium kid is getting the best of the marking.

We found that schools have had mixed results from applying this approach (described in the next section).

Of the interviewees who expressed an opinion regarding teachers’ shifting attitudes to students facing socio-economic disadvantages, the majority said there had been an increasing awareness of this problem and regarded it positively. Some longstanding teachers described the change in personal terms:

If I’m very honest, that’s when I really started focusing on pupil premium.

A small minority of teachers believed that secondary schools have always focused on students from lower income homes and one reported that there had been a greater focus on pupil premium “a few years ago”, which had since waned.

Box 1: The ‘classic classroom approach’ to the pupil premium

This approach requires teachers to give extra support or attention to pupil premium students during everyday teaching activities.

Common examples include:

• marking their books first
• giving additional feedback
• considering where to seat them in classrooms
• monitoring their progress more closely
• asking them more questions during lessons
• checking they have the right equipment
• handing out materials to them first
• conducting whole-school or departmental assessments of books

The challenge of sustaining a zero progress gap

Despite schools’ focus on pupil premium students, the average progress gap for mainstream schools has increased since the introduction of Progress 8 in 2016 (see Figure 4). The gap is proving to be stubborn and even schools that have seen improvements have found this difficult to sustain.

Only 6% of secondary schools achieve a zero (or positive) progress gap each year, and the majority of these are grammar schools, or schools with small numbers of pupil premium students, whose progress gap fluctuates from year to year. We found that only 11 schools had maintained a zero gap for three continuous years. Of these, six are grammar schools, three are former grammar or independent schools, and one was investigated in 2017 for high levels of off-rolling (removing pupils from the school roll without using a permanent exclusion, where the removal is in the best interest of the school, not the pupils).

Figure 4. Average school progress gap over time

Shifting attitudes to whole-school approaches to the pupil premium

Early research into the pupil premium indicated that in many schools it was pooled with general school funds.^{[footnote 7]} This is no longer the case: all schools in the study maintained separate accounts for the pupil premium. However, three pupil premium leads suggested that these funds are less separate than they appear. The pupil premium is being used to support systems that have previously been funded by general funds and were not previously considered to be pupil premium initiatives. Examples included elements of the school’s pastoral support system and the use of teaching assistants. We found further evidence of this tendency for the premium to be used to balance budgets in other schools, for example by funding a school counsellor who had worked in a school for 18 years.

This is no longer the case: no school leader expressed this concern to us and 20% of schools in our survey reported tackling socio-economic disadvantage by taking measures to improve teaching across the school. This may be because the Education Endowment Foundation (EEF)’s Teaching and Learning Toolkit advocates for whole-school initiatives as part of a school’s pupil premium policy. The majority of school leadership teams are aware of this toolkit (see next section for more details).

Over two-thirds of school leaders use the Education Endowment Foundation’s Teaching and Learning Toolkit at least “a little bit” when setting their pupil premium strategy

Teachers’ beliefs and commitments regarding socio-economic disadvantage are not always demonstrated in practice

High quality teaching

When asked about the most effective way to support pupil premium students, almost half of the senior leaders in our survey mentioned high quality teaching. This was the most common theme in the responses (see Figure 5). However, schools’ policies did not usually reflect this: only 37% of senior leaders who expressed this belief reported that they used pupil premium to improve the quality of teaching across the school. This survey finding was supported during fieldwork: in three cases, we confirmed that senior leaders believed that they could best support pupil premium students by providing high quality teaching, but no pupil premium interventions were in place to do this.

We were unable to ascertain why there is a discrepancy between what is believed to be effective and the intervention that is pursued. One teacher suggested that it was simply easier to follow the classic classroom approach:

I think the danger is that people just want a quick fix, and pupil premium is always a long game.

In comparison, only 5% of senior leaders suggested that providing resources was the most effect way to support pupil premium students, but 43% of school leaders reported doing this. (As described in the introduction, however, we have evidence that the pandemic has since increased the value teachers now place on using the pupil premium for resources.)

Figure 5. Approaches believed by senior leaders to be the most effective in supporting pupil premium students^{[footnote 9]}

Theme	Senior leaders who believed this approach to be effective
High quality teaching	48%
Personalisation and tailoring	30%
Home engagement	16%
Interventions and one-to-one support	16%
Staff student relationships	13%
Career guidance and raising aspirations	10%

Tailoring support for pupil premium students

Almost one-third of senior leaders emphasised the importance of personalisation and tailoring when asked what is the most effective way to support pupil premium students. Classroom teachers echoed this sentiment during our fieldwork:

Every child is different, so strategies have varied impact.

Pupil premium is often wrongly used, and the money wasted on trips instead of what that child needs. They are often grouped together rather than seeing the individual.

Again however, the school’s policy frequently did not reflect these beliefs: less than half of these senior leaders reported using pupil premium practices that were tailored to students’ needs. Our fieldwork also supported this finding. For example, at one school where teachers agreed that tailored support was important, all pupil premium students in Years 7 and 8 were placed in compulsory after-school interventions for core subjects (with no other students).

One plausible driver of this situation is the difficulty of personalisation in the face of systems that demand labelling and measurement, as discussed later in this section. As one teacher said:

It’s difficult to group the cohort as this homogenous group … I hate those pupil premium (PP) strategy documents. You try and identify three internal barriers and three external barriers when you’ve got 60% of the pupil cohort [as PP] and trying to say that they all fit into those.

Schools want more guidance and support to tackle socio-economic disadvantage

More than half of the schools that participated in our fieldwork expressed a desire for more help in reducing the progress gap (without being asked directly). The most common areas in which schools would welcome support are:

• improving student attendance
• getting more guidance from schools like theirs
• having more inspiration about what to do next

Most interviewees gave no suggestion of where this support should come from. In three cases, however, schools asked us to partner them with a school in a similar context. In one case, a school called for more direct support and communication from the Department for Education that was not via Ofsted.

Schools with exceptional progress for pupil premium students are more likely to look to a variety of sources for inspiration, including local schools, Twitter and blogs. One headteacher at a school with exceptional progress described this accumulative approach:

I do not know where we have taken all these ideas [from]. I do not think they’re our ideas, I think that we have been good at magpie-ing them from other people and putting them together.

As already indicated, the most commonly used resource to tackle socio-economic disadvantage is the EEF’s toolkit. Some teachers have found it to be inspirational:

I think the EEF stuff is always really helpful because it’s always quite practical, yet realistic in terms of not [being] particularly directive. It presents lots of ideas and that’s really good for stimulating our debate in terms of what we’re going to do here.

Figure 6. How much do you use the EEF toolkit to guide pupil premium policy? ^{[footnote 10]}

How much do you use the EEF toolkit to guide pupil premium policy?	Number of schools
A little	64%
A lot	14%
Not at all	14%
Not heard of it	7%

As Figure 6 illustrates however, most schools only use the EEF toolkit ‘a little.’ Despite positive views on the toolkit, many schools expressed a desire for more specific and detailed guidance. One leader said she would use it more if she had enough time. Most commonly, school leaders felt that they required more detail than the toolkit provided. They requested more of the following information:

• details of what an initiative “looks like on the ground”
• examples of where it has worked
• examples of where it has failed
• ideas to develop cultural capital (discussed in the next section)

Changes to the identification of students facing socio-economic disadvantage

Early research into the pupil premium indicated that schools were using a range of criteria to define socio-economic disadvantage.^{[footnote 11]} This is no longer the case. The majority of schools use pupil premium status as their single marker of socio-economic disadvantage. Some schools officially use it as their primary marker, but are more flexible in practice, for example by using pupil premium money to buy shoes for a student who is not known to be eligible for the premium. Many teachers were unhappy about the restrictiveness of pupil premium status:

To go out of your way actively, to kind of go the extra mile for PP? To me it makes no sense. You go out of your way for this kid. (Head of department)

A small minority of schools do not use pupil premium status to allocate resources at all:

The whole thing is that when you label them … it was a self-fulfilling prophecy … we should in schools be able to teach all children as individuals.

Such schools are more likely to have a large proportion of pupil premium students making exceptional progress. In comparison, an assistant headteacher in a school with very few pupil premium students regarded this approach as “very brave”. She cited pressure from Ofsted as the reason the school had to concentrate on pupil premium students, despite believing that the label did not accurately capture those most in need. Our fieldwork found supporting evidence for this: at her school a student’s postcode was a better predictor of their Progress 8 score than their pupil premium status. The postcode data indicated that many students from lower-income areas who potentially needed more support were being excluded from intervention sessions as a result of the school’s reliance on pupil premium status. The external pressure to do this was echoed by teachers elsewhere:

It does not matter if they’re PP, but I think [there’s] the fear …of Ofsted coming in and they go, ‘OK, we want to look at your pupil premium’ and then you go, ‘OK, where’s our list?’ That’s the first thing.

Shifting priorities for funding

Most schools have a multi-level, broad approach to tackling socio-economic disadvantage. One potential drawback to this is that schools are not identifying and concentrating their efforts on a small number of areas where the evidence suggests that the largest gains are to be made: student attendance, student mobility and the quality of teaching (discussed in the next section). Deploying a myriad of interventions has now been a feature of schools’ pupil premium policies for many years, but this study identified some recent changes in the approaches selected.

Decreasing use of teaching assistants

Previous research has indicated that deploying additional staff has been one of the most common uses of the pupil premium.^{[footnote 12]} This has decreased since 2014: schools reported to us that they have reduced the numbers of teaching assistants in their school, or that the pupil premium was being used to fund teaching assistants that were previously paid for from the staff budget. In our survey less than a quarter stated that teaching assistants were a focus of pupil premium strategy, which was also reflected in our fieldwork schools.

Almost a quarter of teachers reported that teaching assistants are a focus of their pupil premium strategy ^{[footnote 13]}

Cultural capital

The issue of cultural capital was raised by 37 teachers (in interviews and focus groups). These teachers demonstrated their belief that (lack of) cultural capital is a key causal factor in the progress gap. However, there was little consensus about its meaning or how to foster it, which made it impossible for the research to identify such a link.

‘Cultural capital’

A student’s cultural capital is the total non-financial assets that student possesses as a result of their cultural knowledge and experiences

Most commonly, teachers perceived that students’ cultural capital was increased through a variety of compensatory experiences, which usually necessitated travel. Across all school contexts, staff members described the geographical limitations faced by some of their students:

The students do not step foot out of their own towns … so [they] do not see the bigger picture.

Teachers also associated cultural capital with knowledge about careers, conversations at home, and parental expectations. Around 10% of teachers associated cultural capital with improved vocabulary. The various emphases resulted in many different approaches to increasing the cultural capital of pupil premium students. One school had altered the mathematics curriculum to incorporate exercises regarding the use of mathematics in the workplace. Another had introduced a literacy intervention to increase students’ vocabulary. A third invited family members to its breakfast club to encourage family discussions. Some schools paid for trips, others had specifically designed trips with the curriculum in mind, such as visiting a site relevant to a GCSE English text.

One senior leader explained that the increased focus on cultural capital in her school was due to Ofsted. More commonly, staff described the emphasis as coming directly from the headteacher or principal. The cultural capital challenge was taken up optimistically by some schools, but in schools serving both very affluent and deprived communities, it was commonly a source of despair. In these ‘two-in-one’ schools, many teachers felt that they did not know how to provide some students with what others experienced as a matter of course at home. Our pupil premium primer describes the stories of these schools in more detail.

The pupil premium journey

Researchers have previously suggested that schools are on a journey, moving through a series of phases as they address the challenges of socio-economic disadvantage.^{[footnote 14]} Our study supports this view and extends it in the following ways:

1. A significant minority of schools feel that they have come to the end of the road. They do not know where to go next. As one assistant headteacher said: “You won’t be able to find something in the book that we haven’t tried.”

2. Some schools were trying things for a second or third time, which makes pupil premium support feel cyclic: “We’re back to trips … like we were doing a few years ago.” (Pupil premium lead)

3. Some schools have implemented relatively simple strategies and have helped pupil premium students make good progress, so are not moving through stages. One headteacher explained, for example, that unlike other schools, they do not need to try out the ‘classic classroom’ approach. In this school, pupil premium students are making good progress without their teachers having to identify them explicitly.

4. Schools facing extreme contextual challenges have had to approach pupil premium differently from other schools and many feel isolated as a result:

The people who come in and look at schools, your Ofsted or your regional school commissioners … I do not think they understand it. They take it from a tabletop exercise of what disadvantage is.

In conclusion, despite being aware of the EEF toolkit, school leaders believe they need more support to reduce the progress gap. We found evidence that schools implement approaches that are easy to introduce (such as paying for resources), instead of those they believe in (such as tailoring support to students’ needs). Some school leaders are unsure of where to go next or feel alone in their struggle to improve the academic outcomes of their pupil premium students. We also found that in almost every school there are contradictory and negative attitudes to the pupil premium held by school staff and that policies are implemented without considering their impact on students’ identity and wellbeing. Some pupil premium lead teachers were struggling to work against these attitudes. Others reported that they had struggled to know how to perform their role effectively until they had used local networks to connect with colleagues in similar positions in other schools.

Together, these findings suggest that the policy of assuming that schools know best, regarding how to tackle the negative educational consequences of poverty, has not been entirely effective. The progress gap has been increasing each year since the introduction of the Progress 8 measure in 2016 and schools are finding it difficult to reverse this trend. We therefore recommend that a national award is introduced for pupil premium lead teachers, similar to the national award for Special Educational Needs (SEN) Coordination. A series of policy workshops with school leaders demonstrated that a more co-ordinated and collaborative approach is needed nationally and that an award would be one way to achieve this. Its purpose would be to improve knowledge and practice regarding the pupil premium, including trialling pupil premium initiatives and supporting professional learning for and collaboration among pupil premium leads.

Recommendations for national policy makers

• the Department for Education should recognise that there is no single, well-evidenced approach to socio-economic disadvantage that can be implemented to reduce the progress gap
• the Department for Education and Ofsted should review how progress data is presented and used; put measures in place to try to prevent data from being interpreted too crudely without acknowledging schools’ contexts and hidden resources; and recognise that some schools need to refocus pupil premium on a small number of critical issues
• the pupil premium criteria should be reviewed to consider whether it can support schools facing high levels of student mobility or absence. For example, schools could receive an additional pupil premium, paid termly, for students transferring from other secondary schools (provided that permanent exclusions do not rise significantly)
• most importantly, the evidence suggests that the pupil premium criteria should be adapted to recognise and provide additional funds for students facing persistent disadvantage
• pupil premium lead teachers should be recognised by a national award, and supported by local networks, to facilitate collaboration, professional learning and knowledge sharing regarding the educational impacts of socio-economic disadvantage
• extra support should be made available to schools to help them implement research evidence in practice
• large-scale, longitudinal research, with schools as active research partners, should be conducted to investigate the importance of a school’s organisation and culture. This could assess the impact of vertical tutoring and setting arrangements on the wellbeing and achievements of students facing socio-economic disadvantage

Recommendations for school leaders

• school leaders should not just consider what pupil premium approaches to implement, but how to implement them effectively in their school context
• school leaders should regularly explore staff attitudes towards the pupil premium, how well their school responds to the needs and circumstances of its students, and the impact of their policies on students’ wellbeing
• when schools have had little success at reducing the progress gap, they should refocus on a small number of critical, context-specific issues, such as improving the experience of transferring students or reducing student absence

4. Context matters

Most schools achieving exceptional progress for pupil premium students are in contextually advantaged circumstances, able to draw on more ‘hidden resources’ than other schools. Our estimates from our survey suggest that at least 15 to 20% of mainstream state secondary schools in England fall into this category. We use the term ‘hidden resources’ because these contextual advantages, which include historically strong reputations, high levels of parental engagement, and powerful alumni networks, are not easily identifiable from published school data.

Schools with significant hidden resources are more likely to take relatively simple approaches to the pupil premium, which would be insufficient to tackle the deeper levels of disadvantage found in other schools. Extreme contextual challenges include high levels of student mobility and large numbers of students who have experienced trauma or are in long-term care. Schools facing such challenges are more likely to have adopted compensatory initiatives aimed at increasing students’ cultural capital.

A school’s absence rate is the strongest predictive factor of the progress made by its pupil premium students. It is not currently the focus of pupil premium policy in most schools but could be used more extensively to identify students in need of support. In a minority of schools, addressing the challenges resulting from student mobility (transferring from one secondary school to another) is even more important to the progress of pupil premium students. We do not conclude that measures should be contextualised to account for schools’ circumstances, because our evidence suggests this cannot be done simply and is not supported by school staff. Instead, this research highlights the many different aspects of socio-economic disadvantage and raises questions regarding how to support schools and direct funds more effectively.

How important is school context to the progress of pupil premium students?

A school’s context is strongly associated with the progress made by pupil premium students. In line with previous studies ^{[footnote 15]}, it accounted for up to 55% of the variance in schools’ Progress 8 scores for pupil premium students in our statistical models.

This is likely to underestimate the importance of context because these models are limited by the data available. Our fieldwork indicated that there are further elements of a school’s context, which are not captured by standard school characteristic data, but which are associated with the progress of pupil premium students. The progress gap is particularly sensitive to these additional factors, which means its variance is less well modelled by standard school characteristic data (with a variance of up to 22%).

Although contextual features account for more than half of the variance in schools’ Progress 8 scores for pupil premium students, this does not set a limit on the impact a school can have. It also does not mean these factors cannot be addressed or mitigated by the school. The statistical associations do demonstrate, however, that schools are facing challenges which vary considerably in their nature and level of difficulty.

The school context also has an impact on a school’s approach to the pupil premium. When we grouped schools with similar strategies together, we found they shared common contextual characteristics:

• schools facing extreme contextual challenges are more likely to adapt the curriculum to suit their students, adopt initiatives to engage parents and support students transferring into the school; and be transparent about pupil premium and pupil premium status
• schools with ‘hidden resources’ are more likely to keep pupil premium status confidential, tailor resources, have a single point of contact for all pupil premium students, and provide direct access to pastoral support

Schools with hidden resources

Grammar schools

Since the introduction of Progress 8 in 2016, grammar schools have been four times more likely than other mainstream secondary schools to have a positive progress gap and more likely to sustain this year on year.

In 2019, the progress gap at grammar schools was half what it was at non-grammar schools.^{[footnote 16]} In the 13 grammar schools that responded to our survey, pupil premium students were progressing exceptionally in 12 of them.

It is plausible that the selection process itself contributes to the smaller progress gap. However, our study found that non-grammar schools, where pupil premium students made exceptional improved or sustained progress, are likely to share some contextual features with grammar schools. Typically, a school may have converted from a grammar to a comprehensive school in the 1970s and retained some of its prior ethos. It may, for example, continue to play sports tournaments with private schools, have a lower intake of pupil premium students, or retain a powerful alumni network. Our evidence suggests that these contextual advantages, which are usually associated with grammar schools, are also associated with small progress gaps in other schools.

Schools that share similarities with grammar schools

Ten of the grammar schools in our survey have adopted a similar approach to the pupil premium. Their primary strategy is to supply students with personal resources according to their circumstances. Because of the emphasis on personalisation, the systems in place to select and distribute these resources are different to most of our non-selective fieldwork schools, where pupil premium students in the same year were more likely to receive the same resources.^{[footnote 17]} In grammar schools, more members of staff, but particularly PE teachers and form tutors, are involved in determining what a student might need. In one school, for example, suggestions are recorded by staff in a student’s planner.

Grammar schools are less likely to take the ‘classic classroom approach’. In the most extreme case, one headteacher reported that his staff were not informed about which students were designated as pupil premium. In general, systems to support students are simpler. For example, pupil premium lead teachers at grammar schools are more likely to have regular, direct contact with every pupil premium student.

When we categorised schools, according to their shared characteristics with grammar schools, we found other areas of commonality. Through our fieldwork, we came to identify nine common features, listed in the ‘hidden resources’ box. We found that eight of our fieldwork schools providing exceptional progress possess at least four of these characteristics. In comparison, other schools possess less than two. These similarities are largely invisible in the school characteristic data published by the Department for Education.

These patterns have been found in a school sample that is too small for statistical tests to be meaningful. However, they do provide some evidence that hidden contextual factors are playing a role in the progress of pupil premium students.

The characteristics identified are listed as follows:

Hidden resources

1. The school was previously selective or partially selective (based on academic, musical or sporting ability, or religion).

2. There is a high level of parental participation. Parents do not only engage with the school about their own child, but also contribute to the educational experiences of other students, for example, by giving talks, making donations or arranging school trips.

3. A high proportion of a school’s students lives in areas of extreme affluence.^{[footnote 18]}

4. The school is historically oversubscribed and has had a good reputation in the local community for many years.

5. The school has a strong alumni network: former students play a role in the education of current students, for example, by contributing to careers events or sports events.

6. Pupil premium students are from backgrounds with relatively low levels of deprivation.^{[footnote 19]}

7. There are very few students from low-income homes.^{[footnote 20]}

8. The school has valued, long-standing staff members, who have developed successful curricular support systems, or initiatives over many years.

9. The school has stability of identity (it has been using the same building and a similar name for at least 25 years) or has outstanding facilities such as a performing arts theatre or state-of-the-art science block.^{[footnote 21]}

Case study 1: a school with hidden resources

School A lost its grammar school status more than 40 years ago. Its current admissions policy prioritises students from Catholic families. It has a low proportion of both pupil premium students and students with low prior attainment.

The school has exceptional sporting facilities and a strong sporting tradition: students have opportunities to take part in a variety of sports, including tours abroad. Sport plays an important part in the school’s identity and is a factor in students’ sense of belonging. For this reason, giving the Head of PE responsibility for the progress of all pupil premium students has been successful. He knows all these students individually, tracks their academic success, regularly checks on their wellbeing, and organises the tailoring of resources to their needs, which are privately provided to students.

For careers events, the school can call upon former students with aspirational jobs to share their stories. Current students are confident that they can succeed and that the school will help them to achieve their ambitions. They believe that the school is good and that they are lucky to be there. Teachers still fear that the school is not doing enough: some staff do not agree that funding trips or resources really makes a difference. They agree that the strong pastoral system, including peer mentoring, is important for all students, but it is not obvious to them how to further reduce the educational inequality within the school.

More detailed case studies from this project can be found in our pupil premium primer.

The importance of pupil premium students’ sense of belonging

Our fieldwork indicated a potential mechanism by which contextual advantages can contribute to the success of their pupil premium students. Students at schools with hidden resources are more likely to express a strong sense of belonging. Two Year 9 students, for example, compared their respective schools to their life at home:

School is basically like your second family.

It feels like a second home.

This was often connected with feelings of duty and pride. As a Year 10 student said:

I believe school’s more of a commitment you make when you come here.

Students are more likely to say that their school is good or that they are lucky to be there. It is plausible that elements of their schools’ history, reputation, or partial selection contribute to their sense of pride, commitment and belonging. This may be one way in which hidden resources have a positive impact on educational environments.

Evidence for this also comes from the observation that pupil premium students excelling in schools with fewer hidden resources, are more likely to demonstrate a sense of pride, commitment or belonging. One school purposefully attempted to increase students’ sense of commitment and pride in their work, which translated into commitment and pride in the school itself. ^{[footnote 22]}

The common approach to pupil premium in schools with hidden resources

Our (conservative) estimate is that approximately 15% to 20% of schools have significant contextual advantages.^{[footnote 23]} We have assumed that schools with four or more grammar school characteristics are beneficiaries of this. In reality, contextual advantage is a scale, as described in our pupil premium primer, which enables schools to compare themselves on a continuum with other schools to assess the hidden resources they have and the extreme challenges they face.

Almost nine in ten schools with hidden resources have practices in place that reflect their belief that pupil premium status should remain confidential (the figure is based on our survey)

Schools with hidden resources take a common approach to the pupil premium, which is simpler than in other schools.

They are more likely to:

• tailor resources to the needs of the individual students
• provide direct access to pastoral support so students can book appointments directly with counsellors or peer mentors
• have a single person who knows and monitors all pupil premium students
• have mixed attainment classes for students in Years 7 and 8
• use vertical tutoring
• prioritise keeping pupil premium status confidential

Although grammar schools are more likely to avoid using the classic classroom approach, we did not find this to be true of hidden resource schools in general.

Despite their relative success, teachers with pupil premium responsibilities at hidden resource schools are more likely to be at a loss for what to do next or how to improve the educational outcomes. They often expressed an interest in further help:

I would love to have had a spreadsheet or a document that says: ‘Listen. This is what’s working in other schools. These are the options you could do’, because I think we just fall back to the same thing.

The value of progress data given the importance of context

The typical view of school leaders is that the Progress 8 measure is important and valuable, but there are serious issues with the way it is used, interpreted or calculated. The most common problem raised is that Progress 8 data is used without context. While some academics have called for progress data to take account of contextual data, we did not find this view was generally supported by teaching staff.^{[footnote 24]}

It is also the case that proposed models only account for socio-economic disadvantage according to available school characteristic data. Our study shows that this overlooks important contextual factors which affect the progress of pupil premium students.

One headteacher felt he had a battle to prove the quality of the school to prospective parents:

Does it tell us about [the school]? No, because numbers never tell anything about a school, you need to have some context. … It’s a number… And I think you need to have context behind those numbers. You need to have a dialogue behind that and that’s what I find very frustrating as a school leader. My school is good in Ofsted’s eyes, but my headline data is not even, well, it’s closer to zero than we ever have been but it’s not smashing it out the ballpark. And so people say, ‘Oh, that’s not a good school’. (Headteacher)

Another headteacher believes the issue is exacerbated by the way the Department for Education publishes Progress 8 scores using a traffic light system on the government website that compares schools. He expressed frustration that parents saw progress data only in simple categories of ‘good’ or ‘bad’, which he did not believe was a fair interpretation of the data.

Five teachers described their ambition of “getting into the green” or “turning green”, by which they mean achieving a positive Progress 8 score. One teacher saw this as unproblematic:

My focus is to get as close to, as close in the green as possible. Yeah, I do not want a minus number here … 0 would be perfect. Yeah, because it shows that we’re doing our job right. (Pupil Premium lead)

More commonly, however, teachers expressed frustrations with such a simplistic aim, which too easily resulted in focusing on making the school appear good to Ofsted or prospective parents.

As reported in the last section, external pressures are plausibly driving schools to take shorter-term pupil premium strategies, instead of prioritising what they believe to be most effective, and to adopt more simplistic approaches to identifying students for additional support. Some teachers therefore feel that the measure distracts from the real purpose of supporting pupil premium students:

But if we’re trying to say that we want to narrow the gap between our pupil premium students and our non, both in a Progress 8 way, but really, who cares about that rate? … What we really want is [for] those pupil premium [students] to go on and be really successful 20 years from now!

These concerns are felt even when teachers believe that their school is dealing well with the pressure:

The school is principled enough to put student wellbeing above Progress 8 and success for students in receipt of PP should not only be measured through exam grades. (Middle leader)

This contrasts with views expressed in our schools with significant hidden resources. Most of these schools had not been inspected by Ofsted for more than five years and leaders expressed anxiety about the latest inspection framework because it’s less reliant on attainment and progress data.

The Progress 8 calculations were criticised for two reasons. Firstly, it is generally perceived that its weighting towards the English Baccalaureate subjects put pupil premium students at a disadvantage:

So P8 [Progress 8], it’s not the measure itself that’s the biggest issue, I think it’s the way it might be used. We need to be more inclusive and we need to be thinking particularly some children might not need a diet of a curriculum that fills a full eight buckets. What needs to be looked at is a more measured approach. (Headteacher)

And so while students may not achieve their full eight subjects and achieve you know great progress in Progress 8 score, actually what they do achieve is something they would not get somewhere else … And so those students will actually gain a lot of other skills that are not measurable in a Progress 8 way. (Middle leader)

I do not think you have that facility [to use professional filmmaking software] in a school following the Progress 8 kind of EBacc [English Baccalaureate] curriculum, because it’s so restrictive. (Headteacher)

Secondly, although recent changes to the calculation of Progress 8 (to exclude some students) were positively received, teachers feel this has not gone far enough. Some were struggling with the impact on their data of high levels of student mobility (discussed in the next section).

Persistent disadvantage

A student is considered to face persistent disadvantage if they have received free school meals for over 80% of their time at school (the Education Policy Institute use this definition in their annual report 2020).

Persistent absence

A student is defined as persistently absent if they miss more than 10% of school lessons.

In one school, teachers believed that students in one year group would achieve low progress scores because SATs in a feeder school had been annulled and awarded (potentially too highly) by staff. Most commonly, school leaders are concerned about who gets included or excluded:

We had two identical bodies of evidence submitted to the DfE [Department for Education]. They took out the one that was not pupil premium and they left in the one that was pupil premium, which was brutal … It was totally unfair. (Assistant headteacher)

The underlying issue is that school leaders and teachers feel that simplistic presentations of the data, for example in league tables, contribute negatively to school competition. As described in the next section, it encourages schools to roll-off students who are underperforming, refuse to take in student transfers, and seek short-term solutions to complex issues.

In summary, most teachers in our study feel strongly that a progress measure is needed. The majority, however, express frustrations with the calculation and use of Progress 8 data, given the complexity of each school’s context. The study uncovered many plausible negative impacts from the poor use and presentation of this data, although we cannot demonstrate how widespread this is, or how this could be combatted. It seems clear that simple contextualisation of Progress 8 data would not solve the problems raised by teachers. We therefore suggest that further work is carried out to review the calculation, use and presentation of progress data, to reflect the importance of school context.

Recommendation

The Department for Education and Ofsted should: review how progress data is presented and used; put measures in place to try to prevent data from being interpreted crudely without acknowledging schools’ contexts and hidden resources; and recognise that some schools need to refocus pupil premium on a small number of critical issues.

Schools facing extreme contextual challenges

Schools which prioritise cultural capital as part of their pupil premium strategy are more likely to face at least four of the following extreme contextual challenges:

Extreme contextual challenges

1. High levels of severe household deprivation, for example, being located in an area of very high unemployment or having large numbers of students facing persistent disadvantage.

2. High proportions of students in care.

3. High levels of students arriving in school with very low levels of English.

4. High mobility rates, for example, one school receives more than 200 student transfers each year.

5. High proportions of students directly involved in crime (as a victim or perpetrator).^{[footnote 25]}

6. High proportions of students with open safeguarding concerns, for example, one school had open safeguarding files for more than half its students.

7. Poor reputation despite a current rating of ‘good’ or more by Ofsted.^{[footnote 26]}

8. School competition: in close proximity (less than a 15 minute drive) to at least one other school with an outstanding reputation.^{[footnote 27]}

9. High rates of persistent absence over which the school has little control (for example, as a result of family breakdown or illness).^{[footnote 28]}

10. High rates of students with special educational needs or disabilities.^{[footnote 29]}

Case study 2: A school facing extreme contextual challenges

School B is a coastal school, located in a deprived neighbourhood, with high rates of unemployment and crime. The crime beyond the gates filters into the school: the latest year of results included students who were unable to attend school because of their involvement in crime. One student stayed away from school because of threats made by a drugs gang. Another, on bail for a serious crime, posed too much danger to other students and staff to be allowed on site.

Many of the staff are dedicated to working in these conditions and are intensely focused on the pastoral needs of their students, including the large school community of children in care:

We are really student-centred … because we have such a difficult intake, for lots and lots of reasons, that if you do not put students in the front of your thinking and decisions, then you’re lost.

The school has opened a centre to support asylum seekers, most of whom have arrived in the country with no knowledge of English; some of them have lost both parents in conflict.

Teachers feel strongly that Ofsted inspectors have little understanding of the severity of the challenges faced by staff and students and that the school has been left to cope with the reality of supporting young people facing severe social, mental and personal issues. As one teacher said:

Battling those problems is bad enough, but it is all about the individuals … comparing a random selection of our students with a random selection of students from some [other] school, so you can make some meaningful comparison, is sort of an irrelevance.

More detailed case studies can be found in our pupil premium primer.

The challenges of student mobility

Student mobility, which is the transfer of a student between two secondary schools, is not evenly spread through the school system. Last year, one of our fieldwork schools received eight new students from other secondary schools, while another smaller school received almost 200. Some schools are dealing with a constant stream of new students and research shows that this is a critical factor in the progress of pupil premium students.^{[footnote 30]} This research finding is supported by our fieldwork and survey data.

Student mobility

Student mobility is the transfer of a student between two secondary schools. School transfer has a negative impact on academic attainment. Pupil premium students are more likely to move secondary schools than their peers

Leaders in schools with extreme mobility rates feel let down and frustrated by other local schools, which they say are reluctant to take new students. We are able to confirm that such attitudes do exist. One leader of an oversubscribed school admitted that their efforts concentrated on excluding certain students, rather than settling new students in. Another admitted that their school had tried, but failed, to off-roll three students in the previous year.

Schools facing extraordinary levels of student mobility have Year 11 classes in which fewer than half of the students have remained in the school since Year 7. In such circumstances, schools may devise their own methods to account for the effects of mobility in their performance data. As one headteacher, said: “The data does not reflect the mobility issues you’ve got”.

It is a time-consuming task, but such schools attempt to give a better reflection and evaluation of their work by considering and comparing the progress of students according to how long they have been in the school. A school’s lead teacher for children in care also pointed out that it was insufficient simply to exclude some students from the Progress 8 data. A disproportionate amount of school resources is allocated to these students and the school’s work with them needs to be recognised.

These schools are also faced with the additional burden of large numbers of students transferring from other secondary schools. They are more likely to have a stronger focus on the transfer process: pastoral and academic transfer processes take up to a term for students most in need.

More than half of teachers surveyed report that they don’t usually receive enough curriculum information when a new pupil transfers from another school

In general, our research shows that effective policies and processes are not in place for transferring students. Our survey reveals that schools in which teachers are confident about support for new students have higher rates of progress for pupil premium students. However, most teachers feel that their school could do more to provide them with more information on newly transferred students. In a series of workshops with school leaders, we discussed the issue of raising the priority of transferring students. As a result, we are proposing that schools receive a higher rate of pupil premium for these students. It was emphasised during these workshops that the money should be timely, and provisos should be in place to ensure this policy does not encourage exclusions.

Recommendation

The pupil premium criteria should be reviewed to consider whether it can support schools facing high levels of student mobility or persistent absence^{[footnote 28]}. For example, schools could receive an additional pupil premium, paid termly, for students transferring from other secondary schools (provided that permanent exclusions do not rise significantly).

The importance of student attendance

Our statistical models indicate that the strongest predictive factor of the progress made by pupil premium students is the school’s absence rate:^{[footnote 31]}

• schools with lower absence rates have smaller progress gaps
• pupil premium students progress more at schools with lower absence rates

This correlation is regardless of whether they begin with low, medium or high rates of absence.

These findings concur with previous research and we share their interpretation too, that this correlation is most likely to be causal. This is because there is an intuitive underlying causal mechanism: students not in school are less likely to learn the school curriculum.^{[footnote 32]}

The findings are partially supported by our fieldwork, which shows that schools with exceptional sustained progress for pupil premium students have lower absence rates (5.6%) than other schools (6.0%). However, there is no statistically significant difference in absence rates between schools with exceptional improved progress for pupil premium students (6.1%) and other schools (6.0%).

Not all strategies for improving attendance are clearly effective

Our survey indicates that how schools approach the challenge of improving student attendance is important. Not all strategies are associated with higher progress for pupil premium students:

• a home-centred approach to student attendance (for example meeting parents, making home visits, collecting students from home) is associated with higher progress
• a punitive approach to student attendance (for example giving detentions, sending warning letters, applying fines) is not associated with progress
• a reward approach to student attendance (for example holding reward assemblies and giving prizes) is also not associated with progress

The positive impact of a home-centred approach is further evidenced in our fieldwork. The school with the most improved attendance (and corresponding progress for pupil premium students) has a home-centred approach, including phone calls home every day to arrange collection of students by the school’s minibus.

There is still room for improvement

There was a steady decrease in student absence at secondary schools in England from 7.8% (in 2006 to 2007) to 5.4% (in 2013 to 2014).^{[footnote 33]} Since then, student absence has levelled off at around 5%.^{[footnote 34]} Our fieldwork indicates that there is a sense in many schools that once they have reached this target, their work is done. At one school, despite its slightly higher-than-average absence rate, a leader stated: “Funnily enough, no, we don’t have an attendance issue.”

It is also true that although all school leaders we met are aware of the impact of low attendance, and all schools monitor attendance closely, no school regards it as a central focus of their pupil premium policy. Attendance is also most often cited where schools express a desire for outside help.

Recommendation

When schools have had little success at reducing the progress gap, they should refocus on a small number of critical, context-specific issues, such as improving the experience of transferring students or reducing student absence

Other contextual factors

As outlined, student attendance is the most stable and significant predictor of our statistical models, and highlighted in the contextual data available about schools at a national level. It accounts for approximately 15 to 20% of the variance in the progress of pupil premium students. ^{[footnote 35]} This is supported by our fieldwork.

Other factors associated with the progress of pupil premium students have smaller effect sizes or are not all stable under statistical modelling. Some conflict with the outcomes of our fieldwork.

The ethnic diversity of the school

The second most predictive factor of pupil premium progress is the ethnic diversity of the school. There are many different markers of a school’s diversity, which produce similar results when used in statistical models, all of which are in line with previous research. The following attributes are linked with higher progress for pupil premium students:

• schools with a lower proportion of pupils from White ethnic backgrounds
• schools with higher proportions of students with English as an additional language (EAL)
• schools with higher proportion of pupils from Asian and Chinese ethnic backgrounds

Because these factors are highly associated with each other, statistical models that include them all are unable to identify robustly which is the most relevant factor.

Our fieldwork indicates that the impact of ethnic diversity is even more complex in practice. In some of our schools, having high proportions of EAL students represents a contextual disadvantage. In comparison to schools with hidden resources, whose EAL students are more likely to be bilingual, some schools have high numbers of students who arrives at the school with little or no prior English. Such schools must deploy significant resources to support these students and their families, such as translators, support staff to run centres for asylum seekers, and extensive literacy programmes.

One headteacher described how proud he was proud of the diversity within his school; however a rise in the number of students from ethnic backgrounds had impacted negatively on the school’s reputation and the choices local families were making regarding where to send their children.

The location of the school

In schools located in certain areas of England (especially the south-east, south-west, east and north-west) the progress of pupil premium students is poorer than in schools in London and the north-east. However, this association is not stable across different statistical models. This means that location is firmly entwined with other factors which cannot be separated by multivariate linear modelling.

The same is true for schools located in rural areas, in which pupil premium students make less progress than schools in urban areas. However, the associations are not stable across different statistical models to the extent that this reduces confidence in the association.

Our fieldwork did not reveal any differences in schools’ pupil premium strategies according to their region or location (rural, urban, coastal). We visited schools with hidden resources in rural and coastal areas, as well as schools facing extreme challenges.

The impact of being in a school with lots of other pupil premium students

On the face of it, when there is a larger proportion of pupil premium students, these students make less progress than their peers. This association reverses, however, in some models. Statistical modelling is unreliable in this case, because the proportion of pupil premium students is highly correlated with many other factors.^{[footnote 36]}

The fieldwork indicates that schools where they make exceptional progress are more likely to have either a very small or a very large proportion of pupil premium students. Schools with less extreme proportions (likely to be defined as ‘local schools’ in our school typology) find it much more difficult to help such students to progress.

In schools with many hidden resources, pupil premium students are more likely to progress well. However, schools facing extreme contextual challenges are more effective in helping these students if they have high numbers of them. Our evidence suggests that the reason for this is a united staff culture which prioritises socio-economic disadvantage, as the next section outlines.

5. Staff culture matters

Our study emphasises that the success of any pupil premium strategy depends on the school’s context and the ways in which that strategy is implemented. We found that elements of school’s organisation, notably vertical tutoring and setting arrangements, are strongly associated with the progress of pupil premium students. Significantly, our fieldwork also shows that staff culture is crucial in effectively tackling socio-economic disadvantage. The culture includes the attitudes, capabilities, daily practices and determination of the entire team.

A significant minority of school staff express reservations about the pupil premium; however schools that provide exceptional progress have fewer teachers with these concerns. We also observed that many schools do not consider the impact of their policies on students’ wellbeing. Students report that the policies are not always as confidential or effective as teachers believe.

We are unable to identify approaches that could reduce the progress gap in all schools, or even the majority of schools. However our evidence suggests that leaders should consider the attitudes and opinions of both staff and students when forming their pupil premium policy. Our pupil premium primer provides further details of our suggestions for school leaders who are wondering: “Why doesn’t it work here?”

An approach can have different results in similar contexts

Our survey did not reveal associations between the progress of pupil premium students and the following interventions:

• continued professional development
• pastoral interventions
• extra staffing or teaching assistants
• smaller class sizes
• Year 6 transition arrangements ^{[footnote 37]}
• taking a monitoring, punitive or reward approach to student absence

There are many potential reasons for the lack of statistical evidence to support a particular intervention. Our fieldwork shows that approaches that worked for some schools were not successful elsewhere. These included:

• appointing pupil premium champions
• taking the ‘classic classroom approach’
• establishing nurture groups
• providing study skill lessons

Recommendation

School leaders should not just consider what pupil premium approaches to implement, but how to implement them effectively in their school context.

Comparison of the impact of the ‘classic classroom approach’

Schools P, Q and R are similar in their characteristic data. Also none of them face extreme contextual challenges or have a significant amount of hidden resources.

School P has helped its pupil premium students achieve exceptional progress. In 2018 it achieved a positive progress gap for the second time. The headteacher attributes this to the relentless focus on giving pupil premium students additional attention in the classroom, including interventions such as marking their books first. This increases teachers’ awareness of these students and the school has seen improvements in outcomes for all students.

School Q also helps pupil premium students achieve exceptional progress. However, the headteacher disagrees with the strategy of treating students differently and purposefully avoids this:

I deliberately didn’t, when I came in, identify pupil premium kids on your lesson plans or anything else like that, you know, all of those things. We haven’t done all those things that the toolkits used to tell you to do, years ago. We just didn’t do it.

The school has achieved its positive progress gap over the last two years without using any intervention associated with the ‘classic classroom’ approach.

Pupil premium students at School R do not achieve exceptional progress. For many years, they achieved lower levels of progress than the national average. In the last two years, the school has implemented interventions for pupil premium students including training staff, creating guides to remind them what to do, and sharing classroom seating plans across the whole school. However, the school’s results have not improved and one teacher suggested that they simply weren’t working.

Common features of the staff culture in successful schools with few hidden resources

Using estimates from our survey, pupil premium students make exceptional progress in less than 4% of mainstream secondary schools without significant hidden resources. Less than a quarter of these face extreme contextual challenges - approximately 30 schools, in our analysis.^{[footnote 38]} We conducted fieldwork in six such schools and found that, despite not having a pupil premium strategy in common, there are striking similarities in their ethos and culture:

• support staff feel valued and view the pupil premium policy as a critical part of their role
• pro-active and collaborative data monitoring is undertaken to support frontline staff with curriculum, planning and strategic decisions
• key staff members are deployed in critical positions who demonstrate exceptional determination, positivity, or skills relevant to the circumstances faced by pupil premium students
• school staff agree with the approach of the senior leadership team and are especially sensitive to the individual needs of pupil premium students
• interventions are frequently adapted to the needs of students and relentless efforts are made to meet these needs

These are features of the staff culture (defined as the attitudes and daily practices of staff), rather than the particular intervention chosen. This does, however, have an impact on the way in which any interventions are implemented.

In comparison, pupil premium leads in schools with large progress gaps reported feeling “lost” and “on an uphill battle”.

The role of support staff in reducing the progress gap

In schools that achieve excellent progress, that don’t have significant hidden resources, support staff are given key roles addressing socio-economic disadvantage:

What I love now is children have got a greater relationship with more support staff. (SEN Coordinator)

We’re very much encouraged to be pupil-focused and for support staff to be part of that family. (Teaching assistant)

Staff supporting pupil premium students demonstrate passion for their role and are deployed in ways that harness their skills. At one school, for example, a teaching assistant who specialises in literacy is available for library drop-in sessions at lunch times. She helps students choose books and supports them with their English homework. The school introduced these initiatives in response to low levels of literacy identified in their Year 8 pupil premium students.

Case study 3: Proactive data management

In School X, a data manager analyses performance data across all subjects, and provides school leaders and teachers with information. Because he has been in the role for four years, he is familiar with trends across subjects and is quick to see, for example, when pupil premium students in a particular class are losing ground to their peers.

Data on student performance is collated three times a year in a collaborative process. Working with the data manager, each curriculum area has developed models of what work from each grade looks like. This creates a store of exemplars which improves the reliability of year-on-year comparisons. The data review compares pupil premium and non-pupil premium students in and across different cohorts, according to predicted grade, gender, subject and classes. It is not used for staff appraisal. The purpose is to provide evidence of impact and pre-empt any issues as early as possible.

More detailed case studies from this project can be found in our pupil premium primer.

Deploying key staff in critical positions

In the majority of schools without significant hidden resources, support staff play an important role in achieving exceptional progress for pupil premium students. These schools are also more likely to have other staff in critical positions with positive views about the pupil premium; personal experiences of growing up in the area; or of receiving free school meals.

In these schools, teachers with responsibility for the pupil premium do not feel despondent at the challenge before them:

I do not think that … there could ever be enough done to support pupils from disadvantaged backgrounds.

Their positive attitude is also demonstrated in the routine practices of their colleagues, such as sharing relevant information in the staffroom, monitoring data carefully, and constantly adapting initiatives to improve the outcomes. One school moved their parents’ evening to the morning when it was poorly attended, another changed its policy to telephone parents instead of emailing them, and a third began translating emails into the languages of their community to improve feedback.

Two schools believe that their progress is highly dependent on an effective pupil premium lead, who creates a positive culture and ethos throughout the school. When one of these schools lost their lead, they say it resulted in declining outcomes for pupil premium students.

In another school, teachers report that the ethos and drive comes from an exceptional headteacher, who has created a united culture on tackling disadvantage. All six staff interviewed here tell a similar story, of being part of a school family and the importance of supporting all students.

As the pupil premium lead puts it: “There’s a very strong culture around improving social mobility and social justice.”

The students at the school echo these sentiments: one Year 7 student describes how the teachers “feel like a special kind of team”.

Understanding and tailoring to the needs of pupil premium students

Schools with hidden resources are most likely to provide pupil premium students with materials that meet their individual needs. Most commonly teachers raise requests for specific resources, such as highlighter pens and shoes. This method relies on the staff knowing students well and some teachers are sceptical that the approach can always identify the most important needs.

In schools with higher proportions of pupil premium students it is more common to provide the same resources to all these young people. Usually, they are given revision textbooks, although some schools provide additional resources in a more tailored and expensive approach. Examples includes providing a bike, paying for a family’s Christmas, and giving a student a piloting experience. One school provided every single student with vouchers to spend at their shop. Three schools tailored their resources by asking teachers to ‘bid’ for pupil premium funds.

Schools with fewer hidden resources are less likely to tailor their resources but more likely to tailor their approach to pupil premium students. An example of this is the design of an alternative sporting curriculum, for a particular cohort of students.

One school where the students achieve exceptional progress provides teacher training on the experiences of pupils living in poverty. Other leaders express the concern that even the most dedicated teachers do not fully understand what their students’ lives are like. As one headteacher said:

I am very conscious that almost by outcome, teaching is a middle-class profession and we’re fairly well paid (could be more well paid). But you move more into the middle classes and perhaps maybe you were never working class and struggling, or maybe you were, but you have forgotten what it’s like.

The impact of vertical tutoring

Our survey analysis shows that schools using vertical tutoring have a significantly smaller progress gap than other schools.^{[footnote 39]} This finding (involving 33 vertical tutoring schools) was replicated by further data collected via social media and website searches, providing a secondary sample of 62 schools. These schools also show a similar higher rate of progress for pupil premium students.

What is vertical tutoring?

This is the organisation of form or tutor groups whereby students from different year groups are mixed together in the same form or tutor group.

Fieldwork also indicates that in schools where vertical tutoring has been a success, teachers say it contributes positively to the culture of the school and particularly supports pupil premium students.

However, our fieldwork also reveals that up to 10% of schools have failed to successfully implement vertical tutoring. Some teachers have had negative experiences of vertical tutoring. In particular, the changeover period can be very challenging, because students and their families often resist the change.

You can read more on our research into vertical tutoring in our pupil premium primer.

Figure 7. Estimates of the use of vertical tutoring^{[footnote 40]}

Type of tutoring	Estimates of the use of each type
Vertical tutoring	12%
Horizontal tutoring (not tried vertical tutoring)	78%
Horizontal tutoring (Unsuccessfully tried vertical tutoring)	10%

Mixed attainment classes

Our survey shows that teaching Year 7 and 8 students in mixed attainment classes for all subjects is associated with higher progress for all students, but particularly pupil premium students.^{[footnote 41]} This data therefore suggests that setting affects pupil premium students more acutely than others. However, the data is not highly reliable because only 80% of teachers agree with each other when describing their school’s setting arrangements.

The lack of a whole-school approach to setting

Our survey data indicates that even limited setting (for example, in core subjects) can have a significant negative impact on the outcomes of pupil premium students across all subjects. There may therefore be benefits in adopting a whole-school approach to setting to mitigate the negative impacts of whichever arrangements are chosen.

Our fieldwork reveals, however, that setting is widely neglected in school policy. The uncertainty in the data is primarily caused by teachers being unaware of setting arrangements outside their own department. Some senior leaders also struggle to accurately describe the setting arrangements across their school. For example, in one school the headteacher reported that all Year 7 classes were mixed attainment, but Mathematics and English teachers disagreed. In more than three-quarters of our fieldwork schools, setting arrangements are determined at department level.

Constraints on departmental setting policy

What happens in practice in each department is constrained by the opinions of staff, pressures from parents, school customs, and timetabling pressures. There are therefore many tensions between staff regarding setting arrangements. One teacher explained why he had been “battling” to change the setting arrangements:

We do not set here until Year 10, and [mixed attainment groups] is something that really I am not a fan of at all … This idyllic notion that you put a really strong kid with a really weak kid, and you put them working together, and the weak kid comes up and the strong kid feels good: it’s not the case. It’s nonsense.

As a result of navigating these many constraints, some schools regularly change their setting arrangements, use different arrangements for different cohorts, and employ a variety of ‘nurture groups’ (for students with lowest attainment) and ‘grammar sets’ (for students with the highest attainment). This is even in schools with a mixed attainment approach.

We found inconclusive results from the use of nurture groups. In general, the variety of arrangements and opinions expressed mean that we are unable to find associations between setting arrangements, or teachers’ attitudes to these arrangements, and the progress of pupil premium students.

Students’ positive attitudes towards setting

On the whole, our focus groups with young people reveal very positive attitudes towards the setting arrangements in their particular schools (this is in schools where pupil premium students make exceptional progress). This is not entirely consistent with recent research that emphasises the negative impact of setting on pupil premium students.^{[footnote 42]}

It does, however, support the view expressed by some teachers, that students in lower sets are aware that they benefit from smaller class sizes and are happy to receive the additional help. Students are most concerned about being put in the wrong set, as a result of teachers’ misapprehensions. Many find setting motivating:

I think it’s good in a sense because you just know where you’re working at and they strive you to push further and work harder to where you want to get.

At one school, teachers believe that streaming in Year 7 (instead of setting) is contributing to good outcomes because the students are largely unaware of it. Our survey also indicates that pupil premium students make more progress in schools that stream (rather than set). A number of schools have taken action to make setting less visible, for example by labelling sets with teachers’ initials. At one such school, this has coincided with improved performance for pupil premium students and the change in labelling could partially be responsible for this.

Our findings suggest that future research should not merely compare schools that set with those that do not, but also consider the many strategies that schools can take to mitigate the negative effects of setting. What is clear is that setting is a complicated and neglected area, where teachers, students and school leaders have varying opinions.

Recommendation

Large-scale, longitudinal research, with schools as active research partners, should be conducted to investigate the impact of school organisation and culture on the wellbeing and achievements of the most disadvantaged students. The analysis should include an examination of vertical tutoring and setting arrangements

Staff attitudes to the pupil premium

The evidence from our study suggests that staff attitudes are an important factor in creating a positive culture to successfully tackle socio-economic disadvantage. The school workforce shows various attitudes towards the pupil premium. A minority of teachers have reservations about it, which can be a barrier to successfully implementing approaches. In many cases, these beliefs are very strongly held and in some schools as many as a third of teachers express reservations to the pupil premium. We have classified these attitudes into the following three categories.

1. Tackling socio-economic disadvantage via education is “hopeless”

Some teachers express the belief that the overall aim of the pupil premium is hopeless, because socio-economic divisions cannot be addressed through education:

The effects of poverty often range beyond the remit of a school. I also think secondary schools have the difficult task of trying to make a difference when the child has already been affected by their circumstances for over eleven years. (Classroom teacher)

The influence of students’ home lives and immediate social groups has a bigger impact than can be undone by a school. (Middle leader)

Often, the socio-economic factors that are ‘stacked against them’ possess too much weight for the students’ path to be intercepted in the way in which educators aim to. (Classroom teacher)

Our fieldwork suggests that these attitudes are more likely to be found in schools with larger progress gaps. We were able to test this hypothesis by administering our questionnaire to more than 100 staff in two schools. Both schools face extreme challenges and have significant hidden resources. One of the schools has a larger than average progress gap; in the other, pupil premium students are progressing as well as their peers. We found the difference in staff attitudes to be statistically significant: 80% of staff in the school with the zero progress gap agreed with the statement that “schools can make a difference to young people living in poverty”, but only 60% did in the less successful school. In both schools, these reservations are more commonly expressed by middle leaders, and least likely by senior leaders.

2. Pupil premium is “discriminatory”

Some teachers feel that the pupil premium is not fair. On two occasions, teachers used the word ‘discriminatory’ with some hesitancy, aware that they were saying something controversial. One deputy head suggests this belief is widespread at their school:

Some teachers think the PP strategy is bigger than the … school strategy [for all pupils] and that it is inequitable.

Teachers’ own experiences are crucial: one recalls seeing a pupil premium student being brought to school in a BMW every morning. Another describes the frustration of seeing a student who “desperately wanted help” not being allowed into a GCSE revision session because she was not eligible for the pupil premium. An experience like this “changes how you feel”.

Other teachers describe the discomfort with the pupil premium in general:

I feel very uncomfortable about a strategy which by definition makes some kids more important than others.

Some students seem to get priority treatment, others are ignored.

In a survey of almost 5,000 secondary school staff (described in more detail in our pupil premium primer) we found that 10% feel that the pupil premium is “unfair” or only “slightly fair”. Teachers at schools where pupil premium students are not targeted as a group for support, or where there is a high proportion of pupil premium students, are less likely to hold such beliefs.

3. Particular initiatives are ineffective

Some staff say that although they are not against the pupil premium, they are unconvinced by their school’s choice of strategy. The ‘classic classroom approach’ came under fire most frequently, particularly marking the books of these students first, seating them at the front of the class, funding trips and revision guides:

Whenever I hear, ‘We take them on trips and we buy them books’, and you think, you know, where’s the imagination? Two hours, once a year, is not cultural capital. (Head of department)

These attitudes are expressed by teachers in schools with hidden resources that have varying outcomes for pupil premium students. They are also found in schools in more challenging circumstances where pupil premium students have not made exceptional progress.

In the schools with the smallest progress gaps, teachers are united in their efforts and agree on the school’s approach to pupil premium. The classroom teachers reported:

We have a successful, a whole-school approach, to identifying need.

Pupil premium students, to my knowledge, are supported well.

Further details on teachers’ attitudes can be found in our pupil premium primer. This includes guidance and a questionnaire on how schools can effectively investigate the attitudes of their staff.

Schools are unaware of the impact of their policies on pupil premium students

In almost all schools we found that very little effort had been made to find out how pupil premium policies affected the wellbeing of these students. Most schools assume that students are largely unaware of the machinery that is in place behind the scenes to monitor their progress and provide them with additional daily support through questioning, marking and seating plans. They believe students’ pupil premium status is confidential, and that there is no stigma attached to being eligible for pupil premium.

Some pupil premium students reported to us that they were indeed unaware of what pupil premium meant in the early years of secondary school. As a result, it was confusing to be handed free equipment or given money back. Students told us that they often asked for explanations, but teachers were reluctant to give them. One sixth-former explained:

Like the teacher did not understand it himself … or he just didn’t wanna be the one to explain me … He said to me, ‘You do not need to pay that.’ He was just, ‘it’s like you’re lucky or something.

When we asked teachers whether the pupils knew about their status, we found that a few schools, with high proportions of these students, are very open about this:

We’re making a student want to be pupil premium: proud of it rather than hiding from it. We do not want it to be a hidden thing, we want it to be promoted.

In the majority of schools however, teachers are unaware of what their students know and are therefore unable to talk to them about it or to say whether their policies have an impact on their wellbeing, self-esteem or sense of belonging.

Students tell us that the pupil premium status is not negative in itself, but that it is important that it is kept confidential, and that its role is explained to them, their peers, and their family:

It can be quite embarrassing and humiliating…being known like, ‘the special one’ is what people say. (Year 8 student)

I feel like [it] should be explained to parents as well, right? Cos when I first told my parents why, they saw it as like a really negative thing. They thought that they were judging them. (Year 12 student)

Our focus groups showed that students do not share the same attitudes and beliefs as staff. Students do not always agree on whether a particular policy is effective or confidential, what is working well at their school, or what is most important. At one school, for example, staff are proud of an initiative to provide pupil premium students with small Christmas gifts, believing this makes them feel important. Conversely, a student describes the process of being called to reception to pick up “a huge package” as “embarrassing”.

In general, students are much more focused on the financial aspects of the pupil premium and school life, compared with school staff. The most common topics raised in student focus groups are the cost of food, drinks, school uniform, equipment and the lack of subject choice in their school. Staff are more likely to be concerned with cultural capital and educational support at home. This results in a disconnect between staff and students in some schools. One group of students, for example, said that they had been campaigning for lower canteen prices, but that teachers did not even understand the problem, let alone address it. Traditional methods of listening to the student voice, such as school councils, are not working to bridge the gap: students talk of the difficulties of being selected, the ineffectiveness of these methods, and their dominance by popular students. We have collated a more detailed summary of the comparison between students and staff views in our pupil premium primer.

Recommendation

School leaders should regularly explore staff attitudes towards the premium, how well their school responds to the needs and circumstances of pupils, and the impact of their policies on students’ wellbeing.

The long-term challenge facing “ordinary” schools

Our findings regarding staff culture suggest that reducing the progress gap requires a more sustained and long-term effort than is sometimes recognised by teachers and school leaders. It is for this reason that we propose a national award for pupil premium lead teachers to help develop a deeper understanding of socio-economic disadvantage and more consistent approaches to tackling the progress gap.

Because of the nature of this study, we have not been able to follow a school as it builds a positive staff culture regarding the pupil premium. It is therefore not possible for us to say how to create such a culture. Similarly, although we have seen that the success of an initiative depends on the culture and context in which it is implemented, we are unable to provide definitive guidelines regarding how to do this in any particular school. Instead, we have developed case studies of schools and approaches that address their context and students’ circumstances so that we can provide school leaders with some inspiration and reference points. The purpose is not to give a step-by-step guide of what to do, but to help school leaders create strategies and approaches that are more likely to have an impact in their own context.

The evidence indicates that we can make a difference, but that it is a more challenging and collaborative task than we may have previously supposed.

Glossary

Chapter 1 (Introduction)

Pupil premium

A supplementary school fund that is provided to a school for every student on their roll who has received free school meals in the last six years, is from a military family, or has ever been in care.

Progress 8

Progress 8 is a score calculated for each student by comparing their GCSE results with the results of peers who achieved a similar level of attainment at the end of primary school. A score of 0 means that a student has achieved similar attainment to their comparable peers.

Progress gap

In this report, ‘progress gap’ refers to the difference between a school’s Progress 8 score for its pupil premium students, and the school’s Progress 8 score for its non-pupil premium students. A positive progress gap means that pupil premium students have made more progress than their non-pupil premium peers. Unless otherwise specified, this report is concerned with the negative progress gap, which is defined as the disparity between the progress made by pupil premium students compared with the greater progress made by their non-pupil premium peers. The word ‘negative’ may be omitted to avoid repetition.

Chapter 2 (Methodology)

Cultural capital

A student’s cultural capital is the total non-financial assets that the student has, as a result of their cultural knowledge and experiences.

Exceptional improved progress

We categorised a school as having exceptional improved progress for pupil premium students if one of the following applied:

a) the school had a positive progress gap (in the last two years) and had two consecutive years of improvement to the Progress 8 score for pupil premium students (totalling at least 0.3)

b) the school’s last Progress 8 score for pupil premium students was positive and the school had two consecutive years of improvement to the Progress 8 score for pupil premium students.

Exceptional sustained progress

We categorised a school as having exceptional sustained progress for pupil premium students if one of the following applied:

a) the school had positive progress gaps for at least two years in a row

b) the school had one positive progress gap (in the last three years) and the Progress 8 score for pupil premium students was positive for at least two years in a row.

Hidden resource schools

Hidden resource schools benefit from additional resources, capacities, and individuals, which are not immediately obvious from standard school data and are not available to all schools. Hidden resources relate to factors such as school ethos and environment.

Local schools

Local schools have few hidden resources and few extreme challenges. Typically, most students are from the immediate local area and there are no obvious divides in the communities served by the school.

Schools facing extreme circumstances

Some school communities face extreme challenges that have an impact on daily life in a school. Examples include knife crime in the local community; high numbers of transferring students; and long-term difficulties in recruiting staff.

Two-in-one schools

Two-in-one schools face extreme challenges as well as having significant hidden resources. Typically, a two-in-one school serves a large proportion of students from very affluent homes, as well as a number of students facing severe socio-economic disadvantage.

Vertical tutoring

An organisation of form or tutor groups whereby students from different year groups are mixed together in the same form or tutor group.

Chapter 5 (Staff ethos and culture matters)

Horizontal tutoring

This is the most common organisation of form / tutor groups, whereby each form / tutor group is composed of students from the same year group.

Persistent absence

A student is defined as persistently absent if they miss more than 10% of school lessons.

Persistent disadvantage

A student is considered to face persistent disadvantage if they have received free school meals for over 80% of their time at school.

Setting

Setting is the organisation of pupils in classes for a particular subject primarily according to their prior attainment in that subject. Other factors (such as behaviour and friendships) may also be taken into account.

Streaming

Streaming is the organisation of pupils in classes across all (or the majority of) subjects primarily according to prior attainment. Other factors (such as behaviour and friendships) may also be taken into account.

Student mobility

Student mobility is the transfer of a student from one secondary school (or other learning environment) to another.

1. Gorard, S., Siddiqui, N. & See, B. H. (2019). ‘The difficulties of judging what difference the pupil premium has made to school intakes and outcomes in England,’ Research papers in education. Accessed May 2021. ↩

2. The school’s score is not a mere average of its pupils; some outliers are not included. For further details, see the Department for Education’s ‘Secondary accountability measures’, February 2020. Accessed May 2021. ↩

3. The most significant statistical challenge of modelling is the interdependence between variables. We therefore maximised predictive power whilst controlling multicollinearity, keeping the variance inflation factor (VIF) greater than 3. The national data was also used to compare trends in Progress 8 scores over time for schools with different characteristics. For more information please see the technical appendices in this report. ↩

4. The fieldwork schools were plotted according to their hidden resources and extreme contextual challenges. Schools in each of the four categories took similar approaches to the pupil premium. For more details see the pupil premium primer. ↩

5. Department for Education, ‘Evaluation of pupil premium’, Research report, July 2013. Accessed May 2021. ↩

6. From survey of 360 teachers, July 2019. ↩

7. Department for Education, ‘Evaluation of pupil premium’, Research report, July 2013. Accessed May 2021. Previous research has also found that school leaders were reluctant to use the pupil premium for whole-school initiatives, such as behaviour approaches and improving the quality of teaching.^{[footnote 8]} ↩

8. Respondents were able to give more than one answer. ↩

9. Survey of 32 school leaders in the West Midlands. ↩

10. Department for Education, ‘Evaluation of pupil premium’, Research report, July 2013. Accessed May 2021. ↩

11. Ager, R. & Pyle, K. (2013). ‘Spending priorities for the pupil premium’, Teacher voice omnibus. The Sutton Trust. Slough: NFER. Accessed May 2021. ↩

12. Survey of 360 teachers, July 2019. Similarly, 20% of fieldwork schools used pupil premium money for teaching assistants. ↩

13. Department for Education, ‘Supporting the attainment of disadvantaged pupils: articulating success and good practice’, Research report, November 2015. Accessed May 2021. ↩

14. Claymore, Z. (2019). ‘Being present: the power of attendance and stability for disadvantaged pupils.’ Slough: NFER. Accessed May 2021. ↩

15. In 2019, grammar schools had an average progress gap of -0.25, compared to -0.49 for other mainstream secondary schools. ↩

16. We made additional telephone calls and visits to five grammar schools in our survey, in addition to three fieldwork grammar schools, to confirm the details of their pupil premium practices. ↩

17. We used the criterion that the index of multiple deprivation for the school’s postcode was in the lowest decile of deprivation. ↩

18. We used anecdotal evidence in some cases, but also examined the ratio of students currently qualifying for free school meals to pupil premium students. ↩

19. We used the criterion that less than 5% of students qualified for free school meals. ↩

20. Shifts from ‘school’ to ‘academy’ were not considered to be a change. ↩

21. Further details of our case studies can be found in our http://educationobservatory.co.uk/pupilpremiumprimer/ ↩

22. This estimate was made in three ways (1) by scaling up our fieldwork sample (2) by considering the number of schools that were previously selective (3) by considering the number of schools with a very small proportion of pupil premium students and a low ratio of free school meal students to pupil premium students. ↩

23. Leckie, G., & Goldstein, H. (2019). The importance of adjusting for pupil background in school value-added models: A study of Progress 8 and school accountability in England. British Educational Research Journal, 45(3), 518-537. Accessed May 2021. ↩

24. In our fieldwork, these schools were in areas with the highest rates of crime (the lowest decile of the index of multiple deprivation for crime) and we therefore used this as a marker of school crime. ↩

25. Schools that are performing well (according to Ofsted) may still suffer from historic poor performance. We did not include schools with current poor performance, attempting to distinguish the school’s performance from contextual factors that affect this performance. ↩

26. We estimated reputations using admissions data (applications per place). ↩

27. A student is defined as persistently absent if they miss more than 10% of school lessons. We took persistent absence rates of more than 2% of the student population to be particularly high (5% of schools fall into this category in 2019). We accepted schools’ claims that these circumstances were beyond their control. ‘A guide to absence statistics’ (publishing.service.gov.uk). Accessed May 2021. ↩ ↩²

28. We took a rate of more than 5% of students with an educational, health and care plan to be high. The average (in 2019) was 1.9%. ↩

29. Claymore, Z. (2019). ‘Being present: the power of attendance and stability for disadvantaged pupils.’ Slough: NFER. Accessed May 2021. ↩

30. Absence is the most predictive factor for which national comparative data is available. It was a stable factor in our statistical models in which multicollinearity had been reduced. ↩

31. Claymore, Z. (2019). ‘Being present: the power of attendance and stability for disadvantaged pupils.’ Slough: NFER. Accessed May 2021. ↩

32. Department for Education, ‘Pupil absence in schools in England: 2018 to 2019’. Accessed May 2021. ↩

33. Department for Education, ‘Pupil absence in schools in England: 2018 to 2019’. Accessed May 2021. ↩

34. It is not possible to give an exact figure because of collinearity between factors in our models. The range is based on our best models that reduce this multicollinearity to acceptable levels in different ways. ↩

35. Statistical models cannot account for other variables that are highly correlated. In this case, the variance inflation factor (VIF) reached 5 in some models. ↩

36. We investigated the length of transition by Year 6 students, visits to primary schools, summer camps and Year 7 mentoring by older students ↩

37. Our estimates are based on the fact that 8% of schools satisfy our criteria for outstanding progress and that around 10% of our fieldwork schools in this category face extreme contextual challenges. A further 40% do not but are also without hidden resources. ↩

38. Pupil premium students at schools with vertical tutor groups have higher Progress 8 scores (by 0.1) than at other schools. Non-pupil premium students score similarly. The statistical analysis is from 285 schools, 33 vertical tutoring schools, with p=0.05. ↩

39. Estimate of 12% using vertical tutoring from a survey of 285 schools. The estimate of 10% vertical tutoring failures comes from fieldwork in 30 schools. ↩

40. Schools using mixed attainment classes in Y7 and Y8 have higher Progress 8 scores for pupil premium students (by 0.4) and for non-pupil premium students (by 0.25) – this is from a survey of 285 schools and p-value 0.04. ↩

41. Francis and others (2017). ‘Exploring the relative lack of impact of research on ability grouping in England a discourse analytic account.’ Accessed May 2021. ↩

42. Department for Education, ‘Evaluation of pupil premium’, Research report, July 2013. Accessed May 2021. ↩