Background Information for: Perceptions of Vocational and Technical Qualifications in England – wave 4

Question 1

Introduction

Accepted Answer

Background

This document accompanies the findings of wave 4 of the Perceptions of Vocational and Technical Qualifications Survey and should be read alongside the report. This project was commissioned by Ofqual and managed by YouGov. The design template for charts used in the report was provided by Ofqual.

This project was undertaken in Spring 2021, with fieldwork being undertaken between 22nd January and 10th March.

Context

Ofqual is the independent regulator of general and vocational qualifications in England, and is responsible for regulating around 160 awarding organisations who between them provide over 15,000 live qualifications for learners^{[footnote 1]}. Ofqual’s responsibilities include making sure that:

regulated qualifications reliably indicate the knowledge, skills and understanding students have demonstrated
assessments and exams show what a student has achieved
people have confidence in the qualifications they regulate, and
students and teachers have information on the full range of these qualifications

Ofqual regulates general qualifications such as GCSEs, AS levels and A levels, but also a wide range of other qualifications and assessments, including Functional Skills Qualifications (FSQs) and currently around 300 apprenticeship end-point assessments (EPAs), where it acts as an External Quality Assurance provider ^{[footnote 2]}, alongside a wide range of other Vocational and Technical Qualifications (VTQs).

Ofqual collects and utilises insight on perceptions of qualifications and the qualifications system from the range of stakeholders – including learners, teachers, and employers - that take, work with and use qualifications. This research focused on vocational and technical qualifications and assessments.

Ofqual has continued to annually assess employers’, learners’ and providers’ perceptions of FSQs, EPAs and other VTQs since wave 1 (2018). Ofqual has an interest in stakeholder views both from the perspective of a regulator of those qualifications, some of which are currently subject to reform, and as an External Quality Assurance provider. The aim of this project was to understand how stakeholders (employers, learners and providers) view and use these qualifications and assessments.

The first part of the survey consisted of a section focused on perceptions of Vocational Qualifications generally. The second part of the survey asked about other VTQs, FSQs and T Levels, followed by EPAs. The survey was routed dependent on survey respondent’s responses. For example, learners were only asked questions about perceptions of the qualification type which they were taking. In this wave, the survey was expanded to collect data on understanding of T levels.

The outbreak of the coronavirus (COVID-19) pandemic and the resulting lockdowns since March 2020 has had a multitude of impacts on the education sector in England. For significant parts of 2020, schools and colleges in England were shut down and there was a shift to remote and online learning for almost all students. Given the unusual and unprecedented circumstances of the year, the summer 2020 exam series in England was cancelled. Assessment and awarding of Vocational and Technical Qualifications in 2020 was set out in in the extraordinary regulatory framework (ERF) developed by Ofqual in line with government policy.

In May 2020, Ofqual published the ‘Extraordinary regulatory framework: Vocational and Technical Qualifications, COVID-19 Conditions and Requirements’ (“the ERF”). The ERF put in place a framework to allow results to be issued in summer 2020 to learners taking vocational and technical qualifications in circumstances where normal assessments were disrupted by the coronavirus (COVID-19) pandemic. This enabled awarding organisations to award their qualifications via a calculated result or adapted assessment. In some cases, the only option was to delay an assessment until a later date because there was no way to safely carry out an adapted assessment, and for some qualifications the awarding organisations decided that assessments could continue as normal.

In October 2020, Ofqual published the ‘Extended extraordinary regulatory framework: Vocational and Technical Qualifications, COVID-19 Conditions and Requirements’ (“the EERF”). The EERF followed on from the ERF by allowing awarding organisations to adapt their qualifications in 2020 and 2021 in recognition of the fact that learners may have experienced disruption to teaching, learning and assessment beyond summer 2020 because of the pandemic.

Given the uniqueness of 2020, the survey included a 2020-specific section to gauge perceptions of vocational and technical qualifications based solely on that year. Where the original perceptions questions were asked upfront in the survey, respondents were asked to think about each of these qualifications generally and were informed that a 2020 section would follow later in the survey. These modifications were made to strengthen our ability to track key measures over time despite this exceptional year.

Project objectives

The objectives of the project were to investigate training providers’, learners’ and employers’:

Overall perceptions of Functional Skills qualifications, other vocational and technical qualifications (VTQs) and end-point assessments of Apprenticeships; and
Perceptions of a number of topical aspects of the qualifications system for vocational and technical qualifications.

Geographical coverage

Ofqual regulates general and vocational qualifications in England. The focus of this project was to gain the perceptions of training providers, learners and employers in England. Samples of these groups were drawn from England.

Vocational and other qualifications quarterly

User feedback

Ofqual welcomes your comments or suggestions on this version of the Perceptions Survey and how to improve future versions. Please write to Ofqual at statistics@ofqual.gov.uk.

Question 2

Method and guidance

Accepted Answer

Overview of method

Wave 4 of the Perceptions of Vocational and Technical Qualifications Survey was carried out by YouGov, on behalf of Ofqual. The survey was conducted online using YouGov’s proprietary survey system. The fieldwork was conducted between 22 January and 10 March 2021.

The final achieved sample for each of the target groups was as follows:

502 education professionals (referred to throughout this report as training providers) who work for an establishment that offers vocational or technical qualifications
533 people (referred to throughout this report as learners) who are currently taking or have taken in the last three years a Functional Skills qualification, vocational or technical qualification, or non-degree Apprenticeship
2,006 senior managers or directors and above (referred to throughout this report as employers) who have HR decision-making involvement or influence in their organisation

The final achieved samples were structured to be representative of the relevant target populations. We therefore conclude that, based upon the sampling procedures, the samples delivered for training providers, learners and employers were of high quality. While the sampling approach has aimed to identify groups of participants that are representative, given that a survey of this kind captures the views only of those involved. Therefore throughout this report, perceptions attributed to sample groups are inferred from those who took part in the survey.

Survey design

The wave 4 survey was designed in collaboration between Ofqual and the YouGov project team. The survey covered themes explored in previous waves of the survey in addition to new content. A copy of all of the questions included in this wave of the survey is available in Appendix B.

Guidance on analysis

Throughout the report, percentages are rounded to the nearest whole number. Due to rounding, percentages may not always add up to 100%. The report summarises the data for each question using charts. The intention of this report is to summarise the main themes in the data and not produce a full replication of the entire data set collected.

Reported differences are based on statistical significance testing as tested at the 95% confidence level. Whenever a difference between two groups is reported, this refers to a statistically significant difference. For the charts reporting proportions of ‘Agree’ and ‘Disagree’ responses, ‘Don’t know’ responses are not included in the figure. As such, proportions may not add up to 100%.

Question 3

Data limitations

Accepted Answer

Sample effects

It should be recognised that the sampling procedures adopted in waves 3 and 4 differ from the approach taken in the previous waves. For waves 3 and 4, the sample for learners and employers was drawn from the YouGov panel of over 1,000,000 UK adults who have signed up to take part in our surveys. Similarly, for training providers, the sample was drawn from a combination of the YouGov panel and another source. In wave 2, by comparison, each sample was selected from targeted contact databases.

Mode effects

The various modes of data collection have their advantages and disadvantages and differences will be seen in the results of the same or similar questions asked using various modes. These advantages and disadvantages of data collection methods do not mean that one mode is more or less accurate than the other, more that when looking to compare data over time using different modes it needs to be considered that the data may vary.

A key advantage of online research is the neutrality of the interview mode. Independent research has found that respondents modify their answers in the presence of an interviewer, including via telephone. A key finding from a past study conducted by YouGov for Ofqual ^{[footnote 3]} was that respondents had an increased propensity to provide a more positive response when asked a question by an interviewer over the telephone compared to when individuals answered online in their own time and in private. Online responses can be considered more effective for capturing sensitive data in a way that gives the respondent a greater sense of anonymity.

Time series data

Given the changes made to the sampling approach and mode of data collection for this research since wave 3, when looking to compare results over time it needs to be considered that the data may vary. YouGov and Ofqual to advise taking caution when interpreting changes over time between waves 3 and 4 and waves 1 and 2 as time series analysis could be unreliable due to the changes in methods.

Question 4

Glossary

Accepted Answer

Base size

The number of respondents answering the question.

Effective base

When sampling is undertaken, it creates a ‘design effect’ that can impact upon the reliability of the information collected. The effective base size is the base size that is left when removing this effect. It is used for significance testing.

Sample frame

The way that the sample population is structured before recruiting a sample for the survey is drawn.

Statistically significant

If a result is termed statistically significant, it is unlikely to have occurred at random. The process of determining whether a result is statistically significant is known as significance testing.

Weighting

The process of making some respondents in a survey more or less important than others to accurately reflect their position in the population being surveyed.

Apprenticeship framework

These were developed by sector bodies, primarily focused on qualifications. They are being phased out by 2020-21 and being replaced by apprenticeship standards. Framework apprenticeships are assessed throughout by completing a unit at a time but there is no endpoint assessment.

Apprenticeship standard

New apprenticeship standards, developed by employers, to show what an apprentice will be doing, and the skills required of them, by job role. Standards are developed by employer groups known as ‘trailblazers’. Standards are occupation-focused rather than qualification-led, with the apprentice being assessed through an end-point assessment.

Awarding organisation

An organisation recognised by the qualifications regulators in England, Wales or Northern Ireland to develop, deliver and award qualifications.

End-point assessment of apprenticeship (EPAs)

The final assessment for an apprenticeship, taken at the end of the apprentice’s training and used to determine whether they are occupationally competent and will pass their apprenticeship.

Functional Skills qualifications (FSQs)

Functional Skills qualifications are available in English, maths and ICT and are available in levels from Entry 1 to Level 2. Functional Skills assessments test the fundamental, applied skills in these subjects for life, learning and work. Reformed functional skills qualifications were introduced for first teaching in September 2019, and this research does not cover perceptions of these qualifications.

Other Vocational & Technical Qualifications (VTQs)

These normally have a vocational focus and include, amongst others, Technical Qualifications and National Vocational Qualifications (NVQs). For this report, this term refers to qualifications other than FSQs and EPAs.

Further Education (FE)

Further education (FE) includes any study after secondary education that’s not part of higher education (that is, not taken as part of an undergraduate or graduate degree).

Local Authorities (LAs)

Local Authorities (LAs) are responsible for a range of vital services for people and businesses in defined areas, including social care, schools and adult education.

Providers

The scope of the survey sample included providers in the following categories: Further Education (FE); Higher Education (HE); private class-based; private work-based; public sector community based; public sector ‘other’; schools; voluntary sector education.

T Levels

T Levels are new courses which follow GCSEs and are equivalent to 3 A levels. These 2-year courses, which launched September 2020, have been developed in collaboration with employers and businesses. T Levels offer students a mixture of classroom learning and ‘on-the-job’ experience during an industry placement of at least 315 hours (approximately 45 days).

Technical qualifications

Technical Qualifications are a compulsory element of the new T Level courses. Each technical qualification includes the core theory, concepts and skills for an industry area and specialist skills and knowledge for an occupation or career. Ofqual is responsible for regulating Technical Qualifications within T Levels and the first teaching for the first three T Levels started Sept 2020.

Question 5

Appendix A – Technical report

Accepted Answer

Appendix A outlines the composition of the achieved sampling and additional information on the methods used in wave 4.

Training provider sample

Sample coverage

The sample was designed to be a representative sample of the establishments that offer vocational or technical qualifications across England.

The scope of the survey sample included providers from the following types of establishments: Sixth form that is part of a school; Sixth form college; Further Education College; Independent Training Provider (ITP); Adult and Community Learning Provider (ACL); Local Authority or Council; Employer Provider; Higher Education Institution; Voluntary and community sector organisation.

Sample frame

The sample of training providers was drawn from two sources.

Skills Funding Agency registered provider database

Mirroring the approach taken for wave 3, the Skills Funding Agency registered provider database was used as the main sample frame, providing access to c.4,700 providers from which to sample from in wave 4.

This was used as a database to select contacts in establishments to be surveyed. The database contained named establishments with generic email addresses that allowed the survey to be targeted at the establishments that met our criteria.

YouGov Panel

The survey was also conducted using education professionals registered to the YouGov panel, who have consented to participate in surveys with YouGov. A quota sampling approach was used on to the sample drawn from the YouGov panel to ensure broad representativeness by establishment type and region.

Sample selection

The sample was selected to ensure the representativeness of the results in line with the VTQ provider population in England. The sample frame was stratified by the following variables:

Establishment type
English region

Using the Skills Funding Agency registered provider database, all establishments that met the sample criteria were invited to take part in this research. This gave establishments an equal opportunity to take part in the research and for YouGov to deliver a random probability sample.

For the respondents drawn from the YouGov panel, soft quotas were applied to control for establishment type and region. From the available sample frame identified respondents were invited at random to take part in the survey.

Achieved sample size and weighting

The unweighted achieved sample size and breakdown of the samples by key characteristics is shown in Tables 1a and 1b. Alongside this the weighted base and weight factors applied are also shown.

The unweighted base shows the number of completed surveys and the weighted base shows the adjustments that have been made to correct for any sample bias. The largest weight factor demonstrates where the weighting has had a greater impact due to the unweighted based being lower than what would be expected in a representative sample.

Table 1a: Achieved sample size by establishment type

Establishment type	Unweighted base	Weighted base	Smallest weight factor	Largest weight factor
Sixth form	62	70	0.91	1.53
FE College	194	65	0.26	0.44
Independent training provider	153	276	1.43	2.41
ACL	16	15	0.70	1.19
LA or council	39	5	0.10	0.16
Employer provider	16	20	0.95	1.61
HEI	3	15	4.59	5.75
Voluntary sector	10	10	0.83	1.39
Other	9	25	2.76	2.85

Table 1b: Achieved sample size by region

Region	Unweighted base	Weighted base	Smallest weight factor	Largest weight factor
East of England	42	40	0.35	1.92
West and East Midlands	108	90	0.10	4.73
London	62	70	0.15	2.28
North West and North East	99	120	0.16	2.41
South	131	126	0.10	4.59
Yorkshire and Humber	60	55	0.12	5.75

Learner sample

Sample coverage

The sample was designed to be a representative sample of learners who are currently studying or have studied one of the following qualifications in the last three years:

Functional skills in English, maths, or ICT
Vocational or Technical qualifications (e.g. NVQs, BTECs)
Apprenticeship (non-degree)

Sample frame

The sample of learners was drawn from the YouGov panel. Over the last twenty years, YouGov has carefully recruited a panel of over 1,000,000 UK adults to take part in our surveys.

Sample selection

The sample was selected to ensure the representativeness of the results in line with the actual population of people who have recently studied these qualifications in England.

YouGov has a proprietary, automated sampling system that invites respondents based on their profile information and how that aligns with targets for surveys that are currently active. Respondents are automatically, randomly selected based on survey availability and how that matches their profile information.

Achieved sample size and weighting

The unweighted achieved sample size and breakdown of the samples by key characteristics is shown in Table 2. Alongside this the weighted base and weight factors applied are also shown.

The unweighted base shows the number of completed surveys and the weighted base the adjustments that have been made to correct for any sample bias. The largest weight factor demonstrates where the weighting has had a greater impact due to the unweighted base being lower than what would be expected in a representative sample.

Table 2: Summary of achieved learner sample and weighting scheme

Age	Unweighted base	Weighted base	Smallest weight factor	Largest weight factor
Under 19	108	130	1.21	1.21
19-24	103	113	0.89	1.38
25+	322	290	0.88	1.16

Employer sample

Sample coverage

The employer sample was designed to be representative of the English business population by organisation size and sector. The sample definition ensured that only those in senior management and above roles with the ability to answer on behalf of their organisation were eligible to take the survey.

Sample frame

The employer sample was drawn from the YouGov business panel which contains over 9,000 senior decision makers and 8,000 employees with decision making responsibility for HR / personnel within their organisation.

Achieved sample size and weighting

The achieved sample size and breakdown of the employer sample by key characteristics and associated weighting factors are shown in Tables 3a and 3b below.

The unweighted base shows the number of completed surveys and the weighted base shows the adjustments that have been made to correct for any sample bias. The largest weight factor demonstrates where the weighting has had a greater impact due to the unweighted base being lower than what would be expected in a representative sample.

To ensure a robust sample of employers from large organisations, YouGov collected a disproportionately higher number of responses from medium and large organisations than would exist in a representative sample of the business population. Subsequently, weighting was been applied to correct for this imbalance and produce a sample reflective of the actual business population by organisation size.

Table 3a: Summary of achieved employer sample and weighting scheme

Size	Unweighted base	Weighted base	Smallest weight factor	Largest weight factor
Micro (2-9 employees)	1018	1585	0.62	1.77
Small (10-49 employees)	409	341	0.33	0.93
Medium (50-249 employees)	263	60	0.10	0.28
Large (250+ employees)	316	20	0.03	0.08

Table 3b: Summary of achieved employer sample and weighting scheme

Sector	Unweighted base	Weighted base	Smallest weight factor	Largest weight factor
Private sector	1653	1906	0.08	2.04
Public sector	190	20	0.03	0.71
Third or voluntary sector	163	80	0.04	1.07

Question 6

Appendix B - Survey

Accepted Answer

Learners

Perceptions of qualifications

Question 8. In 2020, employers, training providers or colleges paid awarding organisations in the region of £15-£20 per student for each Functional Skills qualification (Source: Ofqual 2020). To what extent do you agree or disagree with the following statement? “Functional Skills qualifications offer value for money”

Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
Don’t know

Base: all

Question 9. To what extent do you agree or disagree with the following statements about Functional Skills qualifications?

Statements

9a - I value Functional Skills qualifications
9b - People holding Functional Skills qualifications have the appropriate level of skill in English needed by employers
9c - People holding Functional Skills qualifications have the appropriate level of skill in maths needed by employers
9d - People holding Functional Skills qualifications have the appropriate level of skill in ICT needed by employers
9e - The availability of Functional Skills qualifications is sufficiently flexible
9f - Functional Skills qualifications are good preparation for further study

Answers

Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
Don’t know

Base: all

Question 10. To what extent do you agree or disagree with the following statements about vocational and technical qualifications?

Statements

10a - I value vocational and technical qualifications
10b - The purpose of vocational and technical qualifications is well-understood by employers
10c - I understand the purpose of vocational and technical qualifications
10d - Vocational and technical qualifications prepare learners well for the workplace
10e - The availability of vocational and technical qualifications is sufficiently flexible
10f - Vocational and technical qualifications are good preparation for further study

Answers

Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
Don’t know

Base: all

Question 11. To what extent do you agree or disagree with the following statements about end-point assessments?

Statements

11a - I value the end-point assessments of apprenticeship standards
11b - I understand the purpose of end-point assessments of apprenticeship standards
11c - End-point assessments of apprenticeship standards test the competence and vocational and technical skills needed by employers
11d - The availability of end-point assessments of apprenticeship standards is sufficiently flexible
11e - Apprenticeships provide the right skills and knowledge needed for my chosen career.

Answers

Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
Don’t know

Base: all

Question 120. Which of the following statements describes your understanding of T Levels?

Very good understanding
Quite good understanding
Limited understanding
Not very good understanding
No understanding at all

Base: all

Question 12. To what extent do you agree or disagree with the following statements about T Levels?

Statements

12a - I value T Levels
12b - I understand the purpose of T Levels
12c - T Levels test the competence and vocational and technical skills needed by employers
12d - The availability of T Levels is sufficiently flexible
12e - T Levels are good preparation for work

Answers

Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
Don’t know

Base: all

Question 130. To what extent do you agree or disagree with the following statement? “It is clear to me which qualifications are relevant to my needs”

Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
Don’t know

Base: all

Qualifications studied

Question 14. Thinking about the Functional Skills qualification you are studying/have completed, is this part of an apprenticeship?

Yes
No
Don’t know

Base: all

Question 15. Which Functional Skill qualification/s are you taking, or have you taken? (Please select all that apply)

Functional Skills Qualifications

15a - Functional Skills English
15b - Functional Skills maths
15c - Functional Skills ICT

Level

Entry
Level 1
Level 2
Don’t know

Base: all who are studying FSQs

Question 16. Why did you decide to take a Functional Skills qualification? (Please select all that apply)

I wanted to improve my English or maths or ICT skills
I did not pass a GCSE in English or maths or ICT
I need to complete this as part of an apprenticeship
My employer advised or required me to
In order to get a job
To progress in my current job
To progress in my studies
Other (specify)

Base: all who are studying FSQs

Question 17. Now that you have started learning/ have completed learning, what benefits do you expect from taking a Functional Skills qualification/s? (Please select all that apply)

My English/maths/ICT skills will improve
I will be able to apply English/maths/ICT skills in everyday life
I will be more confident using English/maths/ICT
I will be able to complete my apprenticeship
I will be able to apply for more jobs
To progress in my current job
To be able to find a better job
I will have better job prospects in the longer term
To progress to higher level qualifications
Other (specify)
Don’t know

Base: all who are studying FSQs

Question 18. Thinking about the vocational and technical qualification(s) you are studying/have completed, is this part of an apprenticeship?

Yes
No
Don’t know

Base: all who are studying VTQ

Question 19. Which level is the vocational or technical qualification that you are taking?

Entry
Level 1
Level 2
Level 1/ Level 2
Level 3
Level 4
Level 5
Level 6
Level 7
Level 8
Don’t know

Base: all who are studying a VTQ

Question 190. What is the name/ subject of the vocational or technical qualification(s) you are studying or have studied in the last 3 years?

(free text entry) Base: all who are studying a VTQ

Question 21. Now that you have started learning/have completed learning, what benefits do you expect from taking a vocational and technical qualification? (Please select all that apply)

My vocational/technical skills will improve
I will be more confident using my vocational/technical skills
I will be able to complete my apprenticeship
I will be able to progress in my current job
I will be able to find a job/a better job
I will be able to progress in my studies
I will be more confident in my work/better at my job
Other (specify)

Base: all who are studying a VTQ