Universal Credit statistics: user engagement survey report 2020
Published 15 December 2020
Applies to England, Scotland and Wales
© Crown copyright 2020
This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: [email protected].
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at https://www.gov.uk/government/publications/universal-credit-statistics-user-engagement-survey-report/universal-credit-statistics-user-engagement-survey-report-2020
1. Survey overview
The Department for Work and Pensions (DWP) routinely publish monthly and quarterly Official Statistics on claims made, starts to, people and households on Universal Credit (UC) for England, Scotland and Wales (Great Britain).
We launched a survey to help identify our external users, better understand who they are, assess whether our statistics meet their needs, and possible areas for improvement.
This follows a compliance check held by the Office for Statistics Regulation in 2019, where the department was encouraged to proactively engage with external users and stakeholders in ensuring that their needs are understood and are fed into further development of Universal Credit (UC) official statistics.
There were 51 responses to the online survey which was administered from 16 July to 27 August 2020.
Respondents were from within local government (35%), charity organisations (22%), and central government (10%). They were also from government agencies, private organisations, research institutes, non-governmental organisations (NGOs), international organisations as well as members of the public. Respondents said they were engaged in mostly analytical (75%), policy (41%) and business strategy (18%) type of work. Most respondents work or are based in England (78%) although there were also responses from users in Scotland, Wales, Northern Ireland as well as outside of the UK.
In addition to obtaining the background of our users, the survey covered the following topics:
- access and data use
- data and user preferences relating to our statistical outputs, such as:
- the data on Stat-Xplore
- interactive tools (UC claimants map and Households on UC map)
- dashboard (households by region)
- quarterly statistical bulletin
- background information and methodology guidance
- quality dimensions
This report summarises the main findings from the user engagement survey.
2. Main findings
In this section, the main findings are reported against the type of users who were identified from the survey:
- expert users
- information gatherers
- policy influencers
- enquiring citizens
This approach enables us to better understand the use, preference, and needs as it relates to each type of user, or their uniqueness.
The improvements to the statistics that users said they would like to see are described in more detail within the annex.
Expert users
Expert users described their use of Universal Credit statistics mostly for research purposes. They described the work they use the statistics for as, analytical – welfare or labour market analysis.
Regarding accessing the statistics and navigating Stat-Xplore, expert users:
- are familiar with the Universal Credit release pages and release calendar
- prefer to access the data using their own account rather than the guest login
- retrieve the data and information that they need directly, and therefore hardly use the pre-defined tables
- re-format downloaded data to create their own datasets for their own analysis
- are familiar with navigating Stat-Xplore
- say Stat-Xplore is not easy to navigate for the occasional visitor and anyone who wishes to look at further data than the pre-defined tables
- are happy with the regional breakdowns provided within Universal Credit statistics
- find it straightforward navigating the Universal Credit statistics web pages to get to the content they are interested in
Expert users reported that they particularly liked:
- the freedom to construct their own tables on Stat-Xplore
- the contents of the background information and methodology guidance
Expert users have asked for:
-
On Stat-Xplore:
- ease of functionality, for example, when adding new variables
- congruence of variables, such as geographies, across all four data series
- the statistics to be more frequent and timely, specifically, the Households on Universal Credit series, than the current three months’ lag and the payment timeliness which has a four months’ lag
- further breakdowns in the geographies and pre-defined tables on Stat-Xplore such as upper tier authority statistics (County Council areas) and flows data onto and off Universal Credit
-
On the background information and methodology:
- more thorough, explicit and detailed methodology with less benefit jargon where possible
- clarity of methodology descriptions, such as, Claims statistics – whether this refers to individuals or to households
- more detail of how conditionality requirements have varied through the pandemic, the date that it was reintroduced and was included in the metadata information
Information gatherers
Information gatherers described their use of Universal Credit statistics mostly for business strategy purposes. They described the work they use the statistics for as journalistic – to monitor unemployment and seek to improve their understanding of the economy to be able to make strategic business decisions.
Regarding accessing the statistics, information gatherers:
- are interested in easily accessible information
- prefer to be able to look at trends in the data
- like to use the pre-defined tables on Stat-Xplore
- will use the interactive tools and the statistical bulletin to help with understanding the context
- find the statistical bulletin to be of an appropriate length and easy to navigate
Information gatherers reported that they particularly liked:
- simple explanations of the headlines and figures within the statistical bulletin
Information gatherers have asked:
- to be notified when there is a new release of the statistics
- for improved guidance on how to use, or create tables on Stat-Xplore such as hints and tips, video demonstrations, and instructions when in large table mode
- for Stat-Xplore to be made more user friendly, such as larger filter buttons with a description of what they do as the mouse hovers over them
- for there to be a choice whether the user would like to include the recent month in the table they create, rather than for it to be there by default
- further breakdowns in the pre-defined tables on Stat-Xplore, such as breakdown by Electoral Ward across all Universal Credit statistics and Conditionality Regime by Disability in the People on Universal Credit statistics
Policy influencers
Policy influencers described their use of Universal Credit statistics mostly as an evidence base for policy needs and welfare analysis. They want information that they can trust and Universal Credit statistics provides the credibility they require.
Regarding accessing the statistics, policy influencers:
- prefer easily accessible information
- are interested in the long-term trend in the data
- prefer the pre-defined tables, but are happy to download data directly from Stat-Xplore
- do not find Stat-Xplore easy to navigate
- look out for summaries of main points and headline figures in the statistical bulletin to help enhance their understanding of the trends
- find the statistical bulletin easy to navigate and the charts within it user friendly
Policy influencers reported that they particularly liked:
- simple explanations of the headlines and figures within the statistical bulletin
Policy influencers have asked:
- to be notified when there is a new release of the statistics
- for video demonstrations on how to use and create tables on Stat-Xplore
- better signposting regarding the additional month’s lag with the employment status data in the People on Universal Credit series
- for more timely quarterly Households on Universal Credit series than the current three months’ lag
- further breakdowns to the statistics, such as diversity characteristics including disability and ethnicity across all Universal Credit statistics
Enquiring citizens
Enquiring citizens are members of the public who described their use of Universal Credit statistics mostly for verifying what they see in the media. They use the statistics to further understand what they see or hear on the news or on social media.
Regarding accessing the statistics, enquiring citizens:
- are more interested in the trends in the data than in the raw data
- prefer easily accessible information
- prefer to be able to drill down further into their region of interest
Enquiring citizens reported that they particularly liked:
- the commentary in the statistical bulletin
Enquiring citizens have asked:
- to limit complex jargon in the commentary within the statistical bulletin
- to be notified via social media of new releases of the statistics
3. Other findings
There were other findings from the survey that were not unique to a specific type of user. These findings relate to the device used to access the data, when the data is accessed, the statistics and outputs that are of most interest, quality dimensions and further engagement.
Devices used to access Universal Credit statistics
Most (67%) survey respondents use a laptop to access Universal Credit statistics.
Proportion of respondents accessing Universal Credit statistics by type of device
| Device | Respondents (%) |
|---|---|
| Laptop | 67 |
| Desktop | 37 |
| Mobile | 10 |
| Tablet | 0 |
When users access Universal Credit statistics
Majority (71%) of survey respondents access UC statistics monthly upon release.
Proportion of respondents accessing Universal Credit statistics monthly, quarterly and at other times
| When accessed | Respondents (%) |
|---|---|
| Monthly upon release | 71 |
| Quarterly upon release | 43 |
| Any other time | 39 |
How frequently Universal Credit statistics are used
Nearly 80% of respondents use the statistics monthly or more frequently.
Proportion of respondents by frequency of use of Universal Credit statistics
| Frequency | Respondents (%) |
|---|---|
| One or more times a week | 16 |
| One or more times in a month | 61 |
| One or more times every 3 months | 22 |
| One or more times annually | 2 |
| Other | 0 |
Which Universal Credit statistics are users most interested in
The Universal Credit statistics users are most interested in is the People on Universal Credit, which was reported by 86% respondents. The Claims made to Universal Credit and Household on Universal Credit are also used by over 60% of users.
Proportion of respondents by Universal Credit statistics series
| Series | Respondents (%) |
|---|---|
| People on Universal Credit | 86 |
| Claims made to Universal Credit | 67 |
| Households on Universal Credit | 61 |
| Starts to Universal Credit | 47 |
Used dissemination platforms of Universal Credit statistics
The data on Stat-Xplore is the most used dissemination platform of Universal Credit statistics, reported by 90% of respondents.
Method of dissemination by proportion of users
| Dissemination platform | Respondents (%) |
|---|---|
| The data on Stat-Xplore | 90 |
| The background information and methodology guidance | 59 |
| The quarterly statistical bulletin | 41 |
| The interactive tools (the interactive maps and the interactive dashboard) | 12 |
Ratings of the Statistical Bulletin
74% of respondents strongly agree or agree that the charts within the statistical bulletin are user friendly, and 79% strongly agree or agree that overall they do not have any accessibility concerns with the general format of the statistical bulletin.
Views on the statistical bulletin by proportion of respondents
| Statement | Disagree / Strongly disagree (%) | Neither agree nor disagree (%) | Strongly agree / Agree (%) |
|---|---|---|---|
| The commentary in the statistical bulletin is easy to understand | 5 | 11 | 84 |
| The charts in the statistical bulletin are user friendly | 21 | 5 | 74 |
| The length of the statistical bulletin is about right | 11 | 21 | 68 |
| The structure of the statistical bulletin makes it easy for me to navigate through the sections | 11 | 21 | 68 |
| The format of the statistical bulletin which gives me the option to switch from the charts to tables is useful | 11 | 32 | 58 |
| The release schedule within the statistical bulletin makes it clear when the next statistical bulletin will be released | 11 | 11 | 79 |
| Overall, I do not have any accessibility concerns with the general format of the statistical bulletin | 5 | 16 | 79 |
Ratings on the timeliness and accuracy of Universal Credit statistics
65% of respondents strongly agree or agree that the monthly statistics are released in a timely manner, 57% of respondents strongly agree or agree that the quarterly statistics are released in a timely manner, and 69% of respondents are satisfied with the accuracy of the statistics.
Views on timeliness and accuracy of the statistics by proportion of respondents
| Statement | Disagree / Strongly disagree (%) | Neither agree nor disagree (%) | Strongly agree / Agree (%) |
|---|---|---|---|
| Monthly statistics are timely | 10 | 25 | 65 |
| Quarterly statistics are timely | 18 | 25 | 57 |
| Satisfied with the accuracy of the statistics | 8 | 24 | 69 |
Preferences for further engagement
55% of respondents prefer to be kept informed by email and 37% of respondents know when the releases are scheduled and prefer to find the updates themselves. Just over a third (35%) of respondents would like to be contacted to discuss their responses further, and similarly, a little over a third (37%) of respondents would like to be part of an external user group.
Further engagement preferences by proportion of respondents
| Statement | Respondents (%) |
|---|---|
| I prefer to receive an email for updates to the statistics releases | 55 |
| I know when the statistics are released and prefer to find it for myself | 37 |
| I would like to be part of the external Universal Credit Official Statistics user group in order to support further development of these statistics | 37 |
| I would like to be contacted to discuss aspects of my survey responses further | 35 |
4. Our next steps
We would like to thank those participants of the survey for their time and comments on how to improve various elements of UC statistics. We will contact respondents who expressed an interest to discuss aspects of their responses, and those who commented that they would like to be part of a user group to help develop these statistics.
Whilst there are certain limitations to what we are able to immediately action given that our publication is hosted on GOV.UK, there are other areas where there are potential quick wins.
Based on user feedback, we have or are implementing the following:
- enhanced the visibility of the research and statistics release calendar, the quarterly statistical bulletin and the background information and methodology document on the Universal Credit home page on Stat-Xplore
- addressed the list of random items on the right hand side (related content) of our GOV.UK collection pages. However, as our collection pages are hosted by the Government Digital Service, and we have no direct control over what features in the related content section, we will request for links to outdated content to be removed
There are some quick wins that we shall shortly implement:
- review the language used in the statistical bulletin and in the background information and methodology guidance to ensure that benefit jargon or technical concepts are reduced without compromising on the information being conveyed. We will contact users who specifically said they can be contacted for further information
- improve the signposting of updates made to the background information and methodology guidance
There are other user comments and feedback which may require further investigation. These include, making variables, such as geographies (Local Authority, Jobcentre Plus Districts, Jobcentre Offices), consistent across all four data series, improve the dashboard functionality, and provide further breakdowns in the pre-defined tables on Stat-Xplore. All comments provided in the survey are listed within the annex.
We will:
- consider these in more detail and, if possible or within our remit, assess the practicalities of implementing them
- forward those that we have no direct influence over due to being outside of our remit, onto colleagues, both within and external to DWP, for their consideration
- produce an action plan with the view to implement a phased approach to possible changes or improvements to the statistics
- keep users informed of further changes that we make through our future releases
Some of the specific feedback we received included:
- “Provide flows data on and off Universal Credit and flows between conditionality groups”. This is an area for future exploration which is on the published Universal Credit statistics release strategy and already on our development plan.
- “Provide the number and age of children in Universal Credit households”. This is an area for future exploration which is on the published Universal Credit statistics release strategy](https://www.gov.uk/government/publications/universal-credit-statistics-background-information/universal-credit-statistics-release-strategy) and already on our development plan.
- “For households on Universal Credit, define tenure type so we can differentiate between social and private sector housing”. We distinguish between social rented sector and private rented sector for those with a housing entitlement on Stat-Xplore.
- “Make diversity information (ethnicity and race) available on the claims made to Universal Credit data series”. The ethnicity measure is currently in development. Further information can be found in the background information and methodology document.
- “The recent ad-hoc releases, that is, the weekly management information during the initial weeks and months of the coronavirus pandemic (“COVID-19”), deemed it possible to provide more timely statistics”. As explained in the accompanying methodology note, management information were published in the interest of transparency and time bound. These are initial figures which have not been rigorously vetted according to the same quality assurance as official statistics. Moreover, in deciding methodology and a publication schedule there is a trade-off between timeliness, quality and resources, as explained further in the Universal Credit statistics: background quality report. Careful consideration has been made over the timeliness of the release, and reducing the lag would risk the figures being misleading and misrepresenting recent trends
5. Annex
Further specific comments and responses from survey respondents on improvements they would like to see made to Universal Credit statistics are listed in this section.
Stat-Xplore (including the data)
- make Stat-Xplore more user friendly and more intuitive, for example:
- currently having to click on the specific breakdown under ‘select all at level’ rather than just clicking ‘select all at level’
- the ‘retrieve data’ button is not easily obvious that it is needed to populate the table
- keeping a selection in the table and having to click ‘untick all’ at the top of the selection is not intuitive
- make variables, such as geographies (Local Authority, Jobcentre Plus Districts, Jobcentre Offices), consistent across all four data series
- introduce more detailed analysis of duration on UC to bring it more in line with the data that is available for Job Seekers Allowance, including duration within / flows between conditionality groups
- further breakdowns in the geographies and pre-defined tables on Stat-Xplore:
- small areas statistics (Electoral Wards and/or Lower Layer Super Output Area (LSOAs) / Middle Layer Super Output Area (MSOAs)
- upper tier authority statistics (County Council areas)
- Local Enterprise Partnership (LEP) areas
- flows data onto and off UC
- number of people who are on UC who are currently under sanction at the count date and at the Local Authority level, including what the sanction is and duration of time on the sanction
- further breakdowns in the pre-defined tables on Stat-Xplore:
- Across all statistics – breakdown by Electoral Ward
- People on Universal Credit series:
- Conditionality Regime by Disability (whether people on ‘no work requirements’ regime are due to sickness or disability; that is, on Work Capability Assessment)
- Households on Universal Credit series:
- average housing payment (so can compare with Housing Benefit average UC housing payment) split by Tenure or household groupings
- tenure type defined so can differentiate between social and private sector housing
- Housing Entitlement by Local Authority
- number and age of children within households
- Claims made to Universal Credit series:
- Jobcentre District or Local Authority
- Diversity information (ethnicity and race)
- Starts made to Universal Credit series – breakdown by Local Authority
- further breakdowns to the statistics:
- Across all statistics – diversity characteristics including disability and ethnicity
- People on Universal Credit series:
- Conditionality Regime by Disability (people on ‘no work requirements’ group due to sickness or disability, that is, on Work Capability Assessment)
- Conditionality Regime by Disability (whether people on ‘no work requirements’ group due to caring for a child with a disability)
- Conditionality regime does not identify between people claiming due to ill health and those claiming due to having young children.
- Households on Universal Credit series:
- further demographic information on the number and age of children in households claiming UC
- number and age of children within households
- Limited Capability for Work entitlement
Interactive Tools
- improve the dashboard functionality – make finding a Local Authority easier to find especially when it is within a small geographical area
- consider figures as a proportion rather than the current raw numbers and display against the country average
Statistical Bulletin
- install a ‘Click here for the latest release’ banner
- list previous releases in the ‘related content’ on the right hand side of the release page in chronological order rather than the random list it currently has
- make the charts more basic such as a slope chart rather than clustered bar charts
- ensure users can access machine-readable versions whereby users can append “/data” to the URL and get a JavaScript Object Notation (JSON) version of the release
Background information and methodology guidance
- make the methodology more detailed, more explicit, with less benefit jargon
- ensure the wording in the Guidance is written in less DWP-oriented language that is more fit for members of the public to use
- make it easy to identify where changes have been made to the Guidance, where exactly in the document changes have been made, date the changes that were made and include links to previous versions
- make methodology descriptions clearer, for example, Claims statistics – whether this refers to individuals or to households
Regional breakdowns
- include more sub-regional breakdowns and other geographies such as Travel to Work Areas (TTWA) and Local Enterprise Partnerships (LEP)
- include Electoral Wards
- include more detailed breakdowns of the London area, for example, by borough, and also for Wales and Scotland
- include detailed breakdowns by deprivation indicators
Quality dimensions
- On timeliness of the release of the statistics:
- the recent ad-hoc releases (that is, the weekly management information during the initial weeks and months of the coronavirus pandemic (“COVID-19”) deemed it possible to provide more timely statistics
- make the statistics more timely, over adding further breakdowns
- release the quarterly statistics with less of a lag
- add a release calendar
- On the accuracy of the published statistics:
- provide more information about the characteristics of claimants as a breakdown, for example, lead carer status, health status, whether awaiting a work capability assessment, probable employment status, flows within UC and off-flows.
- a better explanation as to why the figures are revised after the initial publication
- make the conditionality groups compatible for comparison to actual labour market status, for example, from a labour market perspective a lead carer with a child under the age of 1 is very different from someone a with long-term health condition.
- The absence of the above information affects the interpretation of these statistics and a lengthy explanatory note is not enough to mitigate the user’s wider misinterpretation of the statistics.
- On navigating the web pages:
- a release calendar would be useful
- add a video demonstration
- make it easier to find the statistical tables which provide information on how users can interrogate the data to inform their judgement on how to use the statistics
6. Contacts
Feedback on the content, relevance, accessibility and timeliness of these statistics and any non-media enquiries should be directed to:
Statistician: Ebun Edwin
Email: [email protected]
For media enquiries on these statistics, please contact the DWP press office
For statistics enquiries only. These contact details are unable to provide any information or assistance with claiming Universal Credit.