Quality and methodology information report
Updated 23 February 2023
© Crown copyright 2023
This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: [email protected].
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at https://www.gov.uk/government/statistics/combined-working-age-absolute-low-income-and-material-deprivation-estimates-fye-2011-to-fye-2021/quality-and-methodology-information-report
Future releases
This statistical series has ended as a separate release. The statistics are now included in the annual Households Below Average Income (HBAI) releases.
Introduction
The Working Age Combined Absolute Low Income and Material Deprivation experimental statistics present information on working age adults in combined low income and material deprivation for the first time.
The experimental statistics are based on Households Below Average Income (HBAI) statistics. HBAI provides annual estimates on a range of measures of low income, to capture different aspects of changes in living standards. The statistics are based on the Family Resources Survey (FRS), whose focus is capturing information on incomes, and as such captures more detail on different income sources compared to other household surveys. The FRS captures a lot of contextual information on the household and individual circumstances, such as employment, education level and disability. This is therefore a very comprehensive data source allowing for a lot of different analysis.
This report provides detailed information on key quality and methodological issues relating to the combined absolute low income and material deprivation for working age adults – in particular the experimental ‘working age material deprivation’ element of the measure.
Quality and methodological issues relating to the low income component of the combined absolute low income and material deprivation measure or National Statistics on combined low income and material deprivation for children can be found in the HBAI quality and methodology report.
Fieldwork operations for the financial year ending (FYE) 2021 Family Resources Survey (FRS) were changed in response to coronavirus (COVID-19) and the introduction of national lockdown restrictions. The established face-to-face interviewing approach employed on the FRS was suspended and replaced with telephone interviewing from April 2020 for the whole of the FYE 2021 survey year. This change impacted on both the size and quality of the achieved sample. Further information on this can be found in the technical paper on the impact of the COVID-19 pandemic on these statistics.
What do you think?
We would welcome any feedback you might have on the methodology behind the experimental statistics and suggestions for further development. We would also be interested in knowing how you make use of this data to inform your work. Please contact us via email: [email protected]
Experimental Statistics
Experimental statistics are Official Statistics which are published in order to involve users and stakeholders in their development and as a means to build in quality at an early stage. All Official Statistics should comply with the UK Statistics Authority’s Code of Practice for Statistics which promotes the production and dissemination of official statistics that inform decision making.
About combined absolute low income and material deprivation
A working age person is considered to be in combined absolute low income and material deprivation if they are below an absolute low income threshold and are experiencing material deprivation. Combined absolute low income and material deprivation is a more targeted measure than either low income or material deprivation on its own, as working age people experiencing combined absolute income and material deprivation show two indicators of financial challenge.
Low income
The low income measure used by this publication is taken from the HBAI series. The primary income measure used in HBAI is disposable household income adjusted for household composition (called equivalisation). Income includes contributions from earnings, state support, pensions, and investment income among others, and is net of tax.
An individual is said to be in low income if their equivalised disposable income is below a certain threshold. This publication uses a measure of absolute low income, where the low income threshold is fixed in real terms, at 50%, 60% and 70% of median income in 2010 to 2011 and annually adjusted for inflation. This measure of low income is a before housing cost measure, meaning that housing costs are not deducted from income.
For more information about low income, see the Households Below Average Income quality and methodology paper.
Material deprivation
Material deprivation measures whether a family can afford a combination of goods/services. Questions designed to capture the material deprivation experienced by families with children have been included in the Family Resources Survey since 2004 to 2005, with a change to the items in 2010 to 2011. Respondents are asked whether they have 21 representative goods and services, including twelve child and nine adult items. If respondents do not have a good or service, they are asked whether this is because they do not want them or because they cannot afford them. Being unable to afford a combination of these goods/services and reaching a certain threshold results in the individual being materially deprived. HBAI reports on combined relative low income and material deprivation and combined absolute low income and material deprivation for children.
Working age adults without children are also asked the subset of questions aimed at adults in most circumstances. These nine questions form the basis of the experimental material deprivation measure for all working age adults. A noticeable limitation of this measure is that since their introduction, the working age material deprivation statistics have not been reviewed in the context of a working age measure. Instead they have only been reviewed as contributing questions to a child measure. To address this, the department is conducting a review of the material deprivation items and methodologies.
In FYE 2021, one of the questions asked as part of the measure (‘A holiday away from home for at least one week a year, not staying with relatives’) was affected by legal restrictions introduced in response to the COVID-19 pandemic. This meant that it was not possible for those sampled to access this during periods of lockdown, regardless of financial constraint. Some of those in the sample may have responded to questions on access to such opportunities with their ordinary circumstances in mind. Others may have responded according to their actual circumstances, and it has not proved possible to separate this out in the FRS data. For FYE 2021, all estimates of combined absolute low income and material deprivation are not comparable with previous years. We do not recommend that FYE 2021 estimates are directly compared with previous years, as it cannot be concluded that the change in recorded material deprivation during FYE 2021 represented a real change in household circumstances. More details on this can be found in the technical report on HBAI COVID-19 impacts.
It is also not possible to directly compare levels of combined absolute low income and material deprivation for working age adults to children as they use different items to form their measure.
Material deprivation items
The nine items which form the working age measure are:
- enough money to keep home in a decent state of decoration
- a holiday away from home for at least one week a year, not staying with relatives
- household contents insurance
- regular savings of £10 a month or more for rainy days in retirement
- replace any worn out furniture
- replace or repair electrical goods such as a refrigerator or a washing machine when broken
- a small amount of money to spend each week on yourself, not on your family
- in winter, able to keep accommodation warm enough
- keep up with bills and regular debt payments
Questions for the material deprivation index were first selected from the Poverty and Social Exclusion (PSE) Survey (1999). The items selected were considered necessities by a majority of respondents to the survey [footnote 1]. The full list of questions identified from PSE was shortened for inclusion in the Family Resources survey [footnote 2].
In the FYE 2011 FRS, four new questions were added to the children’s material deprivation index, following a review of the items in the index based on their validity and reliability [footnote 3].
These questions have only been reviewed in the context of a child measure of material deprivation. They have not been reviewed with reference to whether the items work well together to capture material deprivation for working age adults (both those with and without children).
As part of the development of these experimental statistics and the nine items which make up this scale, assessments of validity and reliability were made - set out below. This assessment concluded in deciding to keep all nine items in the experimental statistics. Assessment was made on FYE 2020 data, due to impacts from COVID-19 in the latest year of data.
The department is undergoing a review of the material deprivation questions and methodologies. This work may lead to future changes to the nine items.
To inform this review, and future iterations of this publication, we welcome any feedback you might have on the items that make up this measure. Please contact us via email: [email protected]
Validity
To assess the validity of each item in the measure (ensuring each item appears to measure the concept of material deprivation) logistic regression was used to calculate odds ratios for lacking each of the items by other variables known to be associated with poverty [footnote 4], [footnote 5]:
- absolute low income 60% before housing costs (BHC)
- presence of a disabled person in the family
Higher odds ratios indicate that the risk of lacking an item is higher when an individual is in absolute low income or is living in a family where someone is disabled compared to when this is not the case. Gender and age were controlled for when looking at disability.
For all 9 items, odds ratios showed that a person was more likely to lack the item in FYE 2020 if they were in low income or living in a disabled family.
| Item | Absolute low income (60%), BHC | Someone in the family is disabled |
|---|---|---|
| Keep home in decent state of decoration | 3.3 | 3.1 |
| Holiday for at least one week a year | 3.3 | 3.0 |
| Household contents insurance | 3.5 | 3.3 |
| Regular savings of £10 a month or more | 3.3 | 3.0 |
| Replace any worn out furniture | 3.4 | 3.0 |
| Repair or replace major electrical goods | 3.8 | 3.0 |
| A small amount of money to spend each week on yourself | 3.6 | 3.0 |
| Able to keep accommodation warm enough | 3.9 | 3.7 |
| Keep up with bills and regular debt payments | 3.1 | 3.0 |
The likelihood of being deprived of each item was higher when working age people were in low income or were living in a family with a disabled person. All odds ratios were significant for each independent variable at the 0.05 level suggesting each item is a valid measure of deprivation.
Reliability (internal consistency)
Reliability, as measured by Cronbach’s Alpha, measures how internally consistent the items are and that they are all measuring the same underlying property of material deprivation. The working age material deprivation set of questions had a Cronbach’s Alpha of 0.852, and therefore can be considered to have good reliability. Removing one item in FYE 2020 would slightly increase the reliability of the set – ‘keeping up with bills and regular debt payments’. Given this item shows good validity and removing it only marginally increases the reliability of the overall scale, this item was kept in the experimental measure of working age material deprivation.
Cronbach’s Alpha statistic when each item is removed from the scale
A higher score than the overall index produced (0.852) indicates the measure’s reliability would improve with the question being removed.
| Material deprivation items | Cronbach’s Alpha with item deleted |
|---|---|
| Keep home in decent state of decoration | 0.834 |
| Holiday for at least one week a year | 0.835 |
| Household contents insurance | 0.835 |
| Regular savings of £10 a month or more | 0.833 |
| Replace any worn out furniture | 0.824 |
| Repair or replace major electrical goods | 0.828 |
| A small amount of money to spend each week on yourself | 0.831 |
| Able to keep accommodation warm enough | 0.849 |
| Keep up with bills and regular debt payments | 0.853 |
Suitability
A ‘suitable’ material deprivation measure should reflect items perceived to be necessities in the society where people live. The views of the public on this topic was last assessed in 2011 [footnote 6]. A new review is underway on the items which make up this measure and which seeks the public’s view on which items are necessities.
No statistical assessment of suitability has been made for this experimental statistical release. Assessment of these questions at face value however, does suggest a possible area of concern which will be explored as part of the review mentioned above.
Three of the nine questions are unlikely to be relevant to some working age people living in rented sector accommodation, particularly where this accommodation is furnished:
- enough money to keep home in a decent state of decoration
- replace any worn out furniture
- replace or repair major electrical goods such as a refrigerator or a washing machine, when broken
When these three questions were first introduced in the Family Resources Survey in FYE 2005, just 13% of working age adults lived in the private rented sector, according to Households Below Average Incomes. By FYE 2020, 23% of working age adults were renting privately.
Respondents are able to answer most of the questions on material deprivation items in one of four ways: ‘has/have this’, ‘would like this but cannot afford’, ‘don’t want or need’, or ‘doesn’t apply’. Only ‘would like this but cannot afford’ is counted as materially deprived. Answer categories ‘don’t want / need’ and ‘doesn’t apply’ are treated in the same way as a ‘has / have this’ answer.
If working age adults state ‘do not want/need’ or ‘doesn’t apply’ to multiple deprivation items this reduces the pool of questions to which they are able to state ‘cannot afford’ to reach the 25 / 100 threshold (above which a person is considered materially deprived). This may mean they fall short of a deprivation threshold despite being unable to afford other necessities, e.g. food, social gatherings, transport – which are not asked about in the measure overall. The working age measure may underestimate the proportion of working age adults in privately rented, furnished accommodation in material deprivation.
No breakdowns of working age combined absolute low income and material deprivation by tenure type have been published in this first iteration of the statistics. The question of whether these items adequately capture working age material deprivation for those in rented accommodation should be considered before any future release of these statistics by tenure type.
Question assessment and review
Since FYE 2011 the questions for children and working age adults have not been reviewed or updated. The working age nine questions were reviewed at this time in the context of a child measure and have not been reviewed since their introduction in the context of a working age measure. Limited testing of the questions for this publication suggests that for the purpose of experimental statistics these material deprivation items are largely valid and reliable. For the first publication of these experimental statistics, we therefore have used the entire set of nine items.
About the development of a working age material deprivation measure
Imputation of missing data
Nine material deprivation items are asked of the majority of working age adults. However, the FRS did not gather data for 8% of working age adults in FYE 2021 (this proportion varies year-on-year). Missing data appears to be non-random and predominantly is missing from those living in multiple benefit unit households and working age adults in a couple where the other member of that couple is of pension age.
For this experimental publication, missing values are imputed using a method called hot-decking. Hot-decking looks at characteristics within a record containing a missing value to be imputed, and matches it up to another record with similar characteristics for which the variable is not missing. The specific variables used for the hot-decking procedure are:
- benefit unit income
- economic status
- number of dependent children
- savings held by the benefit unit
- age that the head of benefit unit left education
- ethnic group of the head of the benefit unit
- family type (couple / single)
- disability in the benefit unit
- government region
This method ensures that imputed solutions are realistic and allows a wide range of outcomes which maintain variability in the data. This approach is also used for missing data in the child material deprivation measure.
Measuring deprivation for parents of children
For these experimental statistics, the working age adult measure uses the same set of nine questions for parents and non-parents. This ensures that working age adults are treated as a single group. However, as child material deprivation is measured using an expanded index of 21 questions, parents can be materially deprived while their children are not, and vice versa.
The 2022 review of the material deprivation questions in the FRS will explore the advantages and disadvantages of developing a “core” set of questions for the whole population alongside or instead of measures aimed at specific family types. We welcome views on the advantages and disadvantages of this approach. Please contact us via email at: [email protected]
Prevalence weighting
A prevalence weighted approach has been used. Prevalence weighting is a technique of scoring deprivation in which more weight in the deprivation measure is given to working age adults lacking those items that most working age adults already have. This means a greater importance, when an item is lacked, is assigned to those items that are more commonly owned in the population.
For each question a score of 1 indicates where an item is lacked because it cannot be afforded. If the working age adult has the item, the item is not needed or wanted, or the question does not apply then a score of 0 is given. This score is multiplied by the relevant prevalence weight. The scores on each item are summed and then divided by the total maximum score; this results in a continuous distribution of scores ranging from 0 to 1. The scores are multiplied by 100 to make them easier to interpret. The final scores, therefore, range from 0 to 100, with any working age adults lacking all items which other working age adults had access to scoring 100.
For illustrative purposes, we’ve included the material deprivation weights for working age adults in FYE 2020 (in line with the other analysis included in this document).
Material deprivation weights used for working age adults in FYE 2020
| Item | Weight | Final score |
|---|---|---|
| Money to decorate home | 0.765 | 11.33 |
| Holiday away from home one week a year not with relatives | 0.667 | 9.87 |
| Home contents insurance | 0.643 | 9.51 |
| Make savings of 10 pounds a month or more | 0.700 | 10.36 |
| Replace worn out furniture | 0.617 | 9.14 |
| Replace broken electrical goods | 0.686 | 10.15 |
| Money to spend on self each week | 0.828 | 12.25 |
| Keep house warm | 0.914 | 13.52 |
| Keep up to date with bills | 0.938 | 13.87 |
| Sum | 6.757 | 100.00 |
Setting a threshold
For these experimental statistics the same threshold is used as for children - 25 / 100. As part of the 2022 material deprivation review, we will explore the Department’s use of prevalence weighting, and where a working age threshold, going forward, should be set. We welcome views on this. Please contact us via email: [email protected]
Sampling and non-sampling error
The figures in this publication come from the Family Resources Survey. Like all surveys, it gathers information from a sample rather than from the whole population. The size of the sample and the way in which the sample is selected are both carefully designed to ensure that it is representative of the UK as whole, whilst bearing in mind practical considerations such as time and cost constraints. As the FRS is a sample from the UK population, any statistics derived from it are only estimates of the true numbers for the overall population.
In addition to sampling error, non-sampling error is another area of uncertainty and is present in all surveys as well as in censuses. Non-sampling error encompasses all error other than sampling error. Types of non-sampling error include: coverage error, non-response error, measurement error and processing error. These errors are minimised in this survey by rigorous procedures; however, it is not possible to eliminate it completely.
For more details on these issues, please see the quality and methodology report for the Family Resources Survey. A technical report providing detail of the impact of the coronavirus (COVID-19) pandemic on the FYE 2021 sample and estimates is available and should be considered alongside interpretation of these statistics.
Future development
2022 review of the material deprivation questions in the FRS
The Department for Work and Pensions (DWP), in partnership with independent researchers from the London School of Economics and Political Science (LSE), is conducting a review of these and the HBAI material deprivation measures and the questions in the Family Resources Survey (FRS). This review is considering:
- which material deprivation items for families with children, families with working–age adults and families with pensioners should be included in the FRS?
- what are the advantages and disadvantages of different approaches for determining who is materially deprived?
- what are the advantages and disadvantages of developing a “core” set of questions for the whole population alongside measures aimed at specific family types?
- do the advantages of changing the material deprivation items and methodology/methodologies outweigh the disadvantages, for example a break in the time series?
As part of the review, we would also be keen to hear from current or potential future users of the statistics about what would improve the value of the measure(s) to them. Please contact us via email: [email protected]
Further breakdowns
During the pandemic there was unprecedented change in both household circumstances and incomes which were not comparable to any previous survey period. In the financial year ending 2021 the Family Resources Survey which collects data to inform these statistics achieved a lower sample size than normal due to challenges with rapidly changing the mode of contacting and interviewing participants. Discontinuities and additional biases introduced by the changes to data collection during the pandemic become more evident when the statistics are disaggregated into smaller groups. Disaggregated estimates (e.g. sub-national, family types, employment status etc.) are subject to more sampling variation than usual and carry a greater degree of uncertainty. No additional breakdowns are therefore available for FYE 2021. More details about this decision can be found in a technical report providing detail of the impact of the coronavirus (COVID-19) pandemic on these estimates.
Before introducing breakdowns for the back series (FYE 2011 to FYE 2020), we want to hear from users about which characteristics and breakdowns they are most interested in for future releases. Please contact us via email: [email protected]
Key terminology
| Term | Description |
|---|---|
| Working age | Working age adults are defined as all adults below State Pension age. Since 6 April 2010, the State Pension age for women has been gradually increasing and since December 2018 has been increasing for both men and women. On 6 March 2020, the State Pension age for both men and women increased to over 65 years 8 months. The State Pension age for both men and women continued to increase at the same rate, reaching 66 by October 2020. See details of further planned changes to State Pension age. |
| Income | This is measured as total weekly household income from all sources after tax (including child income), national insurance and other deductions. An adjustment called ‘equivalisation’ is made to income to make it comparable across households of different size and composition. |
| Median | Median household income divides the population, when ranked by equivalised household income, into two equal-sized groups. The median is the value at the very middle of the distribution. |
| Equivalisation | Income measures used in this publication take into account variations in the size and composition of the households in which people live. This process is called equivalisation. Equivalisation reflects the fact that a family of several people needs a higher income than a single individual in order for them to enjoy a comparable standard of living. |
| Absolute low income, before housing costs | Individuals are said to be in absolute low income if they live in a household with an equivalised income below a threshold of median income (for example 50%, 60% or 70% of median income) in a specific year adjusted for inflation. The FYE 2011 median is used in this report in order to measure absolute low income as referenced in the Welfare Reform and Work Act 2016, and in line with the Households Below Average Income publication. Absolute low income statistics fall if low income households are seeing their incomes rise faster than inflation. A ‘before housing costs’ measure means housing costs are not deducted from income. |
| Material deprivation | Working age people are asked whether they have nine goods and services. If they do not have them, they are asked whether this is because they do not want them or because they cannot afford them. These questions are used as a way of measuring living standards for working age people. |
| Combined absolute low income and material deprivation | A working age person is in combined absolute low income and material deprivation if they are both below the relevant absolute low income threshold and in working age material deprivation. |
| 2022 material deprivation review | A DWP-led review of this publication’s and HBAI’s material deprivation measures. |
| Prevalence weighted | Prevalence weighting is a technique of scoring deprivation, in which more weight in the deprivation measure is given to working age people lacking those items that most working age people already have. This means a greater importance, when an item is lacked, is assigned to those items that are more commonly owned in the population. |
| Household | One person living alone or a group of people (not necessarily related) living at the same address who share cooking facilities and share a living room or sitting room or dining area. A household will consist of one or more benefit units. Where a total value for a household is presented, such as total household income, this includes both income from adults and income from children. |
Other relevant statistics
The working age combined absolute low income and material deprivation statistics are released alongside a number of other statistics focused on income and low income statistics across Government.
The statistics highlighted below represent a number of statistical releases which might be considered alongside results from this release in order to give a more complete picture. This is not intended to be an exhaustive list and should be considered alongside ONS guidance on sources of data on earnings and income, the GSS interactive tool on income and earnings statistics, with additional details on important questions also available.
Households Below Average Income
Households Below Average Income (HBAI) is a key source for data and information about household income and is the data source from which these statistics are derived.
Poverty and income inequality in Scotland
In-depth analysis of HBAI data for Scotland can be found on the gov.scot website
Poverty statistics for Wales
In-depth analysis of relative income poverty in Wales can be found on the relative income poverty page of the Welsh Government website, which has links to material deprivation and persistent poverty analysis.
Households Below Average Income Report for Northern Ireland
In-depth analysis of HBAI data for Northern Ireland can be found on the communities-ni.gov website.
EU comparisons
A description of how levels of low income in the UK compare with other EU countries is available on the ec.europe.eu website.
Pensioners’ Incomes series.
Pensioners’ Income (PI) gives more a more detailed analysis of pensioners’ incomes.
Family Resources Survey
Family Resources Survey (FRS) publication gives some further results of FRS data analysis.
Income Dynamics
Income Dynamics (ID) is a publication based on longitudinal data, containing analysis of income movements and the persistence of low income for various population groups.
Personal Incomes statistics
Personal Incomes statistics gives summary information about UK taxpayers, their income and the Income Tax to which they are liable.
Wealth in Great Britain
The Wealth and Assets Survey (WAS) is a large scale longitudinal survey conducted by the Office for National Statistics (ONS). The WAS dataset holds information about the economic status of households and individuals including their physical and financial assets, debts, and pension provision. The latest Wave of WAS can be found on the ons.gov website.
Measuring National Well-being
Drawing on social and economic data (including household income and expenditure) from government and other organisations; painting a picture of UK society and how it changes.
Children in Low Income Families Local Area Statistics
These statistics provide estimates of the number and proportion of children living in relative and absolute before housing costs low income by local area UK. Data are available on the GOV.UK website
English Indices of Deprivation
The English Indices of Deprivation, produced by the Ministry of Housing, Communities and Local Government is a measure of relative levels of deprivation in small areas of England called Lower Layer Super Output Areas.
Welsh Index of Multiple Deprivation
The Welsh Index of Multiple Deprivation (WIMD) is the official measure of deprivation in small areas in Wales. It is a relative measure of concentrations of deprivation at the small area level.
Scottish Index of Multiple Deprivation
The Scottish Index of Multiple Deprivation (SIMD) is the Scottish Government’s official tool for identifying those places in Scotland suffering from deprivation.
Northern Ireland Multiple Deprivation Measure
The Northern Ireland Multiple Deprivation Measure (NIMDM) is the official measure of spatial deprivation in Northern Ireland.
-
Gordon, D., Adelman, L., Ashworth, K., Bradshaw, J., Levitas, R., Middleton, S., Pantazis, C., Patsios, D., Payne, S., Townsend, P. and Williams, J. (2000). Poverty and Social Exclusion in Britain. Joseph Rowntree Foundation. ↩
-
Mckay, S., & Collard, C. (2003). Developing Deprivation Questions for the Family Resources Survey. Department for Work and Pensions. Working Paper No 13, London: DWP. ↩
-
Mckay, S. (2011). Review of the child material deprivation items in the family resources survey. Research Report No 746, Department for Work and Pensions. ↩
-
Guio, A., Gordon D., Najera, H., Pomati, M., (2017). Revising the EU Material Deprivation Variables. Statistical Working Papers – Eurostat. ↩
-
Gordon, D. (2017). PSE-UK 2012 Survey Producing an ‘objective’ poverty line in eight easy steps. Poverty & Social Exclusion. ↩
-
McKay, Stephen (2011). Review of the child material deprivation items in the family resources survey. Research Report No 746, Department for Work and Pensions. ↩