Research and analysis

Data quality and methodology

Published 19 December 2024

Applies to England

Introduction

This report describes the quality assurance arrangements for the registered provider (RP) fire safety remediation in social housing in England statistics, providing more detail on the regulatory and operational context for data collection and the safeguards that aim to maximise data quality.

Background

The statistics we publish are based on data collected directly from local authority registered provider (LARPs) and from private registered providers (PRPs) through the Fire Safety Remediation Survey (FRS). We use the data collected through these returns extensively as a source of administrative data. The United Kingdom Statistics Authority (UKSA) encourages public bodies to use administrative data for statistical purposes, and, as such, we publish these data.

Official Statistics in Development status

In February 2018, the UKSA published the Code of Practice for Statistics. This sets standards for organisations producing and publishing statistics, ensuring quality, trustworthiness and value.

These statistics are drawn from our FRS data collection and are being published for the first time in December 2024 as official statistics in development.

Official statistics in development are official statistics that are undergoing development. Over the next year we will review these statistics and consider areas for improvement to guidance, validations, data processing and analysis. We will also seek user feedback with a view to improving these statistics to meet user needs and to explore issues of data quality and consistency.

Change of designation name

Until September 2023, ‘official statistics in development’ were called ‘experimental statistics’. Further information can be found on the Office for Statistics Regulation website.

User feedback

We are keen to increase the understanding of the data, including the accuracy and reliability, and the value to users. Please email feedback, including suggestions for improvements or queries as to the source data or processing to [email protected].

Publication schedule

We intend to publish these statistics each quarter, with the date pre-announced in the release calendar.

All data and additional information (including a list of individuals (if any) with 24hour pre-release access) are published on our statistics pages.

Quality assurance of administrative data

The data used in the production of these statistics are classed as administrative data. In 2015, the UKSA published a regulatory standard for the quality assurance of administrative data. As part of our compliance to the Code of Practice, and in the context of other statistics published by the UK Government and its agencies, we have determined that the statistics drawn from the SDR are likely to be categorised as low quality risk – medium public interest (with a requirement for basic/enhanced assurance).

The publication of these statistics can be considered as medium public interest, as there has been mainstream media interest, but they have only moderate economic and/or political sensitivity. Concerns over data quality are considered low given the data checks by providers and our data quality checks conducted on the submitted data and analytical processes.

Notwithstanding this, we aim for the highest standards of data quality possible within the constraints of available resources and the existing regulatory and operational context. Through ongoing internal analysis, we seek to understand the strengths and limitations of the data, the overall quality of the data and to identify potential means by which it may be improved.

Regulatory context

The regulatory framework for social housing in England underpins the collection of FRS data. For more information about the Regulator of Social Housing (RSH) and the regulatory framework please see our website.

Regulatory framework

We collect the FRS data in order to support our operational approach to regulating the consumer standards set out in the regulatory framework for social housing in England.

For more information about the individual collections, please see the individual LARP and PRP social housing stock and rents statistical releases.

The regulatory framework and data quality

In Regulating the Standards, we set out the operational approach to regulation and emphasise the importance of providers supplying timely and accurate data.

Governance of data and statistics at RSH

The team responsible for the publication of these statistics is also involved in the FRS data collection and the cleansing of incoming data, and works to support the review of data by regulatory engagement colleagues.

All incoming data are stored and analysed within password-protected government secure networks and access to the sector level analysis work undertaken on the data is restricted until after publication (RP level data is accessed by our staff as part of operational work). Further information on the data quality assurance processes we employ is provided on page 6.

FRS collection

The FRS is a self-reported census of stock for which RPs are responsible which measure more than 11 metres in height or that have five or more storeys, referred to in this publication as ‘relevant buildings’. All large RPs (those owning 1,000 or more units of social housing), small RPs (those owning fewer than 1,000 units of social housing) who reported previously that they were responsible for relevant buildings and any RP who had not responded to any previous surveys are expected to compete the FRS return. These returns provide us with the data required for the risk-based, data-driven approach to regulation.

In addition to the survey return for RSH, in which RPs report on their stock at an aggregate level, RPs were also asked to submit fire safety remediation data at an individual building level for assessment by MHCLG.

The building level data are not presented here, but were used during the data cleaning process to ensure consistency. Building level data are available on the MHCLG website.

History

We began collecting the FRS in October 2023, and since September 2024, the collection takes place on a quarterly basis. The FRS provides details of fire safety assessments, life critical fire safety risks and the progress of remediation programmes, for buildings of 11 metres or more in height, which contain dwellings, and for which registered providers are the responsible entity.

Systems

The return is collected via a web-based system called NROSH+. We control the requirements for data input processes, storage, verification, sign-off and extraction of submitted data and produce the statistical releases. Data are either imported or entered directly into the NROSH+ system by RPs.

Communication with data suppliers

We work closely with RPs, through email messages and phone discussions, to ensure there is a common understanding of the data collection requirements throughout the data collection process. Guidance materials are also promoted to users and published on NROSH+.

Quality assurance processes

We do not have oversight of the systems and data quality assurance processes employed by the RPs before submitting data to FRS. However, we do provide clear guidance and documentation on the NROSH+ system and subject incoming returns to a series of checks to identify potential quality issues before each data return is signed off.

The final data files that support the statistical releases are only created once all outstanding queries which could materially impact the quality of the published data are resolved. Due to the survey timings and the tight turnaround for the published data, there are a small number of submissions included each quarter that have not been signed off in NROSH+. In these instances, any outstanding issues preventing sign off relate to the MHCLG Building Level Spreadsheet, and do not relate to the FRS data. Any returns not meeting our quality standards are excluded from the final datasets.

Details on RPs who have not submitted data or whose data did not meet our quality standards will be published alongside the underlying data.

Submission checks

FRS data submitted to us are subject to both automated validation checks and manual inspection.

Automated validations are programmed into the NROSH+ system and check the data at the point of submission for correct formatting, consistency and logical possibility (within expected limits). For example, ensuring numbers of buildings are consistent across different parts of the return and that chains of follow-up questions are completed. Automated validations are either ‘hard’ or ‘soft’.

Hard validations – PRPs cannot submit without the issue being resolved (e.g. when a cost value is provided but the number of buildings it applies to is missing).

Soft validations – PRPs can submit but are required to check their information and, if correct, to submit a supporting document (e.g. when a value appears to be outside of a normally expected range, such as higher than expected costs).

Manual inspections are systematically undertaken on all data submitted. Returns are checked for basic consistency and likely errors. This includes comparison to previously submitted data, with unexpected movements in building numbers or changes outside those anticipated in remediation progress or costs being queried with RPs.

In addition to the survey return for RSH, in which RPs report on their stock at an aggregate level, RPs were also asked to submit fire safety remediation data at an individual building level for assessment by MHCLG. The building level data are not presented alongside the FRS, but were used during the manual data cleaning process to ensure consistency. Building level data are available on the MHCLG website.

Where we identify a potential anomaly with the submitted data, a query is raised with the submitting RP.

Submission checks and sign-off

It should be noted that the process of signing off data is distinct from our ongoing regulatory work. The sign-off of data confirms that we have investigated areas of potential data error, as highlighted in the validation and checking work, and accepted that the provider has submitted data they believe to be accurate. However, during the course of our regulatory activity, the data will be reviewed alongside other evidence, and we may subsequently challenge building numbers or costs or any other aspect covered by the data with providers. As such, the sign-off and publishing of these data does not constitute our agreement that the provider has appropriately complied with our standards.

Post publication checks

FRS data (excluding data which is commercially sensitive) are published at a disaggregated level for all RPs reporting that they are responsible for more than five buildings of between 11 metres and 18 metres and for RPs responsible for more than five buildings with a height of 18 metres or more as part of the statistical release. Releasing data into the public domain serves as an additional route through which erroneous data may be identified by the RP or third parties.

Misreporting

There are no known numerical measures of misreporting of data by RPs. However, one source of possible incomparability is inconsistent interpretation of guidance with providers not applying this consistently across the sector. This issue is most likely to arise where there are technical or legal definitions that are complex or, to some degree, ambiguous. Please see technical notes and definitions for more information on other factors which impact on the data collected.

Corrections

Where errors in the submitted data are discovered within a survey period, either through regulatory activity or through provider contact, we allow providers to resubmit data through the NROSH+ portal.

Statistical release methodology

The data presented in the FRS statistics are drawn from the FRS data.

Accounting for missing data

Most questions in the FRS return are mandatory, and all RPs required to make a FRS return submission are expected to submit the return. We hold the data for 99% of all RPs registered with us. All large PRPs and LARPs responded to the Quarter 2 2024/2025 survey. In total, 15 small providers have not responded to any survey.

2024/25 Quarter 2 responses

There is a small level of FRS non-response due to non-submission or exclusion due to unresolved errors. This quarter, the overall non-response rate was 2.7%. Using our Statistical Data Release and other administrative data held by us, we have identified that all the RPs excluded from the FRS data are private registered providers which own fewer than 1,000 social housing units/bedspaces. The response rate for these ‘small’ providers in 2024/25 Quarter 2 was 92.5% (with the response rate for private registered providers owning 1,000 or more units and local authority registered providers being 100%).

Imputation

We have included some data from previous surveys in this publication to ensure a more comprehensive dataset. The data included from previous surveys are from small providers who had not responded to the Quarter 2 survey at the time of writing but had responded to a previous survey. It should be noted that small RPs are much less likely to be a responsible entity for any relevant buildings.

2024/25 Quarter 2 imputation

We expect RPs to follow the guidance provided with the survey to ensure responses are accurate. RPs must also ensure their response to the Fire Remediation Survey fully matches the MHCLG Building Level Survey response.

Nine buildings have been imputed from previous surveys for two small providers who had not responded to the Quarter 2 survey. The impact on this quarter’s data is shown in published supplementary tables. The effect of imputation on totals is relatively minor (one provider responsible for one building) given the minimal missing information.

This means that, although response coverage is very good and data accuracy is improving, the data may not provide a complete picture of fire safety remediation plans across all relevant buildings in the sector. As we analyse submissions, we monitor the quality of data provided and engage directly with RPs where we have concerns about data accuracy.

Imputed data are highlighted in the release and are covered in the supporting notes for the relevant tables.

Quality assurance of the published statistics

The data, briefing note, look up tools and tables are quality assured by analysts within our statistics production team. This process ensures the figures are consistent across the release, and match the raw data submitted through the FRS. Each check is signed off and recorded by the responsible statistician when it has been completed.

Revisions

Under the revisions policy, errors identified in the underlying FRS data will be investigated, and revised data gathered. Some corrections may be only minor changes to the data, with little or no impact on the published statistics. These changes will be published at the next scheduled release with no specific announcement.

However, if we become aware of substantial errors in the submitted data, statistical process or other methodology and where a major revision to the published data is required, a non-scheduled revision of the statistical release will be published. This will include full details of the revisions, clearly marked data amendments and summary tables showing the overall impact of the changes.

We seek to ensure transparency in processes to maximise user confidence in the quality of our statistical releases.

Why not have your say on our statistics in 2024/25?

We want to hear your views on how the format and range of documents in this statistical release meet your needs. Please email feedback, including suggestions for improvements to [email protected]. or click below to quickly rate how this document meets your needs.

• All needs met

• Some needs met

• No needs met

Contact information

Responsible statistician: Amanda Hall

Public enquiries: [email protected] or 0300 124 5225