Business population estimates for the UK and regions 2024: methodology note

Question 1

Introduction

Accepted Answer

The Business population estimates for the UK and Regions (hereafter referred to as BPE) is produced by the Department for Business and Trade. Prior to 2023 it was produced by the Department for Business, Energy and Industrial Strategy (BEIS). It superseded the Small and Medium Enterprise (SME) Statistics for the UK and Regions series in 2010 using an improved methodology. The two series should be used independently.

The BPE 2010 publication was the first release based on the improved methodology. Subsequent methodological refinements are detailed in Appendix A.

There is no single UK database detailing every active business. The Government’s Inter-Departmental Business Register (IDBR) contains all businesses operating VAT and/or PAYE schemes (see Annex B for more information). This publication takes data on businesses on the IDBR, and then estimates the number of sole traders and partnerships unregistered for VAT and/or PAYE.

This publication is the only official publication providing information on both registered and unregistered UK businesses.

The status of the publication has changed to official statistics in development (from accredited official statistics) in 2024 to reflect a similar change in status to Labour Force Survey data that underlie estimates of unregistered business data included in the publication.

Between 2012 and 2023 editions of the BPE were accredited official statistics (referred to as National Statistics in the Statistics and Registration Service Act 2007).

This document provides guidance for users on methods used and the quality of the publication. It describes how to interpret the data in the BPE publication, and how it differs from the previous SME statistics series and other data published by the government on the UK business population.

The BPE publications for 2010 to 2024 are available from the GOV.UK website. SME Statistics publications for 1994 to 2009 are available from the National Archives. For further information, contact the production team using the email address given on the BPE collection page.

Question 2

Definitions and coverage

Accepted Answer

Definitions

Business

a business may be VAT and/or PAYE registered and therefore appear on the government business register (IDBR), or may exist without such registration
a registered business or enterprise is the smallest combination of legal units (generally based on VAT and/or PAYE records) which has a certain degree of autonomy within an enterprise group. A branch or office of a larger organisation is not in itself an enterprise. For example, a supermarket chain will count as one business, with the employment and turnover of individual stores, distribution centers etc included within this
unregistered businesses are those businesses run by self-employed people that are not large enough to be VAT and/or PAYE registered and therefore will not appear on the IDBR
a single person may run more than one business and each of these businesses is counted separately in the BPE

Employees

the number of employees refers to the number of people working within the business under a contract of employment in return for a wage or salary. A business can be classed as having no employees if all the business is conducted by working proprietors. For example, 20 partners in a partnership could all be defined as working proprietors if none of them were drawing a salary and therefore classed as an employee
in the detailed tables that accompany the latest publication, Table 1 shows employment split out between working proprietors and employees

Employment

Turnover

turnover refers to the value of sales, work done and services rendered. It excludes VAT. Turnover data for registered businesses comes from the IDBR. Turnover data for unregistered businesses is estimated (see section ‘Economic contribution’ in ‘Methodology and Revisions’ for further details). Turnover data in the BPE are indicative and for registered businesses the recommended source of turnover data is the Annual Business Survey
where there is an enterprise group, turnover for all the VAT enterprises within the group may be notified (to HMRC) by a reporting or ‘standard’ unit. If this is not updated from other sources the IDBR record will hold an unusually high turnover for the reporting enterprise and other enterprises within the group will hold an estimated turnover. Therefore, turnover in a small number of businesses may be too high

Size of business

the size of a business refers to the number of employees within a business. In the Statistical Release, small businesses are classed as those with 0 to 49 employees, medium as 50 to 249 employees and large as 250 or more employees. The term “SME” refers to all businesses with 0 to 249 employees.

Coverage

With the exception of the Whole Economy table (Table 2) in the detailed tables, the BPE publication covers private sector businesses, including those that sell their products exclusively to the government. The private sector includes a small number of public corporations and nationalised bodies. It excludes central and local government, charities and other non-profit organisations, which are shown in Table 2 of the publication only.

The local and central government sector is comprised of all enterprises classed by ONS as having an institutional classification of either local or central government, and private sector enterprises with SIC2007 classification codes 841 (administration of the state and the economic and social policy of the community, 842 (provision of services to the community as a whole), and 843 (compulsory social security activities).

Private households and extra-territorial bodies (SIC code sections T and U) are not included.

There is no lower boundary of business activity for inclusion in the BPE publication. Even a small amount of business activity counts. However, many “no employee” businesses are labour-only sub-contractors; that is, self-employed people trading in their own skills or professional knowledge. They might work for just one customer. Many are genuine entrepreneurs or at least think of themselves as separate from the organisation to which they sell their skills. The improved BPE methodology used since the 2010 edition makes use of additional questions on the ONS Labour Force Survey (LFS) around who pays the salary or wage of the self-employed and whether they pay their own National Insurance and Income Tax to better identify “true” self-employed activity (see section ‘Methodology and Revisions’ for further details).

Apart from Table 2 in the detailed data tables, which also includes Government and non-profit businesses, the BPE publication comprises businesses from three types of legal status only; sole proprietorships (run by one self-employed person), ordinary partnerships (partnerships with unlimited liability – we generally refer to these simply as ‘partnerships’) run by two or more self-employed people, and companies (Companies can take a range of legal forms, including Public Limited Companies, Private Limited Companies, Limited Liability Partnerships, and others such as public corporations and nationalized bodies) in which the working directors are classed as employees.

The methodology used includes all registered companies, sole traders and ordinary partnerships, along with unregistered sole traders and ordinary partnerships. Unregistered companies are not included in the BPE.

The most detailed industry tables provided in the publication are disaggregated to 3-digit SIC 2007 level for businesses identified as employers only. Estimates of businesses with no employees rely heavily on the ONS LFS which is a sample survey and insufficiently robust to provide accurate estimates at the 3-digit SIC 2007 level.

Companies with only one PAYE employee on the IDBR are counted in the “with no employees” category, rather than the “1 employee” category, as the employee is treated as being equivalent to a “working proprietor”. This is done to ensure that incorporations by individuals operating alone do not distort the overall numbers of businesses with employees – in businesses run as sole proprietorships or ordinary partnerships, working owners pay income tax through self-assessment not PAYE, whereas in companies they will usually draw a salary through PAYE.

Companies without employees can be holding companies, investment trusts or pension funds and most of these will have no associated employment. In addition, a small number are companies where, for technical reasons, employee directors are already treated as working proprietors.

Composite and managed service companies set-up to take advantage of specific tax benefits were excluded from previous BPE and ONS publications until 2018. The exclusion was applied at postcode level. Since 2018, these postcodes have been included in the data, as there is no evidence to support their continued exclusion. Over 99.9% of the businesses that would previously have been excluded from the publication are small (0 to 49 employees), so their inclusion has a limited effect on headline small business figures.

Question 3

Methodology and revisions

Accepted Answer

As there is no database of all businesses in existence in the UK, alternative sources are used to estimate numbers of businesses not on the IDBR, that is unregistered businesses. Further background on the IDBR can be found in Annex B.

Methodology in brief

The IDBR contains all VAT and/or PAYE registered businesses. Additional estimates of unregistered sole proprietors and partnerships run by self-employed people either alone (sole proprietorships) or with others (partnerships) are also made. No estimates of unregistered companies are included in the BPE as no existing procedure reliably distinguishes those that are active from those that are inactive. A paper published alongside the 2023 BPE provides further details.

To estimate numbers of unregistered businesses, ONS LFS data are used to estimate numbers of self- employed ‘jobs’, counting self-employment in first and/or second declared jobs separately, where applicable (see Annex C for further details about the LFS). Alongside this, the IDBR is used to count self-employed jobs in IDBR registered businesses (working proprietors). By subtracting the IDBR working proprietor jobs from the ONS LFS self-employed jobs total, an estimate of self-employed jobs there are in unregistered sole proprietors and partnerships can be made.

A ratio of self-employed working alone or in partnership and the average number of partners per partnership (from HMRC self-assessment tax return data) is then used to transform estimates of self-employed jobs into estimates of unregistered businesses. This estimate is added to the IDBR count of registered businesses to give the estimated total business population in the UK (see Figure A).

Figure A: Flowchart illustrating BPE methodology

Note for Figure A: Where IDBR information for self-employed sole proprietors and self-employed jobs in partnerships is greater than the total LFS self-employed for any 2-digit SIC code, total remaining self-employed jobs is set to zero to avoid negative numbers. The IDBR is deemed to be a more accurate source than LFS data. Hence the sum subtracted from the LFS self-employed estimate differs somewhat from the sum of self-employed working in sole proprietorships and partnerships on the IDBR.

Methodology in detail

Self-employment estimates from the ONS LFS

Self-employed jobs are calculated by combining data from the Q4 (of the year preceding the BPE publication year) and the Q1 (of the BPE publication year) quarterly ONS LFS micro-data (seasonally unadjusted) files on the basis of tax paid.

The resulting total is primarily made up of those people self-employed in their main job, with the reminder having a second job with self-employed status. Some of those with self-employment as a second job may also have self-employment as a first job but these figures can be added together because it is the number of businesses that is of interest, rather than the number of individuals.

Overlap of the IDBR and LFS self-employment count

Adding self-employed jobs estimated from the LFS to the IDBR count would lead to double counting of businesses in some instances since not all self-employed people run businesses on their own (some are in a partnership), whilst others run businesses that are VAT and/or PAYE registered and so already appear on the IDBR

The LFS does not ask individuals whether they work alone or in a partnership, or whether their businesses are VAT and/or PAYE registered, so this must be estimated.

Self-employed on the IDBR

The first step involves estimating how many self-employed people (working proprietors) are registered for VAT and/or PAYE with HMRC and are therefore on the IDBR.

The IDBR records employment mainly via survey forms (currently, BRES). If there is no source of employment information (since BRES is a sample survey for the smallest units), then employment is taken from PAYE jobs where available: otherwise it is imputed from turnover data. To allocate imputed employment into working proprietors or employees, the number of working proprietors is assumed to be one per sole proprietorship, and around two per partnership.

Since directors of a company are usually classed as employees, all private sector self-employed on the IDBR will be either in “sole proprietorships” or “partnerships”.

Self-employed not on the IDBR

The second step is to estimate how many businesses are run by people in the remaining self-employed ‘jobs’.

HMRC’s Self-Assessment Tax returns data (SA) is a further source of self- employment data (see Annex D). Analysis of SA data is used to estimate the proportion of unregistered self-employed people working either as a sole proprietor or a partner. Unregistered businesses are approximated in the HMRC SA data by selecting those businesses with turnover below the VAT threshold and that don’t have employee costs (as a proxy for businesses not registered for PAYE).

HMRC SA data consistently indicates self-employed people who are in partnership, rather than acting alone, are in the minority. However, the ratio varies from industry to industry and in the BPE publication, specific ratios are applied for each industry division (2-digit SIC).

Each sole proprietorship is assumed to have only one working proprietor (self- employed person). Therefore, the number of unregistered sole proprietors is assumed to equal the number of unregistered sole proprietorships.

Partnerships are assumed to have two or more working proprietors. Using information from HMRC SA data, DBT calculate the average number of working proprietors (self-employed partners) in each partnership (based on data for self-employed businesses operating below the VAT registration threshold and with zero employment costs to approximate unregistered businesses). This ratio is then used to estimate the number of unregistered partnerships.

Among unregistered businesses, sole proprietorships are more numerous than partnerships, making up more than 90% of unregistered businesses .

Among IDBR registered businesses, the majority are companies, with far smaller numbers of sole propriotorships and partnerships.

Unregistered businesses are small. It is assumed that they are all in the smallest size class (no employees), and are all self-employed people working alone or in partnership. Some may generate employment, but it is likely to be transitory or low paid, otherwise a PAYE scheme would be in operation.

Economic contribution

Turnover in unregistered businesses will generally be lower than that of registered businesses of the same size: turnover in the former is typically below the VAT threshold.

Turnover for unregistered businesses is imputed by calculating the average turnover per working proprietor for registered zero employee businesses at the 2-digit industry division level, and then halving it.

For some 2-digit industry divisions, this still leaves average annual turnover per unregistered sole proprietorship or partnership above the VAT threshold. In these cases, unregistered turnover is adjusted so that turnover per business is just below the VAT threshold.

Turnover from unregistered businesses contributes less than 5% to total turnover reported in the publication.

Statistical disclosure control

The disclosure of data relating to individual undertakings without consent is prohibited under the Statistics of Trade Act 1947, the Value Added Tax Act 1994 and the Finance Act 1969.

Counts of businesses based on IDBR data have been rounded to avoid disclosure using controlled rounding to base five. Controlled rounding means that cells are rounded up or down to the adjacent multiples of five in a way that retains the additivity of tables. For example, an original value of 23 will be rounded to either 20 or 25 and rounded values in a row or column always add up to the rounded row/column total. Original cell values of zero or multiples of the base are unchanged. Values may be rounded down to zero and so not all zeros are necessarily true zeros.

As the BPE publication is sourced from both the IDBR and LFS, there will be a few cases where the addition of estimates of unregistered businesses to the IDBR data means totals do not equal the sum of the data in columns. In addition, percentages may not always add to 100.

Employment and turnover information is suppressed where the count is deemed to be disclosive. To avoid disclosure by deduction, secondary suppression is applied.

The “with no employees” size class estimates rely heavily on the ONS LFS (see Annex C for details)

In April 2004, the Statistical Policy Committee expressed a strong preference for publishing data rather than suppressing data for reasons of quality. Since then, the previously applied threshold of 10,000 (in the region of 20 respondents), below which outputs were suppressed, was removed from labour market outputs. Therefore, such estimates are not suppressed in the BPE publication. Users should be aware that some estimates are based on small sample sizes which could explain volatility in certain estimates (for example, regional data at a broad sector level) from one year to the next.

The exception to this is where estimates are based on fewer than three respondents. These could potentially be disclosive, so they are still suppressed in the BPE publication, along with additional values to avoid disclosure by deduction.

Revisions policy

Until 2019, when re-weighted (for updated population estimates) ONS LFS data became available, revised headline estimates of the number of private sector businesses in the UK for the relevant years were provided. Updated estimates by size band were provided in Figure 3 and Table B of the Statistical Release and Table 25 of the detailed data tables, to reflect the impact on the unregistered business population estimate.

SInce 2019, no revised estimates have been published despite the availability of re-weighted LFS data, as the quantification of uncertainty in the estimates indicates the effects of re-weighting are trivial given the degree of uncertainty present.

DBT will make use of re-weighted LFS data only if such data is judged to have significant impacts on business counts.

Where possible, DBT will revise previously released estimates if new sources of information become available that are shown to improve the robustness of the BPE. Any future substantial methodological changes of this nature will be consulted on and pre- announced before they are implemented.

Unexpected revisions may occur due to the discovery of errors, unforeseen changes in methodology or the unexpected acquisition of new data. If an error is discovered, a corrected version of the publication will be published, together with an explanation of the reason for the change, where appropriate in line with DBT’s statistical error policy. In the case of implementation of unforeseen changes in the methodology or the unexpected acquisition of new data, it will not usually be possible to pre-announce changes, but as far as possible we will produce a consistent headline back series to enable robust comparisons over time.

Question 4

Quality

Accepted Answer

The status of the BPE has been lowered to official statistics in development in 2024, at DBT’s request, to reflect a similar change in the status of the LFS data from which estimates of unregistered businesses included in the publication are derived.

Between 2012 and 2023 editions of the BPE were accredited official statistics (referred to as National Statistics in the Statistics and Registration Service Act 2007). Accredited official statistics are produced to high professional standards set out in the Code of Practice for Statistics. Further information on accredited official statistics can be found on the United Kingdom Statistics Authority (UKSA) website.

These statistics were assessed for compliance against the Code of Practice for Statistics by UKSA between 2011 and 2012. In October 2012 their National Statistics (now called accredited official statistics) designation was confirmed. Following an Office for Statistics Regulation (OSR) compliance check in August 2022, the designation was retained.

Details of improvements since the 2011/2012 UKSA assessment and the 2022 OSR compliance review can be found in the Statistical Release under the heading ‘Official statistics in development’. Designation means the statistics meet identified user needs, are well explained and readily accessible, are produced according to sound methods, and are managed impartially and objectively in the public interest. Once statistics have been accredited, it is a statutory requirement that the Code of Practice shall continue to be observed.

Relevance

No single database contains details of every active UK business. This publication is the most comprehensive official source on the population of UK businesses.

The sources used to compile the estimates are the IDBR, ONS’ LFS and HMRC self-assessment data. Further information on the primary purpose, uses and quality of these sources is available in Annexes B, C and D.

The IDBR provides comprehensive coverage of VAT/PAYE registered businesses.

The LFS is an effective way to estimate the number of sole proprietorships and ordinary partnerships not on the IDBR as it can be used to estimate of numbers of self- employed jobs not on the IDBR.

HMRC self-assessment data is appropriate to allocate these LFS self-employed jobs into businesses due to its comprehensive coverage of the self-employed population, the ability to remove self-employed that are most likely running registered businesses and the information it gathers about whether the self-employed are operating in partnership or as a sole proprietorship.

Key users of this publication include central and local government, business owners, academics and students, business representative bodies, journalists and members of the public.

Key uses of this publication include informing policy making, policy monitoring, market research, impact assessments, facilitating academic research, etc.

Users have been consulted when considering changes to the BPE publication. In December 2008, BIS launched a Public Consultation on SME Statistics to get user views on the usefulness of the publication and suggestions for improvements. The consultation closed in March 2009 and was followed by the publication of a Government Response in June 2009.

Information on the changes introduced following the consultation can be found in the article Business Population Estimates: introducing improved statistics on the UK enterprise population .

In response to user feedback, most of the BPE tables provide specific estimates for unregistered businesses, separately from numbers of non-employing registered businesses. Previously the two were combined to provide only a total number of non-employing businesses.

DBT is aware of user appetite for more information on the socio- demographic details of business owners, but is unable to provide such information as it is not collected on the IDBR and as businesses often have more than one owner-manager. Users are referred to surveys, such as the ONS LFS for characteristics of the self-employed population, or the DBT sponsored Longitudinal Small Business Survey, which provides estimates of the proportion of SMEs that are women-led or ethnic minority-led, for example.

Additionally, DBT are aware of user interest in more detailed dis-aggregations of BPE data, for example to more detailed geographic levels and more fine grained industrial levels for the zero-employing businesses. However, the estimate of the unregistered business population relies on survey data and sample sizes are not sufficiently large to produce robust estimates at more detailed dis-aggregations under the current methodology. The level of suppression required in producing more granular estimates would also be unacceptably high.

Further information relating to relevance is available elsewhere in this methodology note, including definitions of key concepts and descriptions of key sources used can be found in the ‘Definitions and Coverage’ section of this document.

Accuracy

Producing robust information on the accuracy of the estimates in the BPE publication is complicated by the fact that it is based on a combination of survey and administrative data.

Information in the publication relating to registered businesses will not be subject to survey error since it is based on administrative information from the IDBR (see Annex C for more quality information on the IDBR). However, a range of other factors affect the quality of IDBR data, and although errors are likely to be small, they will nevertheless be present.

Information in the publication relating to unregistered businesses is based on a combination of administrative data from the IDBR and HMRC self-assessment data and estimates from the ONS LFS. Since the LFS is a sampled survey, estimates will be subject to error, as will estimates based on a combination of registered and unregistered business data.

The 2020 publication quantified uncertainty around point estimates of business numbers, using 95% confidence intervals (CIs) and coefficients of variation (CVs), at the country and English region level, as well as for broad industry sections. From 2021 these estimates were extended to include information on uncertainty at the country/English region by industry section level.

To quantify uncertainty, the uncertainty around estimates of self-employed job counts deriving from LFS data were calculated, as these are the primary data source used to produce estimates of unregistered business numbers. Because estimates of self-employed jobs are based on two consecutive LFS datasets, the correlation between the two datasets must be calculated. Rather than calculating the correlation separately at each level for which uncertainty was quantified (that is for each region/country and each sector), the top-level correlation value (that is the correlation between the total self- employed job count) was used in all calculations. This method is more likely to produce robust estimates over successive editions of the publication than one in which correlations between LFS datasets are calculated separately for each level at which uncertainty is quantified.

Uncertainty around estimates of self-employed job counts is transformed to represent uncertainty around numbers of unregistered businesses. In doing so, no adjustments are made to account for uncertainty either in the HMRC data or inaccuracies in the IDBR data used transform estimates of self-employed job numbers into estimates of unregistered business numbers. Since the IDBR holds information on all registered enterprises it is not subject to uncertainty of a kind that can be quantified using measures such as confidence intervals. HMRC data provides a near census of the self-employed, though will still be subject to some uncertainty. However, the main source of uncertainty in the BPE estimates of business numbers derives from sampling error in the LFS and the CIs and CVs produced reliably capture this source of error in the estimates.

Information on the accuracy of the individual underlying data sources used to produce the BPE publication (ONS LFS, IDBR and HMRC self-assessment data) is available in Annexes B, C and D.

Most employment data for businesses registered on the IDBR come from PAYE data, some from the BRES conducted in September 2023, while other data will come from earlier BRES surveys, other ONS surveys or will be imputed. For the smallest businesses which are less likely to be surveyed by BRES, either PAYE jobs or imputed employment is used, based on sector average employment. For further information refer to Annex B. In BPE 2015 a time-series of employment in the private sector by size band (2010 to 2015) was included for the first time in the publication.

Turnover data for businesses registered on the IDBR primarily use VAT returns for a 12-month period. For BPE 2024, this means VAT returns relating to the 12- month period ending December 2022, or January/February 2023, depending on the reporting pattern of the business. However, for some businesses this data could be less timely or, for newly registered businesses, turnover will be based on estimates made by traders at the time of registration. Some turnover data comes from the annual ABS survey but if the business is small and not registered for VAT, turnover is imputed from sector average turnover per head. For further information refer to Annex B.

Employment and turnover data are also provided, primarily for use in classifying businesses by employee size band and to calculate shares of employment and turnover across industrial sectors, regions, and legal statuses. For registered businesses, the preferred source of turnover data is the Annual Business Survey, whilst ONS publishes other sources of employment data more suitable for monitoring total jobs and people in employment. See the Workforce Jobs series, the Business Registers Employment Survey (BRES), and the wider Labour Market Statistics

CIs for counts of all businesses are the same as those for unregistered businesses at any given level of breakdown. Such counts comprise a combination of estimated numbers of unregistered businesses and registered businesses numbers, which are not estimated. CVs for counts of all businesses will be smaller than for counts of unregistered businesses.

Timeliness and punctuality

The BPE provides an estimate of the number of businesses at the start of a calendar year, together with estimates of their associated employment and turnover. The estimates are published annually, approximately 10 months after the reference point, which is the earliest possible date after the data have been prepared and fully verified.

A count of IDBR businesses that were “live” at the start of the year is achieved by referring to each business recorded start date and, where appropriate, its closure date.

To produce the estimate of the number of unregistered businesses, information on the number of self-employed jobs is taken from the LFS. Pooled data for the two quarters either side of the start of the year are used in the production of the BPE estimate (that is, quarter 4 (October to December) and quarter 1 (January to March)).

The publication date for the next edition is pre-announced on the GOV.UK Release Calendar. The statistics have always been published on the pre-announced target date.

Accessibility and clarity

The BPE publication is available free of charge to the end user on the GOV.UK.

The BPE publication comprises a Statistical Release containing key results and commentary to help interpret the data, detailed data tables in accessible Excel format (since 2022) and in ODS, along with this Methodology and Quality Note. All elements published are in accessible format.

Queries about the publication should be directed to the email address given on the BPE collection page.

Comparability

To enable a robust comparison over time the BPE publication includes a headline time series of the estimated number of UK businesses each year since 2010 by employee size band. This has been produced on a consistent basis using the latest BPE methodology (see Annex A for more information). Where consistent data was not available (some key LFS information prior to 2007), modelling has been used to estimate information to produce these counts.

Since 2014, a time series of businesses numbers for the English regions and UK countries has been published. This covers the years 2010 to the latest date. DBT does not have sufficiently detailed data to produce consistent estimates for earlier years. Since 2015, a time-series of business numbers by legal status has been provided in the detailed tables.

As the IDBR is a live register, continually updated each time a business registers or de-registers for VAT and/or PAYE, it provides a reliable record of the number of registered businesses in operation at any point in time. Since the BPE publication is based an IDBR extract referencing the start of January (and a combination of the fourth and first quarters ONS LFS data for unregistered businesses), comparisons of business counts from year to year can be considered reliable.

The main purpose of the BPE publication is to provide information about the number of businesses operating in the UK at the start of each year. The publication also includes information about employment and turnover. Both turnover and employment information is updated on the IDBR via numerous sources and at different times (particularly for employment with the introduction of BRES in 2009). This makes it difficult to obtain accurate employment and turnover estimates on the IDBR because the figures for each business may not relate to the same point in time.

For this reason, ONS recommends employment and turnover changes are assessed using other headline indicators where available (see Annex E). The primary purpose of the employment and turnover data provided in BPE is to understand the overall contribution of SMEs including unregistered businesses, and to compare shares across different groups within the population, for example industrial sectors. The BPE has included an employment time series since 2015 as no other estimates of employment by size of business are currently available from ONS.

When making comparisons over time, consideration should be made of the discontinuities explained in Annex A.

In the BPE publication, the Standard Industrial Classification (SIC) 2007 is used to classify businesses by the type of economic activity in which they are engaged. This industrial classification system is used extensively by ONS in its’ publications, and is fully consistent with the EU (NACE) and UN (ISIC) systems.

Coherence

For a comprehensive summary of the various official statistics relating to the UK business population, their key characteristics and when to use each one, please refer to the User Guide

The headline estimate of self-employment published in the ONS Labour Market Statistics release (see Table ‘EMP01’) will differ from the estimate of self-employment used in the production of these estimates. This is because the ONS published measure describes the number of people who said they were self-employed in their main job (and is seasonally adjusted), whereas the BPE estimate counts self-employment in respondents’ first and second jobs, and uses additional variables to ensure those included are ‘true’ self-employed jobs (by checking who pays the respondent’s National Insurance, income tax and wage). Further information can be found in the Economic and Labour Market Review article - Business Population Estimates: introducing improved statistics on the UK enterprise population . A minor refinement to the process described here was introduced in 2013, using new LFS variables to directly identify ‘true’ second job self- employment, rather than imputing based on the relationship between self-reported and ‘true’ main job self-employment. This had only a marginal impact on the final figures.

Estimates of employment in the BPE publication differ from other published sources from ONS. See the sections below on Workforce jobs and BRES for more information.

Workforce jobs

Workforce jobs is the ONS preferred source for the number of jobs and the industrial composition of these jobs.

The ONS publishes official estimates of workforce jobs (see Table ‘JOBS01’). The total estimate of employment shown in most tables in BPE is smaller because it is for the private sector only, and excludes employment in central and local government, charities and other non-profit organisations, HM Forces and participants on government training programmes who do not have a contract of employment.

The “Whole Economy” table (Table 2) will also show total employment as lower than the workforce jobs estimate above – which also includes individuals on government training programs.

Workforce jobs are drawn from a number of different sources, including employer surveys, and are seasonally adjusted.

Business register employment survey (BRES)

BRES is the ONS’ preferred source for employee data at detailed regional and industrial levels. It has the benefit of being able to provide sub- regional employee estimates but does not currently provide a size breakdown.

Regional estimates in BRES and BPE are fundamentally different: in the BPE if a business has establishments in more than one region the total employment is allocated to the region where the business is registered, whilst in BRES allocation is based on the physical location of the jobs.

Question 5

Annex A : Business population estimates methodology changes and discontinuities

Accepted Answer

This section provides information for those who wish to use the BPE series (2010 to 2024) to make comparisons over time.

Due to the significant methodological changes between SME statistics and BPE, direct comparisons should not be made between SME statistics and the BPE series. For information on discontinuities within the SME statistics series, please refer to other sections of this Methodology Note.

Since 2023, the BPE includes a headline UK level time series of estimates of the business population between 2010 and the latest year (published in Figure 2 of the Statistical Release and Table 25 of the detailed tables. In earlier editions, the equivalent time series run from 2000. These UK level estimates revise and supersede those annual estimates previously published in earlier editions of the release. The time series has been produced on a consistent basis using the latest BPE methodology (although please refer to the section below on ‘Discontinuities in the BPE time series 2000 to 2024’ for further information).

Methodology changes in the Business population estimates series

The BPE 2024 publication has been produced using the same methodology as that used for the 2023 edition (see Annex D for information on changes made in 2023 relating to use of HMRC self-assessment data to estimate numbers of unregistered businesses). While there have been changes in methodology over time (detailed below), the time series included in BPE 2024 include historical data consistent with the methodology used in 2011 onwards.

For the BPE 2011 publication, there was a change in the methodology used to estimate the number of unregistered businesses due to the availability of more detailed self-assessment data from HMRC. This allowed us to approximate to those individuals working within unregistered businesses by using information on turnover and employment costs as proxies for VAT and PAYE registrations. Because the data more closely represents the unregistered business population, and because of its comprehensive nature, the ratios of sole proprietors to partners and partners per partnership in unregistered businesses could be produced and used at a more detailed level. This meant that calculation of the number of unregistered businesses became more accurate because it is based only on information on the unregistered businesses of interest. As a result, the ratio of sole proprietors to partners in the final estimate of these businesses is more in line with the source HMRC self-assessment data than was previously the case.

The impact of this change was to increase the estimated number of unregistered partnerships and therefore reduce the estimated total number of unregistered businesses, so that at the start of 2011 the total estimated number of businesses was approximately 90,000 fewer than it would have been using the old methodology.

In the BPE 2011 publication, businesses in SIC 2007 Division 78 (Employment activities) were included for the first time. It had previously been thought their inclusion could lead to double counting of employees, who could be counted both as employees of the agency, and as employees of the business they are currently working for. However, the ONS agreed that this was not the case and that robust estimates of both counts of businesses and employment in the ‘Employment Agencies’ sector could be provided from the IDBR.

The impact of this change was to increase the total number of businesses in BPE 2011 by around 30,000. In the BPE 2012 and BPE 2013 publications, turnover for these businesses was also included, in line with ONS methodology for the UK Business publication.

Discontinuities in time series 2000 to 2024

In the publication, a time series of counts of businesses by size band at the UK level is provided (see Figure 2 in the Statistical Release and Table 25 of detailed data tables), produced using a consistent methodology (using some modeled data where real information was not available).

Since 2023 these time series run from 2010, whereas in earlier editions, they start from 2000. More detailed time series provided for country/regional level counts of businesses and for different legal forms, have always run from 2010 because insufficient data is available to produce this for earlier years. When interpreting these time series bear in mind the discontinuities listed below (listed in reverse chronological order).

HMRC introduced a new Real Time PAYE reporting system (RTI) in 2014. Although it is possible that the change may have had marginal effects on employee numbers recorded through PAYE, HMRC found no evidence of behavioural changes in the timing of scheme registrations through the year and no clear impact on the IDBR has been identified
in 2013, a minor refinement was made to the identification of ‘true’ second job self-employment. Rather than imputing based on the relationship between self-reported and ‘true’ main job self-employment, new LFS variables were used to directly identify this in line with the method used for main job self-employment. This had a minimal impact on the final figures
in BPE 2012 and 2013, there was an increase in each year of just over 2,000 local and central government bodies that were registered for either VAT and/or PAYE. (see Table 2 ‘Whole Economy’ in the detailed data tables that accompany the release). This increase was partly driven by changes in how parish council clerks are treated for tax purposes by HMRC. From April 2011, according to guidance issued by HMRC, Parish Clerks can never be considered self-employed for tax or National Insurance contribution purposes, must not be paid “gross” and also must be taxed under PAYE
some of the extra businesses added to the BPE in 2011 existed before 2011 and therefore should have been included in previous estimates of the total business population. In 2014 BIS worked with ONS to identify the correct year of birth for these businesses and include them at the correct point in the time series
due to the implementation of a new computing system at HMRC, there were no quarterly updates of the PAYE data between June 2010 and March 2011 (when the IDBR extract for BPE 2011 was taken). Therefore, in BPE 2011 employment data for many smaller businesses related to a period of time at least 6 months before the reference point of the publication, that is the start of 2011
after the publication of BPE 2010, ONS reclassified Further Education (FE) Corporations in England and Wales from Non-Profit Institutions Serving Households (NPISH) to Central Government, from their inception in April 1993. NPISH is often known as the ‘third sector’; in National Accounts. NPISH is part of the private sector. Consequently, in BPE 2011 over 400 FE colleges have been reallocated from the private sector to the central government sector. For further information refer to ONS classification of sixth-form and further education institutions provided here
between 2000 and 2002, Primary Care Trusts and National Health Service trusts were classified as public corporations / nationalised bodies, and therefore included with the figures for companies and the private sector
in 2003, the ONS reclassified these trusts, resulting in 275 businesses, with employment of 990,000 and turnover of £41 billion, being moved out of the figures for companies and private sector, and into the figures for Central Government. So, in the BPE headline time series before 2003, private sector figures will appear larger and Government figures will appear smaller than in subsequent years

Question 6

Annex B: ONS Inter-departmental business register (IDBR)

Accepted Answer

The IDBR is a business register administered by the Office for National Statistics (ONS). It holds records of all businesses registered for Value Added Tax (VAT) and all businesses operating a Pay As You Earn (PAYE) scheme. The IDBR holds information on each enterprise’s employment (and hence size), turnover and industry.

The IDBR is the main sampling frame for surveys of businesses carried out by the ONS and other government departments. It is also a key data source for analyses of business activity.

The main administrative source for the IDBR is HMRC for VAT information (passed to the ONS under the Value Added Tax Act 1994) and PAYE information (transferred under the Finance Act 1969). Use the links provided here for the information on both on the HMRC VAT and PAYE registration thresholds. Other information is added to the register, from various sources, if required for ONS statistical purposes.

Some employment information currently on the IDBR is drawn from the most recently available Business Register Employment Survey (BRES) data, whilst some will be drawn from earlier BRES survey or from PAYE data. BRES is a survey of 80,000 businesses, with survey forms being sent out in September every year. Therefore, some of the employment data on the IDBR will relate to a point about four months earlier than the reference point for BPE 2024 (the start of 2024), whilst some may be significantly more lagged.

Since the implementation of BRES in 2009, employment data is updated on the IDBR on a monthly basis rather than annually, as updates are made as completed surveys are returned. This means that, at any point in time, employment data on the IDBR will be as accurate and up to date as possible, based on the survey returns that have been received at that time. Because this is based on a sample of businesses, estimates from previous surveys and from other ONS surveys have also been used. For the smallest units, either PAYE jobs or employment imputed from VAT turnover is used. Where employment data for a single business is captured from both PAYE and BRES, then the BRES data is given precedence and is used to update the IDBR.

Turnover provided to the ONS for the majority of businesses at the start of each year is based on ONS surveys or VAT returns for a 12-month period ending in December the previous year, or January / February of the reference year, according to the reporting pattern of the business. For other records, in particular VAT group registrations, turnover may relate to an earlier period or survey data. For businesses who have registered more recently, turnover represents the estimate made by businesses at the time of registration. For the PAYE-only records for which there are no data, turnover is imputed from employment.

The IDBR is continuously updated as new data from surveys are received and as businesses register or de-register for VAT and/or PAYE. In addition, a variety of checks on individual records improve the quality of the register on an ongoing basis. Thus, there will be differences in results taken at different times from the register as new data are added and other data revised.

Regular VAT and PAYE updates are received from HMRC and these updates are checked with the new VAT and PAYE registrations. Many of the PAYE-based businesses will have no VAT registration; some units may be waiting to be matched to existing VAT records on the IDBR.

Businesses that are neither registered for VAT, nor operate a PAYE scheme are excluded from the IDBR, although some businesses below the VAT threshold register voluntarily for VAT. Employers who have no employees earning above the lower earnings limit for PAYE and no other reasons for deducting PAYE (such as an employee having two jobs) do not need a PAYE scheme, although some may still have one.

Businesses with a turnover above the threshold are not required to register if they trade exclusively in exempt goods. If a business has no employees or only low paid (perhaps part-time) employees, then it is unlikely to operate a PAYE scheme.

The IDBR provides all the data on registered businesses in this publication. It is also the key source for the ONS publications UK Business: Activity, Size and Location and Business Demography.

The ONS provides further information on the quality of the IDBR.

In 2021, following an Office for Statistical Regulation (OSR) review of UK Business Demography, the ONS published plans to meet the review requirements. These included an undertaking to further development of a new Statistical Business Register (SBR) to replace the IDBR. A contract to perform the work was awarded in March 2022.

Question 7

Annex C: ONS Labour force survey

Accepted Answer

The LFS is a quarterly sample survey of households, that is people living at private addresses, in Great Britain and Northern Ireland. It provides information on the UK labour market that can then be used to develop, manage, evaluate and report on labour market policies. The questionnaire design, sample selection, and interviewing are carried out by the Social Survey Division of the ONS on behalf of the Statistical Outputs Group of the ONS.

ONS publishes full UK LFS results. However, the fieldwork is carried out separately by ONS for GB, and by the Central Survey Unit of the Northern Ireland Statistics and Research Agency (NISRA) on behalf of the Department of Finance and Personnel.

The survey seeks information on respondents’ personal circumstances and their labour market status during a specific reference period, normally a period of one week or four weeks (depending on the topic) immediately prior to the interview.

The LFS collects data on economic activity, which is self-reported.

Additional questions on who pays the respondent’s income tax and national insurance, and their wage or salary are also included in the survey.

This source is used by DBT to estimate the number of self-employed jobs in the economy. By excluding those already included on the IDBR (see Annex A) and converting these jobs into businesses, DBT is then able to estimate how many unregistered businesses there are.

For the 2 quarters of LFS used by the BPE 2024, sample size were 50,783 individuals in 23,143 households for January to March 2024 and 44,338 individuals in 20,324 households in October to December 2023.

In 2014 ONS undertook a major re-weighting exercise for LFS data to bring it the outputs in line with population estimates generated using results from the 2011 Census.

In 2016 and 2017 ONS re-weighted the LFS data again, so that LFS estimates would reflect the most recent population estimates available at the time of the re- weighting. The effect of this change means that the un-registered business population numbers published between 2013 and 2016 were revised in BPE 2016 and BPE 2017.

In March 2020, as a result of the COVID-19 pandemic, housing tenure was introduced into the LFS weighting methodology. Self-employed job estimates used in the BPE to underlie calculations estimates of unregistered business used the new tenure weights. ONS has since provided further information on the impact of this re-weighting on LFS key indicators.

ONS provides further information on the quality of the LFS.

Due to falling LFS response rates, the ONS introduced changes to LFS data collection and methodology in late 2023 and further re-weighted data in early 2024.

ONS intends to replace the LFS with a transformed version of the survey. This will affect estimates of unregistered businesses in the BPE and may result in a break in time-series.

Question 8

Annex D: HMRC Self-assessment tax returns data

Accepted Answer

Self-assessment data is collected by HM Revenue & Customs through tax returns completed by self-employed workers either as individuals or in partnership.

Data coming from the self-assessment record offers several advantages over the previously used Survey of Personal Incomes. For example, it is a larger record, being a near-census of the self-employed. Self-assessment data also allows us to approximate to those individuals working within unregistered businesses by using information on turnover and employment costs as proxies for VAT and PAYE registrations.

DBT analyses HMRC Self-assessment data, for every 2-digit SIC classification (and for every region at 1-digit level), to produce an estimate of the ratio of sole proprietors to partners, enabling DBT to convert the number of unregistered self- employed individuals into numbers of actual businesses.

Fundamentally, we would expect HMRC Self-Assessment data to be high quality; there is a legal obligation for people to respond accurately and it is in HMRC’s interests to ensure accurate information. But, as DBT is using the data for a purpose other than the main one for which the data are collected, there are other considerations. The main areas where there are quality considerations are as follows:

Classification into SIC

the BPE calculations are carried out at 2-digit SIC 2007 and also at 1-digit SIC 2007 at regional level. However, at present HMRC use their own classification – the Trade Classification Number (TCN). DBT converts from TCN to SIC 2007 and this is a possible source of error
TCN codes, or equivalent industrial classifications, are no longer used by HMRC in self- assessment datasets although historical TCN codes are retained for business which have them. Consequently, the quality of TCN data has gradually declined over a long period of time. Analysis of data HMRC from sole traders carried out in 2023 showed that in 2010/2011 around half of TCN codes fields associated with new businesses to new businesses were either left blank or assigned to either of 2 general categories (6616 – Building services not elsewhere specified or 3001 – General building and building contractors). In BPE processing of HMRC self-assessment data TCN codes starting 66 are distributed equally between 2-digit SICS 62, 63, 70, 73, 78, 80 and 82, whilst TCN codes starting 30 are distributed equally into SICS 41, 42 and 43). The percentage unassigned or assigned to the general categories increased gradually between 2010/11 and 2013-14. There was then a big increase in 2014/15 and since then approximately 95% of TCNs have been recorded in these categories
in response, from 2023 BPE estimates of the ratio of sole proprietorship to partnerships at the 1 and 2-digit SIC level was made using historical 2013/2014 self- assessment data, whilst the partners per partnership ratio, which does not use TCN values, was calculated using the most recent HMRC data. In previous years, the BPE used the latest available ratio of sole proprietorship to partnerships. Given ever increasing numbers of businesses with TCN fields either blank or allocated values of 6616 or 3001, this is the most appropriate course of action as it uses the most recent data for which TCN fields were relatively well populated in the self- assessment data
using the 2013/2014 self-assessment data for 2023 and later unregistered business estimates, rather than the most recent available data to obtain ratio of sole proprietorship to partnership has little effect on overall estimates of unregistered businesses, which are 0.2% lower when the earlier self-assessment data is used. However, using this methodology estimates of numbers of sole proprietorships are 1.2% lower, whilst estimates of partnerships increased by 7.3%
a potential weakness of this approach, that is using more robust though outdated self- assessment data, is the assumption that the ratio of sole proprietorship to partnerships has remained relatively constant since 2013/2014
the absolute number of cases overall within the SA data does not matter other than the normal rule of thumb that more data is better. HMRC data is used solely to apportion numbers that we already have, so the missing TCN codes should not matter greatly, other than when it varies between sole traders and partners as described above

Regional data

as with TCN data, some of the regional data are missing. However, this is a much smaller problem with less than 2% missing and there is no obvious bias (e.g. concentration in certain TCN categories etc)

Partnership data

there are potentially different ways of looking at partnership data. One way is to look at the HMRC partnership return and take the number of partners reported on that. The alternative is to look at partner returns from individuals and the partnerships that they are linked to. The latter approach yields approximately 10% fewer partners. DBT follows this approach as HMRC advice is that this gives a better coverage of active partners, although it may exclude some valid partnership data

Question 9

Annex E: Links to other sources of business and employment data

Accepted Answer

There are three official statistics publications that provide information on the total UK business population. In addition to these publications, similar related official statistics are released by the three devolved administrations for their countries, the Insolvency Service publishes statistics for corporate and individual insolvency and Companies House publishes statistics based on activity on their register. The Guide To The Business Population And Demographic Statistics Publications sets out key features of the different products to help users make an informed choice about which outputs they should use.

Small and medium enterprise statistics for the UK and regions: 1994 to 2009 (DBT)

for more information on the SME statistics please contact DBT’s Business statistics team.
e-mail: business.statistics@businessandtrade.gov.uk

UK business: activity, size and location (ONS)

this annual publication is compiled from the IDBR. The publication contains detailed information on VAT registered and/or PAYE registered businesses in the UK including size, classification (to 4-digit SIC), business age and location
e-mail: idbrdas@ons.gsi.gov.uk

Business demography (ONS)

this annual publication provides the best official guide to business births, deaths and survival rates. The figures cover businesses registered for either VAT and/or PAYE
e-mail: idbrdas@ons.gsi.gov.uk

Quarterly business demography (ONS)

Experimental quarterly statistics on business births and deaths from the Inter- Departmental Business Register, with high-level breakdowns by industry and region
e-mail: idbrdas@ons.gsi.gov.uk

Workforce jobs (ONS)

Business register employment survey (ONS)

this annual publication provides the best guide to employment at a sub-regional level, and compliments figures available from Workforce Jobs at a national and regional level
e-mail: bres@ons.gov.uk

Company register statistics (Companies House)

Companies House provides a monthly record of all companies registered in the UK, along with a range of other statistical reports
e-mail: enquiries@companies-house.gov.uk

Labour force survey (ONS)

Inter-departmental business register (IDBR) (ONS)

the IDBR is managed by the Office for National Statistics. More detail on the IDBR can be found at the website below
e-mail: idbr.helpdesk@ons.gov.uk

People in work (ONS)

Employment data covering employment rates, hours of work and earnings

Business insights and impact on the UK economy (ONS)

Business insights and impact on the UK economy provides fortnightly information on the impact of challenges facing the economy and other events on UK businesses
e-mail: bics@ons.gov.uk

The insolvency service

the Insolvency Service publish monthly company and individual insolvency statistics for England, Wales and Northern Ireland, as well as monthly company statistics for Scotland
e-mail: statistics@insolvency.gov.uk

Census data (ONS)

the Census 2021 is the most recent census for which full data is currently available. Published and planned outputs can be found on the Census release calendar
e-mail: census.customer.services@ons.gov.uk

Cookies on GOV.UK