Department for Health and Social Care and NHS Digital: QCovid algorithm

Tier 1 - Overview

Name

COVID-19 Clinical Tool/ QCovid algorithms

Description

The aim of the clinical tool is to create a way of predicting how at-risk individuals might be from coronavirus. The tool was developed for use by clinicians in support of conversations with patients about personal risk.
The tool utilises algorithms developed by the University of Oxford, called QCOVID. These combine a number of factors such as age, sex, ethnicity, height and weight (to calculate BMI), and specific health conditions and treatments in order to estimate the combined risk of catching coronavirus and being hospitalised or catching coronavirus and dying.
More information is available from the following link.
The original version of the QCOVID algorithms were also used as part of the Population Risk Assessment to add patients to the Shielded Patient List in February 2021. These patients were advised to shield at that time were provided support for doing so, and were prioritised for COVID-19 vaccination. The Shielded Patient List has since been closed.

URL of the website

Information on the current clinical tool can be found at the following link
More information on the Shielded Patient List can be found at the following link

Contact email

[email protected]

Tier 2 - Ownership and responsibility

1.1 Organisation/ department

Department for Health and Social Care
NHS Digital

1.2 Team

Commissioner: Department for Health and Social Care – Shielding / Enhanced Protection Programme
Provider: NHS Digital

1.3 Senior responsible owner

Senior Responsible Owner during delivery: Phil Harper (DHSC)

1.4 Supplier or developer of the algorithmic tool

BJSS Limited/Sparck
University of Oxford
Hippo
Kainos

1.5 External supplier identifier

BJSS Limited - company number 02777575
Hippo - 09877239
Kainos – company number 09047312

1.6 External supplier role

The University of Oxford developed the algorithm used for the Population Risk Assessment, and those housed within the clinical tool for clinicians.
BJSS Limited developed the architecture, processes and tools for delivering the clinical tool.
Kainos provided resources to lead on requirements and design development for the tool as well as owning the approvals process.
Hippo provided User Researchers and produced reports and guidance that informed development.

1.7 Terms of access to data for external supplier

Client/supplier agreements to cover use of data.
The Statement of Work covers suppliers to NHS Digital who must be Baseline Personnel Security Standard (BPSS) confirmed.
The Oxford University team were given temporary access to central government data in order to develop the algorithm (for example, NIMS and HES) to combine with GP data already held.
Government data was not used in the development of the wider tool that housed the algorithm from Oxford University.
12 team members (for example Data Architects and Testers) also have Disclosure and Barring Service clearance.
Redacted version of the procurement contract can be found here.

Tier 2 - Description

2.1 Scope

The original version of the QCOVID algorithms were used as part of the Population Risk Assessment to add patients to the Shielded Patient List in February 2021. These patients were advised to shield at that time, were provided support for doing so, and were prioritised for COVID-19 vaccination.
The tool has been designed to allow clinicians to input information about individuals in order to calculate risk of suffering adverse outcomes in the event that an individual contracts Covid.
The tool estimates the likelihood of a particular outcome. It does not guarantee outcomes.
The current version of the tool makes use of separate algorithms for vaccinated and unvaccinated patients, but the original research methodology was not designed to establish the effectiveness of vaccines.
The absolute risk estimates produced by the tool were ultimately derived from the study period used in the research. This predates new variants of the virus and other developments in population immunity, so the main purpose of the tool remains as a guide to personal relative risk.

2.2 Benefit

The key benefits of the tool when it was used in the Population Risk Assessment was to identify individuals at a high risk who should be advised to shield and/or be placed into a high priority cohort for the vaccine rollout. The ultimate aim was to ensure protective interventions were targeted at the most vulnerable, to reduce mortality and protect the NHS.
The clinical tool for clinicians is intended to support individual conversations with patients about risk. Originally, the goal was to help patients understand the reasons for being asked to shield and, where relevant, help them do so. Since the end of shielding requirements, it is hoped that better-informed conversations about risk will have supported patients to make appropriate decisions about personal risk, either protecting them from adverse health outcomes or to some extent alleviating concerns about re-engaging with society.
Furthermore, the technology introduced a new capability, created at speed during the early stages of the pandemic, as it adds input to aid more targeted clinical interventions.
The underlying principles that drive the tool may in the future lend themselves to application outside of Covid-19, particularly given that the technology has been proven in a real world application.

2.3 Alternatives considered

Before the development of QCOVID, the Shielded Patient List was developed by identifying patients for protection on the basis of whether or not they had any of a list of medical conditions. This did not take into account demographic factors or combinations of co-morbidities which QCOVID was then able to accommodate in its decision-making process.
England’s Chief Medical Officer commissioned a predictive model for a data-driven approach to COVID-19 clinical risk assessment which resulted in the development of QCOVID.

2.4 Type of model

In essence, the tool creates a risk calculation based on scoring risk factors across a number of data fields pertaining to demographic, clinical and social patient information. This process is detailed below:
The algorithm creates a risk prediction model - QCovid - that provides a weighted, cumulative calculation of absolute risk using the variables associated with poor COVID-19 outcomes. The research on which the model is based uses a Cox proportional hazards survival model using a range of health and demographic variables, excluding those not significant from final models used in decisions. More information on the research behind the current versions of QCOVID can be found at the following link.
The factors incorporated in the model include age, ethnicity, level of deprivation, obesity, whether someone lived in residential care or was homeless, and a range of existing medical conditions, such as cardiovascular disease, diabetes, respiratory disease and cancer.
For the latest clinical tool, separate versions of the QCOVID models were estimated for vaccinated and unvaccinated patients. When using the clinical tool, clinicians enter patients’ vaccination status, and the tool uses this to determine which algorithm will be used in the calculations.

2.5 Frequency of usage

The Clinical Tool has been used within and outside of the NHS. In the period between 1st January 2022 and 31st March 2022, there were 2,180 completed assessments. Increased uptake was driven, in part, due to increased access, as technical restrictions on who could access the tool were removed in January 2022.
Assessment numbers often move with relative infection rate (e.g. higher infection rate leads to more usage of the tool).

2.6 Phase

Production/run and maintain

2.7 Maintenance

Maintenance schedules for the algorithm are dictated by the emerging understanding of Covid and any new variants and therefore do not fit into a routine schedule.
Changes are processed through a RFC (request for change) procedure and enacted by the NHS Digital delivery team.
To date, maintenance has included the addition of extra data points to the algorithm (e.g. Vaccine status).

2.8 System architecture

QCovid tool architecture diagram

Tier 2 - Oversight

3.1 Process integration

The use of the tool does not override any clinical decision making but is a supporting device in the decision making process.
The tool amalgamates known factors about Covid at the time the tool was released. The update schedule for the tool isn’t confirmed as it is dependent on scientific understanding of new variants as they emerge.
The tool promotes shared decision making with the patient and is an extra point of information to consider in the decision making process.
The tool helps with risk/benefit analysis around decisions (e.g. recommendation to shield or take other precautionary measures).
The tool is treated, in essence, as a clinical device (class 1) to aid decision making, not to make automated decisions by itself.
As of April 2022, there are no further planned updates to the tool.

3.2 Provided information

Example outputs from the tool are below:

Risk of dying of COVID-19 following a positive PCR test

Absolute risk: 0.0102%

We’d expect that 10 out of 100,000 people with the same characteristics and risk factors as this patient would die of COVID-19 following a positive PCR test.

Relative risk: 1.1

We’d expect that the patient is 1.1 times more likely to die of COVID-19 following a positive PCR test than someone who shares the same age and sex registered at birth as them, but none of their other characteristics and risk factors.

Result generated on 25 February 2022 and valid for 90 days.

Release R1 Version 2.0.5.9292

3.3 Human decisions

The tool is designed to help guide discussions but does not mandate the treatment or next steps agreed for the patient.
The tool only provides one strand of information as guidance. In order to make a decision on vaccination or treatment, many strands of information to determine those who are eligible would be required.

3.4 Required training

The tool is designed for use by clinicians who are reminded to look through clinical guidance before using the tool

3.5 Appeals and review

Patients will first review results with their clinician and for any further clarification can refer to the guidance pages.

Tier 2 - Information on data

4.1 Source data name

Data is entered by the user and is not stored in a database. No external datasets are used/loaded.
Townsend deprivation index is looked up by postcode and fed into algorithm (provided by the team at Oxford University)
Current daily infection rate for England is taken from the UK Coronavirus dashboard.

4.2 Source data

The team used the QResearch database (version 46) of 12 million patients with personal, clinical, and drug data that have been used for clinical and drug safety research. QResearch is linked to multiple datasets at the individual patient level. For this analysis, we used the National Immunisation Database of Covid-19 vaccinations to identify data on vaccine date and doses for all people vaccinated in England. For hospital admissions, we used the linked Hospital Episode Statistics dataset supplemented by the more regularly updated Secondary Users Service data. We also obtained and linked the following datasets: national data for mortality; SARS-CoV-2 infection; systemic anticancer treatment; radiotherapy treatment datasets; and national cancer registry data.

4.3 Source data URL

n/a

4.4 Data collection

No data is collected other than by the clinician at point of use
The algorithm undertakes a calculation on patient data and the result of that calculation is retained by postcode. Patient level data is not retained.
Data is not patient identifiable at the point of input.

Anonymised data is stored in the logs for reporting purposes (i.e. patient data input by clinician – age, gender, risk factors, etc.).

4.6 Data access and storage

Management Information (MI) is retained for 90 days and then a summary version is retained for 8 years. This is only available to approved NHS Digital staff and access is controlled in line with the following governance standards:

SO 13485:2016: Medical devices - Quality management systems - Requirements for regulatory purposes.
DBC0129: Clinical Risk Management: its Application in the Manufacture of Health IT Systems.
DCB0160: Clinical Risk Management: its Application in the Deployment and Use of Health IT Systems
IEC 62304:2006 Amd 1:2015: Medical device software – Software lifecycle processes.

Tier 2 - Risk mitigation and impact assessment

5.1 Impact assessment name

A clinical risk assessment in accordance with mandated Safety Standards DCB0129: Clinical Risk Management and MDD Medical Device Directive 93/32EEC.

5.2 Impact assessment description

Risk Assessment conducted considered Risk Analysis, Risk Evaluation, Risk Control and implementation.
In accordance with DCB0129: a Safety Case Report and Hazard Log describing the hazard(s) scenario, understanding risk impact and suitable mitigation to alleviate risks through design, test, business process.

5.3 Impact assessment date

A risk assessment is an iterative process throughout the lifecycle of the tool. As new software components are introduced, fresh assessments are undertaken, and the Hazard Log is iteratively updated to align with the latest version of the software.

5.4 Impact assessment link

Example outputs from the Hazard log (when the shielding programme was active) are included below:

Hazard: A person who should not be identified as meeting the threshold for higher risk using QCOVID are identified inaccurately
Cause: Information held in GP/hospital records is clinically inaccurate, out-of-date or not recorded, i.e. poor quality of coding.
Effect: Output from COVID-19 Predictive Risk Model misidentifies person as meeting the threshold for higher risk

5.5 Risk name

As part of patient safety risk assessment, Hazardous scenarios are documented, yet haven’t occurred as suitable mitigation is introduced and implemented to alleviate the risk.

5.6 Risk description

Example potential scenarios include:

Hazard - COVID-19 Predictive Risk Model Tool not generated for a Patient when expected.
Hazard - Patient not provided with the most accurate risk score (either excessive or insufficient score) from the algorithm.
Patient does not have an output generated from the COVID-19 Predictive Risk Model

5.7 Risk mitigation

Defined within the Hazard Log. In summary, each hazard identified is reviewed, risk classified and- through design, test, business process or training- the appropriate mitigations are defined and implemented.
Hazard log inputs outlined in 5.4

Published 21 February 2024

Cookies on GOV.UK