Monitoring and evaluation of hepatitis C initiatives

Question 1

What is evaluation

Accepted Answer

Evaluation is the activity that assesses the impact of an intervention or service (delivery) model. The terms of intervention and service delivery model are used interchangeably, but it refers to evaluation of any change in practice, for example outreach testing being undertaken in homeless hostels for the first time.

Evaluation does not need to be a complex academic exercise and should be done as a standard part of project management.

Question 2

Benefits of evaluation

Accepted Answer

Evaluation results can be used to:

clarify objectives of the intervention
demonstrate whether the intervention has achieved what was intended
identify how we can further improve how an intervention is delivered
establish whether an intervention has improved or worsened health inequalities
demonstrate value to senior managers and commissioners (or explain why it didn’t work as intended)
share findings with others planning similar projects

Question 3

Conducting rapid evaluations

Accepted Answer

The below questions set out in table 1 are a useful guide to structure rapid evaluations. The brief example questions provide an indication of the type of questions that can be raised.

Table 1. Rapid evaluation: 9 questions

	Question	Brief example
1	What was the problem you were trying to address?	We aimed to improve HCV treatment uptake among people experiencing homelessness and rough sleeping in the city
2	What caused the problem?	We noted that hospital appointment attendance to commence HCV treatment among this cohort was low
3	Describe the intervention	We offered drop-in HCV clinics rather than set appointments in a city centre hospital outpatient clinic setting
4	What were the reasons for choosing this particular intervention?	This intervention was quick and easy to pilot with minimal resource implications
5	What have you measured to demonstrate the initial problem has been addressed?	See dataset table for examples
6	What were the outcomes?	65% of patients who were homeless or rough sleeping who had previously missed scheduled appointments attended the drop-in clinic; 100% were commenced on treatment during the appointment; 100% of patients attending the drop-in clinic reported it being much easier to attend; clinical staff had mixed views on the clinic model with two thirds seeing it as effective and one third feeling it was not a good use of time
7	Was the intervention delivered as planned?	The intervention was delivered as planned for a 2-week pilot
8	What lessons have been learnt?	Drop-in clinics are more accessible for some people living with HCV who are homeless or rough sleeping but needs to be part of a wider set of outreach interventions to engage this community with care. The wider impact of this model on other groups experiencing health inequalities for example, migrants also needs to be assessed
9	What next?	We intend to pilot drop-in clinics in outreach settings for example, drug and alcohol services, alongside peer supporter provision, and capture data on service access by other vulnerable populations. We will also explore how this service can be sustained in the long term

In order to answer these questions, quantitative and qualitative data needs to be collected.

This should include the HCV intervention minimum dataset, see table 2. This minimum data set is based on requirements for national surveillance and progress towards elimination. Not all variables will be needed for evaluation purposes, but its use is strongly encouraged. For further details on using the minimum dataset and linking up with other nationally collated data, please contact the PHE Hepatitis Surveillance team at phe.hepcdiagnoses@nhs.net

Additional data that will explore other interventions on the HCV care pathway could also be considered; see table 4 for examples. What additional data that is useful to collect will depend on the specific intervention being piloted.

Staff and service users’ surveys to ascertain views on their experience of the service delivery model, its effectiveness and efficiency.

Collecting data on the denominator (for example, total number of people tested or eligible for treatment, total number of people housed in hostel setting) is very useful for calculating indicators such as uptake rate or proportion tested or treated and should be done where possible.

Table 2. HCV minimum data set

Field name	Specifications
Test setting	Use one of the following options:
	Primary care (accident and emergency, drug dependency services, general practitioner, GUM clinic, occupational health, prison services, pharmacy)
	Secondary care (antenatal, fertility services, general medical or surgical departments, obstetrics and gynaecology, other ward type (known service), paediatric services, renal, HIV, specialist infectious disease services, unspecified ward)
	Unknown
Source of laboratory	(if relevant)
Type of test	DBS, blood, oral swab, capillary
Patient identification	At least one of: Patient NHS Number Patient name Patient hospital number
Date of birth	dd/mm/yyyy format
Sex	Female, male
Anti-HCV test date	dd/mm/yyyy format
Anti-HCV result	Positive, negative, equivocal
HCV Antigen test date	dd/mm/yyyy format
HCV Antigen result	Positive, negative, equivocal
HCV RNA test date	dd/mm/yyyy format
HCV RNA result	Positive, negative, equivocal
Genotype
Postcode of testing site
Ethnicity	White Mixed or Multiple ethnic groups Asian or Asian British Black, African, Caribbean or Black British Other ethnic group
Country of birth
Probable route of infection	Injecting drug use, prison, blood transfusion, blood or tissue product, occupational, sexual contact, renal, vertical, household, needlestick, tattoo or piercing, other
Injecting status	Past, Current, Never

Table 3. Additional fields for HIV and hepatitis B testing

Field name	Specifications
HBsAg result	Positive, negative, equivocal
Anti-HBc result	Positive, negative, equivocal
HBV DNA result	Positive, negative, equivocal

If undertaking TB testing, please see guidance from the National TB screening programme on required data items.

Table 4. Additional data items to support a rapid evaluation

Theme	Field name	Specifications
Adequacy of harm reduction in PWID	Reporting adequate needles to meet needs	Yes, no
	Receiving opiate substitution therapy	Yes, no
	Engaged with harm reduction interventions	Yes, no
Awareness of infection	Aware of ever HCV infection	Yes, no, unknown
	Aware of current HCV infection	Yes, no, unknown
Treatment	Treatment status	Past, current, never
	Referred for treatment	Yes, no
	Started treatment	Yes, no
	If treated, outcome	Sustained virologic response (SVR), end of treatment (EOT), lost to follow up, died

An example of an evaluation report that used a variety of data sources to evaluate test and treat interventions targeted at homeless populations that were housed into self-contained emergency accommodation in commercial hotels, bed and breakfast accommodations, and hostels during the pandemic, can be found at COVID-19: evaluation of hepatitis C homeless interventions.

Question 4

Further resources

Accepted Answer

PHE can support service evaluations through facilitating virtual or face-to-face discussions with involved staff and stakeholders. This can be done at any stage in the project, but ideally before rolling out. This will help you to undertake a rapid evaluation that will be useful, inform further service planning, and support improving the quality and effectiveness of HCV services.

PHE facilitators will support teams to complete a structured list of 9 questions (table 1). It is anticipated that the session will run for about 90 minutes. This is a guide only and can be flexed according to need. The aim of the session is to provide time for the team to get together, to share expertise, inform local practice and explore the impact of the intervention collaboratively.

Following the session, PHE will provide a brief report to participants to take it forward.

For more information, please contact phe.hepcdiagnoses@nhs.net

Below is a list of other useful resources regarding evaluation. Some are generic, and some were initially developed for other areas such as sexual health, reproductive health and HIV, but these can all be applied to any service.