Research and analysis

Evaluation Synthesis Report

Updated 17 May 2022

Applies to England

DWP Research Report No. 991

A research report carried out by research consortium led by ICF on behalf of Department for Work and Pensions /Department of Health and Social Care (Employers, Health and Inclusive Employment directorate).

© Crown copyright 2021.

You may re-use this information (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence. To view this licence, visit The National Archives or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: [email protected].

If you would like to know more about DWP research, please email: [email protected]

First published July 2021.

ISBN: 978-1-78659-264-4

Views expressed in this report are not necessarily those of the Department for Work and Pensions or any other government department. 

Acknowledgements

This study was commissioned by the joint Department for Work and Pensions and Department of Health and Social Care Work and Health Unit. We are particularly grateful to Pontus Ljungberg, Sian Moley, Lyndon Clews, Sarah Honeywell, Caroline Floyd, Rachel Shanahan, David Johnson, Mark Langdon, Anna Bee, Craig Lindsay and members of the DWP Policy Psychology Division for their guidance and support throughout the study.

Dr Adam P. Coutts would like to thank the Health Foundation for the three-year fellowship (Grant ID – 1273834) which enabled him to conduct the research. Many thanks to Liz Cairncross at the Health Foundation who provided support and advice throughout the fellowship.

We would also like to thank the Jobcentre Plus staff, Group Leaders, provider representatives and individual benefit claimants who gave their time to participate in the fieldwork.

Views expressed in this report are not necessarily those of the Department for Work and Pensions, Department of Health and Social Care, or any other government department.  

Author’s credits

This report was prepared by Tim Knight, Richard Lloyd and Matt Rayment of ICF Consulting Services Ltd, Dr Susan Purdon and Caroline Bryson of BPSR, Christabel Downing and Siv Svanaes of IFF Research, and Dr Adam Coutts of the University of Cambridge, Professor Stephen McKay of the University of Lincoln, and Dr Clara Mukuria of the University of Sheffield.

Glossary of terms

Active elements	Components and features embedded within an intervention or course, such as the nature of the learning materials, the quality of the Group Leader and whether a course offers or facilitates social support and interaction with other participants. These combine with the background social, economic and psychosocial characteristics of participants which may result in changes in their health, wellbeing and job search behaviour.
Active Labour Market Policy	Active Labour Market Policies (ALMPs) aim to increase the employment opportunities for job seekers and improve matching between jobs (vacancies) and workers (i.e. the unemployed). In so doing ALMPs may contribute to reducing unemployment and benefit receipt via increased rates of employment and economic growth.
Active learning techniques	Active learning techniques are based on actively involving participants in a learning activity rather than just requiring them to passively listen.
Carer’s Allowance	Carer’s Allowance (CA) is the main welfare benefit for carers and was formerly known as the Invalid Care Allowance.
Caseness	A person is described as having suggested case level anxiety or depression if their scores on the Generalised Anxiety Disorder (GAD-7) and Patient Health Questionnaire (PHQ-9) scales suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnoses of anxiety or depression respectively would be based on a clinical interview and would take account of additional evidence, to which the GAD or PHQ scores may contribute.
Cost Benefit Analysis	A cost benefit analysis (CBA) examines all the costs and benefits of the intervention and quantifies them in monetary terms as far as possible, in order to examine the balance of costs and benefits.
Disability Employment Advisor	Disability Employment Advisors (DEAs) are people employed by Jobcentre Plus to support and upskill Work Coaches and other members of jobcentre staff to deliver tailored advisory services to disabled people.
Employment and Support Allowance	Employment and Support Allowance (ESA) is a benefit for people who have an illness, health condition or disability that affects how much they can work. ESA offers financial support if you are unable to work, and personalised help so that you can work if you are able to.
Financial strain	Financial strain refers to when an individual’s financial outgoings start to exceed their income to a degree that psychologically threatens their sense of self, identity, relationships and/or self-esteem.
General self-efficacy	General self-efficacy is the strength of an individual’s belief that they are effective in handling life situations.
Group Leader	Group Leaders are the individuals who delivered the Group Work course, using active learning techniques, to participants.
Group Work	Group Work is a job-search course designed to also enhance self-efficacy, self-esteem and social assertiveness among those looking for paid work. It aims to prevent the potential negative mental health effects of unemployment and help unemployed people back into work. The course, is the application of JOBS II model, originally developed by the University of Michigan, in the UK labour market.
Income Support	Income Support (IS) is an income-related benefit for people who have no income or are on a low income, and who cannot actively seek work. It is mainly for people who cannot seek work due to childcare responsibilities.
Initial Reception Meeting	All Group Work participants were invited to an Initial Reception Meeting (IRM) which preceded the course itself. The IRM was designed as an opportunity for participants to meet the Group Leaders who would deliver their course and learn more about what it would involve.
Intention to Treat	Intention to Treat (ITT) refers to the analysis of the impact of an intervention based on comparing outcomes for all individuals who were offered the opportunity to participate in the intervention with a control group of individuals who were not offered this opportunity.
Jobcentre Plus	Jobcentre Plus (JCP) is an organisation under which the Department for Work and Pensions offers working-age support services, such as employment advisory services. In the context of this report, ‘jobcentre’ refers to the physical premises in which Jobcentre Plus services are offered.
Job-search self-efficacy	Job-search self-efficacy is the strength of an individual’s belief that they have the skills to undertake a range of job-search tasks.
JOBS II	JOBS II is the course originally designed by the University of Michigan, and the Group Work course is the application of JOBS II in the UK.
Jobseeker’s Allowance	Jobseeker’s Allowance (JSA) is an unemployment benefit for people who are actively looking for work.
Latent and Manifest Benefits	Latent and Manifest Benefits (LAMB) are material and psychosocial benefits associated with being in work such as social interaction, social support, activity, identity, collective purpose, self-worth (Latent benefits) and the absence or lessening of financial strain (Manifest).
Learning and Development Officers	Individuals responsible for delivering training that was provided to Work Coaches at the participating Jobcentre Plus offices.
Mastery	The mastery outcome was a composite measure taking into account scores on job search self-efficacy, self-esteem and locus of control indexes. It was designed to be a measure of someone’s emotional and practical ability to cope and take on particular situations.
Mental Health Issue(s)	Mental Health Issue is a broad term that includes those who have: deteriorating mental health (for example, related to the experience of unemployment); elevated but not clinical levels of a symptom; mental health conditions; or are post-treatment; have symptoms but may not recognise they have a condition; or are aware of their condition / situation but choose not to disclose. Many individuals with Mental Health Issues are found to struggle with their job search.
Psychosocial	Psychosocial indicators concern psychological and social factors that can influence health and wellbeing outcomes. Typical examples of such indicators include social support, employment status, job quality, poverty and marital status.
Self-efficacy	Self-efficacy is the strength of an individual’s belief that they have the skills to undertake a task and achieve an outcome.
Single Point of Contact	Single Points of Contact (SPoCs) were the designated point of contact in each of the Jobcentre Plus districts in which Group Work was trialed, involved in monitoring volumes, training and delivery.
Statistically significant	A statistic derived from a study, such as the difference between two groups, is said to be statistically significant if the size of that statistic has only a low probability of arising by chance alone. The probability of a statistic of that size occurring by chance alone is termed the ‘p-value’. By convention, if the p-value is less than 0.05 then it is stated that the statistic is ‘significant’.
Trial Integrity and Support Officers	Trial Integrity and Support Officers were designated DWP staff responsible for monitoring and supporting the fidelity of the DWP input to the Group Work trial.
Universal Credit	Universal Credit (UC) is an in and out of work benefit designed to support people with their living costs. Most new claims by people with a health condition or disability are now made to UC.
Wellbeing	Wellbeing is an individual’s self-report as to whether they feel they have meaning and purpose in their life, and includes their emotions (happiness and anxiety) during a particular period.
Work Coach	Work Coaches are frontline Jobcentre Plus staff based in jobcentres. Their role is to support benefit claimants into work through work-focused interviews.
Work and Health Unit	The Work and Health Unit (WHU) is a joint unit between the Department for Work and Pensions and Department of Health and Social Care. It leads on the Government’s strategy to support working-age disabled people or those with long-term conditions, to access and retain good quality employment.
Zelen design	The Zelen design is randomised control trial methodology in which randomisation is applied before any potential beneficiaries are informed of the possibility of participating in the intervention being trialed. Only those randomised into the experiment group are informed of the possibility of participating in the intervention after randomisation has been applied.

Abbreviations

Abbreviation	Expansion
ALMP	Active Labour Market Policy
CA	Carer’s Allowance
CBA	Cost Benefit Analysis
CV	Curriculum Vitae
DEA	Disability Employment Advisors
DHSC	Department of Health and Social Care
DWP	Department for Work and Pensions
ESA	Employment and Support Allowance
FIOH	Finnish Institute of Occupational Health
GAD	Generalised Anxiety Disorder
GSE	General Self-Efficacy
GW	Group Work/JOBS II
IRM	Initial Reception Meeting
IoP	Impact on Participants
ITT	Intention to Treat
JSA	Jobseekers Allowance
JSSE	Job Search Self-Efficacy
LAMB	Latent and Manifest Benefits
ONS	Office for National Statistics
pp	Percentage Point
PHQ	Patient Health Questionnaire
PIP	Personal Independence Payment
RCT	Randomised Control Trial
SCBA	Social Cost Benefit Analysis
SPOC	Single Point of Contact
UCLA	University of California, Los Angeles
WHO	World Health Organisation

Executive summary

Introduction

ICF, in partnership with IFF Research, Bryson Purdon Social Research, Professor Stephen McKay of the University of Lincoln, Dr Clara Mukuria of the University of Sheffield and Dr Adam Coutts of the University of Cambridge were commissioned by the Department for Work and Pensions (DWP) to undertake a programme of research to evaluate the Group Work trial.

Group Work is the application of the JOBS II model in the UK labour market. It was originally developed in the United States by the University of Michigan, as a preventative intervention aimed at enhancing the job search skills and psychological resilience of individuals experiencing work and career transitions. Group Work is a 20-hour job search skills workshop, comprising five four-hour sessions delivered over the course of a working week designed to also enhance self-efficacy, self-esteem and social assertiveness. Trials of JOBS II in other countries have found evidence of positive work and wellbeing impacts on participants. Following a recommendation by Van Stolk et al (2014) a Randomised Control Trial (RCT) was undertaken between January 2017 and March 2018 to test the effectiveness of JOBS II in a UK labour market context.

This report provides the synthesised findings from the research commissioned to evaluate the Group Work trial, which comprised a process evaluation, impact evaluation, and cost benefit analysis, with the detailed findings from these three elements being provided in technical reports that accompany this report (Knight et al, 2020; Purdon and Bryson, 2020; Rayment et al, 2020). It also draws on findings from a review of previous evidence, and observational and follow-up research conducted by Dr Adam Coutts.

Recognition, uptake and participation in Group Work

The Group Work trial was targeted at claimants of Jobseeker’s Allowance, Employment Support Allowance, Universal Credit Full Service and Income Support (Lone Parents with child(ren) aged three and over) who were struggling with their job search and/or feeling low, anxious or lacking in confidence concerning their job search. Participation was entirely voluntary.

Work Coaches in the five Jobcentre Plus districts where Group Work was trialled were responsible for recognising benefit claimants who could potentially benefit from Group Work. By the end of the trial a total of 16,193 benefit claimants had been recognised by Work Coaches as potential beneficiaries, against the original target of 26,000. The timing of the training that Work Coaches received before the trial and other challenges associated with implementing a trial within the complex environment of local Jobcentre Plus delivery were cited as the main reasons for this shortfall.

The potential beneficiaries recognised by Work Coaches in the trial were also diverse. It included benefit claimants who were potentially struggling with job search, lacking in confidence, anxious and with likely depression but it also included others who did not exhibit these characteristics. In addition, 51 per cent of benefit claimants participating in the course reported never having been in work previously. Some Group Work provider staff felt this potentially diluted the impact of the trial but equally there was a belief that a mix of participant characteristics was a positive for the group dynamic on the course.

A total of 11,900 benefit claimants were randomised into the Group Work group in the trial and offered the opportunity to go on the course. Of these, 5,355 (45 per cent) initially accepted this offer, 4,046 (34 per cent) attended the Initial Reception Meeting that preceded the course, and 2,596 (22 per cent) attended day one of the course itself. The reasons that benefit claimants gave for declining to take part in Group Work when initially offered the opportunity or for subsequently withdrawing centred around health-related factors, childcare responsibilities, and sometimes a perception that either the course would not help them or that they did not need help.

Experiences of the Group Work course

Overall, 92 per cent of participants said they had found the course useful when they were surveyed six months later. The qualitative findings suggested a similar but more nuanced picture, in which participants’ reflections on the course varied from the overwhelmingly positive, to the mildly appreciative and, much less frequently, the negative. The most positive participants typically being those who were struggling with their job search and had been anxious or low on confidence.

The Group Work course was facilitated by two trained Group Leaders who, as found in previous trials in other countries, played an important role in shaping participants experience of the course. Over three-quarters of participants agreed that their Group Leaders understood the challenges of finding work (79 per cent) and had personal experience of being unemployed (76 per cent). In the qualitative research, participants also talked about being treated as an equal by Group Leaders, the participative environment they engendered and their responsiveness to individual needs. The Group Leaders themselves commonly described the course as challenging, but enjoyable and rewarding to deliver, and all described a strong belief in the course design and the theory that underpinned it.

The group dynamic was another feature of the course that participants and Group Leaders emphasised. Around two-thirds of participants agreed that working with a group of others had meant they made more progress (62 per cent), felt more supported (65 per cent), and gave them new ideas (68 per cent). Equally, the qualitative research indicated that how well the group dynamic worked depended partly on their mix of participants on the course together. Too much commonality or too much difference in the group was viewed as potentially unhealthy, with the ideal being a balance between the two, which further emphasised the importance of the participant selection process to effective delivery.

Other factors, including the usefulness and relevance of the content of the course, were viewed positively by the majority of participants, although less central to their overall experiences of it. The participant observation research also identified three “active elements” which may influence changes in the health, wellbeing and job search behaviour of participants:

1. Active participation in a group context - the combination of active participation and the group dynamic established was important, with the balance between Group Leader-led and interactive elements being considered to have worked well.
2. Replicating the time structure and routine of employment – the structure and routine of attending the course was an important element, alongside providing participants with constructive activities and a change to what were often described as monotonous daily routines. 3. Group Leader effectiveness and credibility - Group Leaders was another important element, and they acted as a catalyst for the other active elements of the course.

The impacts of Group Work

The impact evaluation was based on administrative and survey data collected six and 12 months after a baseline survey^{[footnote 1]} of benefit claimants who were offered the opportunity to take part in Group Work and those who were identified as potential beneficiaries but allocated to the control group and received ‘business as usual’ Jobcentre Plus support. As only a proportion of benefit claimants who were offered the opportunity ultimately took part in the course, the impact evaluation findings reported here focus specifically on outcomes for course participants in comparison to a matched sample of benefit claimants in the control group^{[footnote 2]}.

There were no statistically significant differences at the five per cent level between the work status of Group Work participants and the matched comparison group at the six and 12-month points. However positive but not statistically significant change was observed across a range of work-related outcomes, with 23 per cent of participants being in paid work compared to 20 per cent of the control group at 12 months (20 per cent vs 18 per cent at six months).

Despite the lack of statistically significant impacts on work status after 12 months, Group Work appeared to equip participants with certain attributes and behaviours that would increase their likelihood of finding work in the future. After 12 months, 57 per cent had higher levels of job search self-efficacy^{[footnote 3]} compared to 45 per cent of the matched comparison group (58 per cent vs 36 per cent at six months), and they were more likely to be submitting a high volume of CVs to employers at the six and 12 month points. Both these differences were statistically significant.

Statistically significant impacts were detected on well-being outcomes of life satisfaction, happiness and loneliness after six months. After 12 months, impacts on life satisfaction and loneliness were no longer statistically significant (largely due to improvements in the matched comparison group) but the impact on happiness still was, with participants having a mean score of 6.5 compared to 5.8 among the matched comparison group.

There was a similar pattern in terms of the impact of Group Work on mental health. After six months, participants were statistically significantly less likely than the matched comparison group to have likely depression or poor wellbeing on the WHO-5 well-being scale (49 per cent compared to 59 per cent). After 12 months the difference was smaller (50 per cent compared to 55 per cent) and not statistically significant. Participants scored better on other mental health indicators (the Generalised Anxiety Disorder Assessment (GAD-7) and Patient Health Questionnaire (PHQ-9) scales) but these differences were not statistically significant after either six or 12 months. These impacts did not translate into effects on people’s health as measured by the EQ-5D scale or on people’s use of health services.

The impact evaluation also included extensive subgroup analysis to examine the impacts of Group Work on participants with different baseline characteristics. Broadly in line with the international evidence, Group Work had the greatest impact on participants:

• With lower levels of general self-efficacy at baseline^{[footnote 4]};
• With suggested case level anxiety on the GAD-7 scale at baseline^{[footnote 5]}; and
• With suggested case level depression on the PHQ-9 scale^{[footnote 6]} at baseline.

For participants with lower levels of general self-efficacy or suggested case level anxiety, there were statistically significant positive impacts on people’s participation in paid work. After six months, those with lower levels of general self-efficacy or suggested case level anxiety were around twice as likely to be in paid work as their matched comparison groups, with the proportions in paid work of 30 hours or more continuing to be significant after 12 months.

There were also significant positive impacts – at both six and 12 months - on these groups’ levels of general and job-search self-efficacy where, as with paid work, those with lower levels of general self-efficacy or suggested case level anxiety were twice as likely as their matched comparison groups after six months to score as having higher levels of general and job search self-efficacy. For those with lower levels of general self-efficacy and suggested case level anxiety, Group Work also appears to have a significant impact on levels of mental health (as measured by the GAD-7 and World Health Organisation (WHO-5) scales) after six months, sustained after 12 months for those with lower levels of general self-efficacy but not for those with suggested case level anxiety. A similar, but more limited pattern of statistically significant impacts was detected among those with suggested case level depression at baseline.

The costs and benefits of Group Work

The total costs of Group Work were estimated at £3.3 million, of which £1.7 million was spent on delivering the intervention, and £1.6 million on running the trial and evaluation. After deducting the trial and evaluation elements, the delivery costs averaged £656 per participant beginning the course and £886 per participant completing it.

The cost benefit analysis found that, overall, the costs of delivering Group Work outweighed the value of the monetised benefits, even employing best case assumptions for employment effects and costs. The societal benefit cost ratio ranged from 0 (main assessment) to 0.67 (assuming a positive employment effect and lower cost estimate). However, this analysis does not reflect the statistically significant benefits to metal health, wellbeing and job-search outcomes detected in the impact evaluation, which could not be valued in monetary terms.

The cost benefit analysis also assessed the benefits and costs of Group Work for different subgroups of participants. The monetised benefits were found to exceed the intervention costs for two groups, with a benefit cost ratio of 1.76 for the subgroup with suggested case level anxiety at baseline, and 1.39 for the subgroup with lower levels of general self-efficacy at baseline (main estimates). The sensitivity analysis estimated that benefits would exceed costs even if the assumed duration of employment effects was reduced and a higher estimate of programme costs used.

This suggests that the intervention could deliver net benefits to society if targeted at these groups. This finding relies on the assumption that the costs of delivery to these groups can be held at the average intervention cost of Group Work, and that similar levels of benefits can be maintained by participants drawn from benefit claimants whose characteristics suggests they are at greatest distance from the labour market.  

Conclusions

On the basis of the findings of the impact evaluation and the cost benefit analysis, the evaluation concluded that for Group Work to be most effective it requires being more tightly targeted, and specifically with benefit claimants with lower levels of general self-efficacy and poorer mental health, for whom several statistically significant impacts and positive benefit cost ratios were detected after 12 months.

Lessons learnt from the Group Work trial include the following:

1. If Group Work was trialled again, consideration should be given to targeting the sub-groups found to benefit most (those with low levels of general self-efficacy, and/or higher levels of anxiety, for whom the course was shown to be cost effective), building on lessons from this evaluation.
2. If Group Work is implemented in the future, consider how Work Coaches or others can be trained and helped to accurately recognise benefit claimants whose behaviour and psychological resources indicate they are likely to benefit most.
3. Recognition, recruitment and other processes associated with the intervention should be monitored closely if Group Work was implemented, to ensure the eligibility criteria are being adhered to and the appropriate benefit claimants are recruited.
4. If Group Work was adopted in the future, it should seek to replicate the structure and delivery of the Group Work trial, which was found to be effective, and ensure it continues to be delivered by suitably trained and experienced facilitators.
5. Explore how each of the ‘active elements’ of Group Work identified in the trial may be applicable to wider provision.
6. Consider the inclusion of mental health, wellbeing and self-efficacy measures in other interventions and provision as indicators of the intervention’s effectiveness, in addition to the routine measures such as rates of job entry.

1. Introduction

ICF, in partnership with IFF Research, Bryson Purdon Social Research, Professor Steve McKay of the University of Lincoln and Doctor Clara Mukuria of the University of Sheffield, were commissioned by the Department for Work and Pensions in January 2017 to undertake a programme of research to evaluate the Group Work trial, which is the UK version of the JOBS II programme. Dr Adam Coutts of the University of Cambridge provided background empirical evidence from the international literature on the links between Active Labour Market Policies (ALMPs), health and wellbeing, and findings from observational research of the Group Work intervention.

This report provides a synthesis of the research findings, with additional information and evidence from its three main elements being provided as technical annexes, namely:

• A process evaluation technical report;
• An impact evaluation technical report; and
• A cost benefit analysis (CBA) technical report.

1.1 Group Work/JOBS II

Group Work is the application of the JOBS II model in the UK labour market. Developed by the Michigan Prevention Research Center at the University of Michigan, U.S., it is a group-based course delivered in five half-day sessions (20-hours) over the course of one working week. The content of the JOBS II course has the objective of developing participants job-search skills, the motivation to apply them and the resources to cope with any setbacks they experience in job search. While the content of the course focused on job search skills, the underlying processes, facilitated by Group Leaders, are also intended to enhance self-efficacy, self-esteem and social assertiveness of the participants. Evidence from international trials has shown JOBS II to have both work and wellbeing impacts.

Between January 2017 and March 2018, the Department for Work and Pensions (DWP) and Department of Health and Social Care (DHSC) Joint Work and Health Unit conducted a Randomised Control Trial (RCT) to test the effectiveness of JOBS II in a UK labour market context, targeting individuals in receipt of Jobseeker’s Allowance (JSA), Employment Support Allowance (ESA), Universal Credit Full Service (UC) and Income Support (IS - lone parents with child(ren) aged three and over) who were struggling with, and/or feeling low or anxious and lacking in confidence regarding their job search.

JOBS II is one of several interventions being trialled by the Joint Work and Health Unit to build a strong evidence base on what interventions work best to help those with health issues move into or retain work. The trial in the UK is also intended to contribute to the wider international evidence base on JOBS II.

The trial operated in five Jobcentre Plus districts across England and was delivered by two third party providers. Work Coaches were responsible for recognising potential beneficiaries, who were then either randomised into the treatment group and offered the opportunity to go on the course or into a control group. Participation in the course was entirely voluntary, with 5,355 individuals initially accepting the offer of attending the Group Work course and 2,596 benefit claimants attended it during the trial period^{[footnote 7]}.

Key characteristics of those who started the course included^{[footnote 8]}:

• Gender – 63 per cent were male;
• Age - 66 per cent were aged between 35 and 59, and 27 per cent aged 16 to 34;
• Qualifications – 41 per cent reported achieving grade C or above in both Maths and English at GCSE level;
• Benefit receipt – 82 per cent were claiming Jobseeker’s Allowance, 12 per cent Universal Credit, and six per cent Employment Support Allowance, Disability Living Allowance or Carer’s Allowance. Within the last three years, 35 per cent of starters had been on benefits for over two years, and 30 per cent for six months or less;
• Work history – nine per cent were last in work in the previous six months, with 21 per cent last being in work more than two years ago and 51 per cent had never been in paid work.

In addition, the participants showed a degree of diversity across a range of job search, confidence, wellbeing and mental health indicators^{[footnote 9]}, including being fairly evenly distributed between those confident and not confident of finding work (51 per cent being confident and 49 per cent not confident), and between those whose GAD-7 (anxiety) and PHQ-9 (depression) scores suggested that they had and had not reached caseness^{[footnote 10]} (49 per cent suggesting caseness versus 51 per cent not for GAD-7, and 45 per cent suggesting caseness versus 55 per cent for PHQ-9).

As described in subsequent sections of the report, those starting the Group Work course had different characteristics to those who participated in previous international trials, which has implications for the ability to directly compare the findings from each trial (see Chapter 2 in this report and Chapter 2 of the Process Evaluation Technical Report for more detail on the other international trials).

1.2 Research aims and objectives

The overall aim of the research, as set out in the Statement of Requirements, was to examine: “What works to improve employment and health outcomes for people who are out of work and struggling with their job search?”

The primary research questions to be addressed included:

• Did Group Work improve benefit claimants’ employment rates and wellbeing?
• For whom is this support most effective and why?
• Is the support cost effective?
• Was the Group Work delivered as intended, and what worked well and less well in its delivery?
• What were the experiences of provider staff delivering the course and Work Coaches making referrals to it?
• What were the experiences of benefit claimants participating in the course, the reasons why some declined to attend or did not complete the course?

1.3 Overview of research methodology

A mixed methods approach was followed, details of which are provided in each of the technical reports. In summary, the approach comprised three strands of activity:

A process evaluation: to assess how the trial processes operated, capture the experiences of course providers and participants, and identify what worked well and less well, the overall utility of Group Work and the active elements of the intervention which may lead to behavioural and health changes. The process evaluation comprised a programme of interviews with Jobcentre Plus staff (n=45), provider staff (n=15) and benefit claimants (n=80, comprising course completers, those exiting early and those declining the offer) across the five trial districts. Interviews were also undertaken with compliance managers (n=3) and the Jobcentre Plus leads in each of the five districts. This fieldwork was undertaken by ICF between July 2017 and January 2018.

Data from a survey of participants, conducted on Days 1 and 5 of the course, was also analysed to identify short-term changes in outcomes reported by participants during the course.

Observational research: a programme of participant observational and follow-up research was also conducted by Dr Adam Coutts between July 2016 and April 2019. This allowed Dr Coutts to observe the whole participant journey from initial attendance at work focused interviews within jobcentres, reception interviews with Group Leaders, to participation in the course and after completion. The participant observation research involved:

• Semi-structured interviews with DWP staff, single points of contact (SPOCs), Work Coaches and jobcentre managers (n=40), and Group Work Group Leaders (n=14);
• Observations of 17 courses across the five Districts between April 2017 and April 2019 (representing over 300 hours of direct course observations); and
• Interviews with 100 course participants (20 per District) at various points between Day 1 to 5 of the course and post-completion.

Post Group Work experiences were explored through semi-structured interviews with a cohort of 25 participants from across the trial sites. They were contacted at one week, one month, three months, six months and twelve months after completing Group Work. This amounted to 125 follow-up interviews conducted between April 2017 and April 2019. Finally, a one-day workshop took place with all the Group Work leaders (nine) and DWP policy psychologists (two) in June 2018, which allowed observations and research insights to be fed back and further validated.

An impact evaluation: a Zelen-design RCT was used to measure the impact of the Group Work course on a range of work, job search, well-being, health and mental health outcomes, and to identify whether the course works better for some population groups than others. Under the Zelen design, eligible benefit claimants were randomised into either an ‘offered Group Work’ or control arm, which meant that only those offered Group Work were informed they were part of the trial and given the option of accepting or declining the intervention. Those in the control group were not offered Group Work and were instead offered the standard range of interventions or support through Jobcentre Plus.

Initially an Intention to Treat (ITT) approach to the analysis was taken, which compared outcomes at six and 12-months after the baseline survey for all those offered the course to those in the control group. However, only 22 per cent of those offered the course attended it, severely limiting the ability to detect a significant impact amongst all those randomised. Consequently, much of the analysis focused on estimating the impacts on those who participated in the course against a matched comparison group selected from the control group (an Impact on Participants (IoP) analysis using propensity score matching). This report concentrates on the analysis of those who attended the course (IoP analysis). More information about the ITT and the IoP approach is included in the Technical Report on the Impacts of the Trial.

The impact analysis used baseline data collected at the point of randomisation (in a survey administrated by Work Coaches) and as part of a baseline survey (administered by Group Leaders on the first day of the course and by the evaluation team with random samples of the control group and those declining the course). Additional follow-up data was collected from participants, decliners and members of the control group at the six and 12-month points after baseline by telephone survey. DWP administrative data was also analysed for the whole trial population on the nature, duration and monetary value of benefits claimed at randomisation and six and 12 months after randomisation. The data collected is described in detail in the Technical Report on the Impacts of the Trial.

Figure 2.1 provides an overview of the trial population, with 16,193 benefit claimants being recognised as potential beneficiaries, 11,900 randomised into the treatment group (so offered Group Work) and 4,293 into the control group. Of those offered Group Work, 5,355 accepted and 6,545 declined, with 2,596 attending at least Day 1 of the course and 1,922 completing it.

A cost benefit analysis (CBA): a CBA framework was developed which structured the analysis, identified the range of costs and benefits to be considered and an approach to assessing and valuing them. Costs included expenditure associated with course delivery, DWP staffing and administrative costs, and Jobcentre Plus staff costs. Benefits considered included those for participants (net monetary benefits for those finding work, direct benefits to mental health from attending the course, and indirect health and wellbeing benefits from changes in employment status); savings to the Exchequer; and benefits to employers and the wider economy through enhanced productivity. The CBA examined benefits for participants as measured by the IoP impact analysis. Expenditure data was provided by DWP.

Some outcomes, such as mental health, wellbeing and job search related outcomes (e.g. related to self-efficacy), could not be monetised so are not accounted for in the calculations. As a result, the CBA may underestimate the value of the benefits of the intervention

Box 1.1 provides an overview of the key outcome measures that were used in the research to quantitatively assess the impacts of Group Work, with more detailed descriptions of these measures and their use in the research being available in the Technical Report on the Impacts of the Trial.

Box 1.1: Key measures used in research

Work-related measures

• Currently being in paid work.
• Currently being in paid work of 30 or more hours a week (i.e. in full-time work).
• Currently being in paid work that they are satisfied with.
• Currently earning above or below £10,000 per annum.
• Benefit receipt and value of benefit payments.

Job search-related measures

• FIOH (Finnish Institute of Occupational Health) Job Seeking Activity Scale Revised: A measure of how frequently individuals undertake job search activities and, following revisions, includes two items that measure number of vacancies applied for and number of CVs submitted in past two weeks.
• Perceived ability to influence their propensity to find work, the extent to which personal qualities make it easy to get a job, and the demand for their experience.
• Gaining relevant skills or experience: Measured by whether someone has (a) attended training or courses, (b) done voluntary work, or (c) attended work placements in the previous six months.
• JSSE (Job Search Self-Efficacy) Index - Modified: A measure of the strength of an individual’s belief that they have the skills to undertake a range of job search tasks.
• GSE (General Self-Efficacy) Scale: A broader measure of the strength of an individual’s belief that they are effective in handling life situations.
• Confidence in finding a job: A measure of how confident individuals are of finding a job within 13 weeks.

Wellbeing and mental health measures

• WHO-5 (World Health Organisation) Wellbeing Index: A measure of an individual’s wellbeing based on particular feelings experienced in the last two weeks. The WHO-5 can also be used to indicate likely depression.
• ONS 4 (Office for National Statistics) Subjective Wellbeing: Four related items measuring an individual’s wellbeing based on their subjective happiness, life satisfaction, feeling that life is worthwhile, and anxiety.
• UCLA Loneliness Scale: A measure of an individual’s loneliness.
• LAMB (Latent and Manifest Benefits): A measure of benefits associated with work that can also be used to measure psychosocial deprivation and financial strain.
• PHQ-9 (Patient Health Questionnaire): A measure designed to facilitate the recognition of the most common mental disorders, notably depression.
• GAD-7 (General Anxiety Disorder) Assessment: A measure designed primarily to facilitate the recognition of generalised anxiety. Wider health measures
• EQ5D-3L (EuroQol Group): A standardised measure of an individual’s overall health.
• EQVAS (EuroQol Group): A measure of an individual’s subjective overall health on that day.
• Visits to a GP in the last two weeks and use of Casualty and Outpatients services in the past three months were also used as measures of overall health.

1.4 Headline findings

The impact evaluation found that for those participating in the course, Group Work works better for some groups of benefit claimants than for others, with the greatest impacts being detected for those with lower general self-efficacy, and suggested case level anxiety (on the GAD-7 scale) and depression (on the PHQ-9 scale) at baseline. Six months after baseline, statistically significant positive impacts were detected for those with lower general self-efficacy and suggested case level anxiety at baseline in terms of being in paid work, general and job search self-efficacy and mental health. For both sub-groups, impacts on work and self-efficacy were sustained at the 12-month point, with positive mental health outcomes sustained for those with lower levels of baseline general self-efficacy. There is a similar, but not so pronounced, pattern of statistically significant impacts among those with suggested case level depression at baseline. These findings were reflected in the qualitative process and observational research, which found that individuals participating in the course reported a range of positive benefits during or shortly after participation, including perceived improvements in confidence, motivation, mental health and wellbeing and enhanced job search behaviour, and are broadly in line with the international evidence. This indicates that positive wellbeing and mental health outcomes can be achieved outside of a clinical setting by an employment intervention.

Looking across all course participants, no statistically significant evidence was found of Group Work having an impact on participants’ participation in paid work. However, there are statistically significant positive impacts six months after baseline across a range of mental health, wellbeing and self-efficacy measures, as well as on confidence in finding paid work – and although for the most part these are no longer detected at 12 months post-baseline these non-significant differences between participants and the matched comparison group are still positive. Importantly, there is little evidence of any negative impacts resulting for individuals attending a Group Work course.

The impact findings were reflected in the cost benefit analysis, which found that for all participants the costs of delivering Group Work outweighed the benefits that could be valued in monetary terms^{[footnote 11]}. However, when analysed at the level of the subgroups found to benefit most, the reverse was true, with cost benefit ratios of 1.76 for those with suggested case level anxiety and 1.39 for those with lower levels of general self-efficacy at baseline.

The process evaluation found that overall the trial and the Group Work course were broadly delivered as intended, although as the trial progressed people less confident of finding work or who felt they were struggling with their job search made up a smaller proportion of those offered the course.

Most of the participants interviewed clearly enjoyed attending, and along with Group Leaders delivering, the course, saw it as something different to, and better than, other employability provision they had previously experienced – with both finding the course to be positive in terms of both learning content and facilitating group interaction compared to other employability provision. Group Leaders commonly described the course as challenging, but ultimately rewarding, to deliver.

In addition to good course content, three active elements^{[footnote 12]} were identified in the course observations which may lead to changes in health, wellbeing and job search behaviour of participants:

• Active participation in a group context which provided social support and reduced feelings of isolation;
• Replicating the time structure and routine experience of employment; and
• The Group Leader quality and credibility.

Overall, the combination of these elements influenced feelings of stability, control and confidence in participants which could enable changes in mental health, wellbeing and behaviour. These elements could potentially be applied to other DWP and Jobcentre Plus employability provision.

1.5 The structure of this report

The remainder of this report is structured as follows:

• Chapter 2 provides the policy context for, and an overview of, the Group Work trial;
• Chapter 3 reports on recognition, uptake and participation in Group Work;
• Chapter 4 provides findings from the process evaluation on the delivery of the Group Work course;
• Chapter 5 presents the findings from the impact evaluation;
• Chapter 6 presents the findings from the cost benefit analysis; and
• Chapter 7 provides our conclusions and recommendations.

2. Policy context and the Group Work/JOBS II trial

This Chapter summarises the policy context for the Group Work/JOBS II trial, evidence from international studies, and describes the course as implemented in the trial.

2.1 Policy context and evidence base

Unemployment and mental health are at the forefront of the policy agenda in the UK. Since 2013 the Department for Work and Pensions (DWP) and the Department of Health and Social Care (DHSC) have been working jointly to explore more integrated policy approaches and interventions to address the links between the two. This has included identifying and trialling evidence-based interventions which can help people return to work and improve wellbeing and mental health.

Research evidence shows that mental health issues are prevalent in the UK, with around one in six people having a common mental health condition such as anxiety or depression at any given time (McManus et al 2016). People with mental health issues are also shown to fare worse in terms of labour market participation:

• The Annual Population Survey (April 2018 to March 2019) showed the employment rate of disabled people aged 16 to 64 who had any mental health condition as their main health condition was 43.6 per cent, compared to 51.4 per cent of all disabled people and 81 per cent of non-disabled population^{[footnote 13]} ; and
• Quarterly data to February 2020 shows that 50 per cent of Employment and Support Allowance claimants have a mental and behavioural disorder as their primary registered condition^{[footnote 14]} .

At the same time, research evidence shows that unemployed individuals are more likely to experience common mental health conditions than those in work (Kim and Von dem Knesebeck 2015). These conditions may in themselves hinder individuals’ ability to return to the labour market after a period of unemployment, where poor mental health and low self-esteem have been found to influence job search behaviour, motivation and confidence (Waters and Moore 2002). Conversely, appropriate and good quality employment has been shown to counter some of the negative wellbeing and mental health effects of unemployment (Coutts 2009, Coutts et al 2014, McKee-Ryan et al 2005).

However, the evidence is unclear concerning the health and behavioural impacts of the transition between unemployment and employment resulting from participation in employability provision, why these impacts occur, and who is most responsive in terms of improved mental health, wellbeing and job outcomes (Coutts 2009, Coutts et al 2014). Most studies which examine the impacts of Active Labour Market Policies (ALMPs)^{[footnote 15]} and the employability provision interventions used to deliver them, focus on what are traditionally considered the more tangible economic impacts or outcomes such as job entry rates and time off benefits.

While evidence is limited in the UK, a small body of international evidence suggests that participation in employability provision can have positive effects on mental health and wellbeing^{[footnote 16]}. More detail on the processes and active elements by which these positive effects are generated are detailed in the Process Evaluation Technical Report.

2.1.1 The international evidence base

The JOBS II approach has been used internationally for over 20 years, with previous evaluations showing positive impacts on health, wellbeing and job outcomes in a range of contexts. An overview of two key Randomised Control Trial-based evaluations in the United States (US) and Finland (Vinokur et al in the US (2000) and Vuori et al in Finland (2002)), along with the UK trial, is provided in this section, and their designs are summarised in Table 2.1.

Table 2.1: Summary of the trial designs in the United States, Finland and the UK.

	United States Trial	Finnish Trial	Group Work Trial (UK)
Eligibility	Unemployed for less than 13 weeks	Unemployed or had received termination notice. No criteria set in terms of unemployment duration	Benefit claimants struggling with job search. No criteria set in terms of unemployment duration
Recruitment and random allocation	Trial participants initially recruited by interviewers. Those interested were asked to complete a screening questionnaire. Only those screened in were randomly allocated.	Potential participants were contacted about the trial. Only those expressing interest were randomly allocated.	Zelen design. All those identified as eligible were included in the trial and randomly allocated. Those allocated to the intervention arm were then invited to take up the course.
Number Randomized	1,801	1,261	16,193
Take-up of the programme	54%	70%	22%
Range of outcomes collected	Employment, financial strain; assertiveness; role and emotional functioning; job search self-efficacy; self-esteem; internal control orientation; mastery, depression; distress symptoms.	Employment, wage rate, job stability, job satisfaction, job-search intensity, psychological distress, and depressive symptoms	Employment; job-search activity; general self-efficacy; job-search self-efficacy; latent and manifest benefits; well-being; depression; anxiety; and overall health.

While the content of the intervention did not differ significantly from the UK version of Group Work/JOBS II, the design of the US and Finnish trials and the characteristics of their participants means their findings need to be considered in light of the nature of the UK sample population and some of the measures used.

The US trial (Vinokur, Price and Schul, 1995)^{[footnote 17]} focused on the short-term unemployed – that is those unemployed for less than 13 weeks), who were recruited by trained interviewers in unemployment offices. Those meeting basic eligibility criteria were told about the programme and asked to complete a screening questionnaire, and if judged eligible were randomly allocated to a treatment or control group. The trial was designed to test whether the course was more, or less, effective for those at high risk of depression (relative to low risk), and actively over-represented those at high risk (with a proportion of those at low risk being screened out prior to randomisation).

The characteristics of those starting the course in the US trial were different to those of the UK trial (see Table 3.1), particularly in terms of their duration of unemployment (where US starters were unemployed for less than 13 weeks, compared to the UK where nine per cent were last in work six months prior to the course, and 21 per cent who were out of work for more than two years) and wider work histories (51 per cent of starters in the UK trial had never been in work). The US trial also had a much higher take-up rate than the UK trial (54 per cent compared to 22 per cent), which may reflect the fact that the trial recruited only those recently unemployed, and the UK trial was implemented in a live policy setting competing with other research trials and labour market policies such as the national roll out of Universal Credit. It may also be that the recruitment process in the US, where the course was introduced to potential participants prior to randomisation, led to the exclusion of many benefit claimants who were simply not interested in participation.

A similar range of outcomes were studied to those of the UK trial, although the questionnaire measures were different using more up-to-date measures for mental health, wellbeing and job search outcomes.

The key findings from the US trial at six-months were that:

• The treatment group had significantly higher mastery scores^{[footnote 18]} than the control group;
• Those at high risk of depression were significantly more likely to be in work if they were in the treatment rather than the control group (around 10 percentage points), although there was no significant impact on employment for those at mild risk of depression;
• The programme had a positive impact on measures of depression for those at high risk of depression, but no impact on those at mild risk of depression.

The key findings from the US trial at two years were that the treatment group had higher levels of employment than the control (as well as working a greater number of hours and earning greater levels of income - although there was no improvement in the stability of jobs obtained). They were also significantly less likely to have experienced a major depressive episode than the control group.

In Finland (Vuori, Silvonen, Vinokur and Price 2002)^{[footnote 19]} the JOBS II programme was also tested using a RCT, which recruited people from a longer-term unemployed population than the US trial and so was closer in that respect to the UK JOBS II trial, although Finnish participants overall were better educated, more likely to be female, and the trial was directed towards office workers. It was supported by the Ministry of Labour and the country-wide national network of employment offices.

The recruitment process for the Finland trial was different to that in the UK, with potential participants being contacted and informed of the trial, with only those interested in taking part, agreeing to randomisation, and completing a baseline assessment being included. This generated a much higher take-up rate amongst those allocated to the treatment group (70 per cent) than in the UK trial. This recruitment approach provides a trial of JOBS II for a group of people who believe that the programme will benefit them and so are willing to engage. This means that the impacts from such a trial are unlikely to be replicated in a trial with a broader population, such as the UK.

The Finish trial collected a similar range of outcomes to the UK although the actual scales used were different, so as with the US trial, direct comparisons of measures with the UK trial are generally not possible. The Finnish trial found at six-months:

• There was no statistically significant impact on reemployment, but a positive significant impact on stable employment^{[footnote 20]} (greatest for those unemployed for three to 12 months at baseline). There was no statistically significant impact on those unemployed for longer than 12 months;
• No statistically significant impacts on wages or job satisfaction were found; and
• There was a statistically significant positive impact on psychological distress^{[footnote 21]} , greatest for those at the most risk of depression at baseline. No statistically significant impact was detected on depressive symptoms.
  At two years, the intervention participants continued to be more likely to be in stable employment and continued to experience reduced levels of psychological distress than the control group. The reasons for the observed effectiveness of JOBS II was explored in research by Price and Vinokur (2014), who identified three main components influencing participants’ experiences of the intervention:
• The social and psychological components/active elements of the course, such as teaching of job search skills, networking and motivational training;
• Participants’ initial levels of motivation and job search intensity; and
• Initial levels of financial strain, depression and emotional and social functioning.

The research suggested that the intervention operates via influencing and strengthening individual factors such as self-efficacy and motivation, which interact with the group context and social support. A series of ‘enabling factors’ for the effective delivery of the course were also identified, including establishing an atmosphere of trust and an open environment for sharing ideas, encouraging and modelling more effective behaviours, an emphasis on support and the absence of criticism (Coutts et al, 2014).

2.2 Overview of the Group Work trial

2.2.1 Background to the trial

DWP and DHSC jointly commissioned research (Van Stolk et al 2014) to further explore how employment and health prospects can be improved for individuals with common mental health conditions. The research found that the complex interaction between mental health and work require tailored approaches, and called for timely access to, and the greater integration of, treatment and employment support services and the use of evidence-based models of support. The research proposed four models for further investigation, three of which led to small-scale feasibility pilots to examine the most effective intervention designs and delivery modes.

Group Work was initially piloted in two Jobcentre Plus districts between August and December 2014, with the subsequent evaluation (Callanan et al 2015) finding evidence of mental health and wellbeing impacts, and recommending the scaling up of the intervention to test its effectiveness in different labour market contexts. The evaluation also recommended that further implementation would benefit from a clearer definition of the target group for the intervention, enhanced Work Coach understanding of who the intervention is most suitable for, and some changes to the terminology used in the course materials.

Following the initial piloting, the Group Work trial was established to test and evaluate the JOBS II model on a larger scale in the UK labour market context, and so develop the evidence base of what interventions work best to help unemployed job seekers with (or at risk of) mental health issues move into work.

2.2.2 The Group Work course in the UK

In comparison to previous JOBS II trials which consisted of specific population groups, the UK version was set within a routine and live labour market setting. It was targeted at benefit claimants who were struggling with their job search and/or feeling low or anxious and lacking in confidence in relation to their job search, and receiving Jobseeker’s Allowance (JSA), Universal Credit (UC), Employment Support Allowance (ESA) or Income Support (Lone Parents with child(ren) 3years+).

The trial started in January 2017, with the expectation of supporting approximately 3,000 starts in the 12 months of its duration. In late 2017, the participation criteria were broadened to include Universal Credit (UC) Live Services claimants, and the trial period extended to the end of March 2018 to help ensure sufficient participant numbers.

The trial operated in five Jobcentre Plus districts, including 50 jobcentres, and the Group Work course was delivered by two providers: one covering the Durham and Tees and Merseyside districts; and the other the Midland Shires, Mercia and Avon, and Severn and Thames districts.

Work coaches were responsible for recognising potential beneficiaries of the course based on the criteria above. Having recognised a potential beneficiary, they asked the benefit claimant to complete an onscreen survey (the Work Coach would also help them to complete this if necessary), and then randomisation was applied. Potential beneficiaries were randomised into either the treatment or control group. The responses the benefit claimant had given in the survey had no bearing on this allocation.

Under this Zelen design, Work Coaches were instructed to only mention the course to benefit claimants allocated to the treatment group after randomisation. If those offered the opportunity to go on the course accepted, a referral was made to attend an Initial Reception Meeting (IRM) which preceded the course. Those randomised into the control group received Jobcentre Plus ‘business as usual’ support. Participation in Group Work was on an entirely voluntary basis, with no sanctions (reductions in benefit amounts if certain mandatory provision is refused) being applied if benefit claimants decided not to attend or withdrew part way through the course. The completion of the onscreen survey, and the randomisation process, were required for trial purposes and would not be needed if the course was rolled out more widely.

Figure 2.1 provides an overview of the trial population, in terms of the total number of benefit claimants recognised as potential beneficiaries, how they were allocated between control and treatment groups, and how many attended day one of the course.

Figure 2.1: Overview of the Group Work trial population

1. Recognised as potential beneficiary and randomisation applied (n=16,193) 1.1. Allocated to control group (n=4,293) or 1.2. Allocated to treatment group and offered opportunity to participate (n=11,900) 1.2.1. Initially accepted offer to participate in Group Work (n=5,355) 1.2.1.1. Attended Initial Reception Meeting (n= 4,046) 1.2.1.2. Started Group Work course, atttended Day 1 (n=2,596) 1.2.1.3. Completed the course (n=1,922) 1.2.2. Declined opportunity to participate in Group Work (n=6,545)

2.2.3 Course design and delivery

The Group Work course delivered in the trial was the application of the JOBS II model. The detailed manual for course trainers was updated for the UK by DWP policy psychologists who played a key role in preparing for the delivery of the course (including training for DWP staff) and in ensuring fidelity to the model (see Meehan et al., 2015). Although the course content focused on job search skills, the processes by which it was delivered were intended to enhance participants’ self-efficacy, self-esteem and social assertiveness with the objective of developing participants job-search skills, the motivation to apply them and the resources to cope with any job search setbacks, and in so doing help improve individual mental health and wellbeing.

The course was led by trained facilitators (known as Group Leaders) employed by the contracted providers, who were required to receive seven weeks of training to fulfil their role and used active learning techniques and social support to encourage participation. Benefit claimants who agreed to attend the course were invited to attend an IRM, to meet the Group Leaders and other participants and find out more about the course. Both the IRM and the full course were delivered at non-jobcentre venues.

The course was delivered through five half-day modules, averaging four hours a day, over the course of a working week, with the modules covering:

1. Discovering your job skills; 2. Dealing with obstacles to employment; 3. Finding job openings; 4. CVs, contacts and interviewing; and 5. Full interview rehearsal and planning for setbacks.

The job skills content of the course was partly used as a vehicle for achieving its more fundamental aims of increasing participants’ self-efficacy, self-esteem and social assertiveness. This is reflected in the “essential components” of the course, as described in the UK edition of the JOBS II manual:

• Job-search skill training - participants are invited to acquire and rehearse job-search skills in a safe and supportive environment.
• Active teaching/learning methods - the learning process is almost entirely active, with an emphasis on engagement with the course content, participation by all participants, and collaborative working through small and large group discussions, brainstorming and other activities.
• Inoculation against setbacks - the process involves identification of specific problems, generation of possible behavioural or cognitive responses, evaluation of responses, skill acquisition, behaviour rehearsal, try-out and re-evaluation.
• Trainer referent power - Referent power is a term that describes a source of empowering influence. When a participant in the Group Work programme sees and believes that a trainer can be trusted to enhance their self-esteem no matter what, that trainer will be able to play a key role in promoting positive changes in that person. That is, they will be seen as an authentic and credible source of motivation and influence by participants.
• Social support from trainers and the group – where trainers model and reinforce supportive behaviour, and group exercises provide opportunities for the participants to demonstrate supportive behaviour toward each other.

The UK edition of the JOBS II manual recommended that there were 10 to 20 participants per course, with the groups being led by trained facilitators (preferably, one male and one female, who have some experience or understanding of unemployment), and using active learning techniques to encourage participation.

Given the highly specified and theory-based intervention model, and in contrast to many other forms of employability provision, Group Leaders were required to deliver the JOBS II programme with a high degree of fidelity to the principles and ethos of the JOBS II model, and to closely adhere to the manual and topic scripts.

3. Recognition, uptake and participation in Group Work

This chapter synthesises findings on participation in the Group Work trial, their reasons for participating, and the effectiveness of the processes involved. It is based on qualitative evidence from the process evaluation and observational research and the analysis of management information and survey results, and is structured around three areas:

• Recognition of potential beneficiaries;
• Initial uptake of the Group Work offer; and
• Subsequent participation in the course.

3.1 Recognition of potential beneficiaries

Jobcentre Plus Work Coaches were responsible for recognising benefit claimants who would potentially benefit from Group Work. Potential beneficiaries were defined in the trial as benefit claimants who were struggling with their job search or work-related activity and/or were feeling low or anxious and lacking in confidence in relation to their job search.

3.1.1 Numbers of potential beneficiaries recognised

The target of the trial was for a total of 26,000 benefit claimants to be recognised as a potential beneficiary and randomisation applied. At the end of the trial in March 2018 16,193 potential beneficiaries had been recognised by Work Coaches – 62 per cent of the target. Figure 3.1 shows the number of beneficiaries recognised in each month of the trial, and that the peak months for recognition were July to November 2017.

Figure 3.1: Percentage of total potential Group Work beneficiaries recognised by trial month

Month	Percentage
Jan-17	1.4
Feb-17	4.1
Mar-17	6.9
Apr-17	6.4
May-17	6.1
Jun-17	6.7
Jul-17	9.7
Aug-17	9.1
Sep-17	9.8
Oct-17	7.2
Nov-17	9
Dec-17	5.6
Jan-18	6.2
Feb-18	7.2
Mar-18	4.4

Source: DWP management information

Based on the findings from the qualitative interviews in the process evaluation, the main factors determining the number of potential beneficiaries recognised were:

• Work Coach training. Work Coaches were generally positive about the quality, coverage and appropriateness of the training they received, and reported particular benefits to additional activities such as briefings by Group Work Group Leaders and opportunities to observe delivery. However, they felt that the time lag between the training and implementation, the level of detail provided about the course and what differentiates it from other provision, could have been improved. Although post-implementation training helped to address these issues, some Work Coaches remained unclear on the distinctive elements of the course at the time of the process evaluation fieldwork.
• Interpretations of the criteria for recognising potential beneficiaries. Work Coaches had often understood from the pre-implementation training that the course was aimed at those with or at risk of developing mental health issues. They thought a broader interpretation of the criteria had been communicated subsequently, which led to them identifying more potential beneficiaries.
• Drives to increase numbers. Work Coaches reported that from March 2017 onwards they had increasingly been encouraged to consider more and/or more wide-ranging types of benefit claimants as potential beneficiaries.
• Time constraints. Work Coaches reported that the time necessary to complete the onscreen survey with benefit claimants they thought could benefit from the course (a required process in the trial) was the main barrier to them recognising more potential beneficiaries^{[footnote 22]}.
• Claimant feedback. Over time, positive feedback from benefit claimants who had been on the course encouraged Work Coaches to identify more potential beneficiaries.
• Underlying Work Coach attitudes. A minority of Work Coaches were reported by the jobcentre managers and Group Work leads interviewed to be unenthusiastic about any new initiative introduced in their jobcentre - not specifically Group Work - and had recognised few or no beneficiaries.

These factors reflect some of the challenges associated with implementing a trial within the complex environment of local Jobcentre Plus delivery, where there are competing priorities and demands on staff time.

3.1.2 Characteristics of potential beneficiaries recognised

Table 3.1 provides an overview of the characteristics of the potential beneficiaries recognised in the trial, drawing upon data collected by Work Coaches in the randomisation survey, the baseline survey undertaken on Day 1 of the course and from administrative data supplied by the Department for Work and Pensions (DWP)^{[footnote 23]}.

This data illustrates that the potential beneficiaries recognised by Work Coaches in the trial were diverse - both demographically and in terms of confidence about job prospects, wellbeing and mental health. It included benefit claimants who were potentially struggling with job search, lacking in confidence, anxious and with likely depression. Equally it included benefit claimants who did not exhibit these characteristics.

Table 3.1: Characteristics of potential Group Work beneficiaries recognised

Gender

Response	Percentage (%)
Male	58
Female	42

Age¹

Response	Percentage (%)
16 to 24	14
25 to 34	23
35 to 49	33
50 to 59	24
60 to 65	6

Achieved grade C or above for both English and Maths GCSE¹

Response	Percentage (%)
Yes	41
No	52
Not answered	7

Benefit receipt²

Response	Percentage (%)
JSA	74
Universal Credit	12
ESA/DLA/CA	14

Length of time on benefits in last three years²

Response	Percentage (%)
Up to 7 days	6
8 to 31 days	7
1to 6 months	28
6 to 12 months	16
One to two years	15
Over two years	28

When last in work³

Response	Percentage (%)
In the six months before randomisation	10
6 to 12 months	6
1 to 2 years	5
Over two years	15
Can’t remember	12
Never in paid work	53

Confidence in finding job¹

Response	Percentage (%)
Confident will find a job	55
Not confident will find a job	45

Job search self-efficacy³

Response	Percentage (%)
Higher job search self-efficacy	49
Lower job search self-efficacy	51

General self-efficacy³

Response	Percentage (%)
Higher self-efficacy	54
Lower self-efficacy	46

WHO-5 wellbeing³

Response	Percentage (%)
With likely depression/poor wellbeing	60
Other	41

Life satisfaction (ONS measure)¹

Response	Percentage (%)
Satisfied with life	32
Other	68

Life worthwhile (ONS measure)¹

Response	Percentage (%)
Thinking life worthwhile	44
Other	56

Happiness (ONS measure)¹

Response	Percentage (%)
Happy	41
Other	59

Anxiety (ONS measure)¹

Response	Percentage (%)
Anxious	30
Not	70

PHQ-9 depression³

Response	Percentage (%)
Depression suggesting caseness[footnote 24]	46
Other	54

GAD-7 anxiety³

Response	Percentage (%)
Anxiety suggesting caseness	51
Other	49

Base: onscreen survey / administrative data: 16,193.

Base: baseline survey: 2,029.

Source¹: Randomisation survey, Source²: DWP administrative data, Source³: Baseline survey

As described in Section 2.1.1, which reviews the international evidence on the effectiveness of JOBS II, the profile of individuals participating in the UK trial shows several differences to those participating in the trials described:

• In the US, those taking the course had been unemployed for less than 13 weeks, and so were arguably closer to the labour market and so responsible for the higher rate of take-up compared to participants in the UK.
• In Finland, while the trial population was closer to that in the UK in terms of focussing on the longer-term unemployed, the Finnish participants tended to be more likely to be more educated, more likely to be female, and were mainly former office workers. The recruitment process for the Finland trial was also different, as potential participants were contacted about the trial and only those expressing interest and agreeing to the randomisation process took part. The trial generated a much higher participation rate than the UK trial, potentially because the recruitment approach recruited those who believed that the programme would benefit them and so were willing to engage. This means that the impacts from such a trial are unlikely to be replicated with a broader population, such as the UK. In contrast, the UK trial followed a Zelen methodology, with the course being introduced to benefit claimants once they were randomly allocated into the treatment group. As participation in the course was voluntary, only those who had been randomly allocated to the treatment group and were interested in the course participated in it.

During the trial the profile of potential beneficiaries being recognised also changed in some respects, with benefit claimants who were more confident about their job prospects being considered for the intervention by work coaches. In the first quarter of the trial, data from the randomisation survey showed that around a half (52 per cent) were benefit claimants who thought they were likely or certain to find a job within 13 weeks. By the final quarter of the trial this had increased to over two-thirds (69 per cent), possibly due to Work Coaches adopting an increasingly broad interpretation of the eligibility criteria and the extension of the trial to Universal Credit (UC) claimants in late 2017. There was less apparent change over time in other benefit claimant characteristics.

3.2 Uptake of the Group Work offer

A total of 11,900 benefit claimants were randomised into the Group Work group in the trial and offered the opportunity to go on the course by their Work Coach. Of these, 5,355 (45 per cent) initially accepted this offer.

3.2.1 Introducing and explaining Group Work

The intended process in the trial was for Work Coaches to introduce and explain the course to benefit claimants following randomisation. Participation was voluntary, with no sanctions being applied if benefit claimants declined the opportunity to take part.

The process evaluation found that not all Work Coaches had initially followed the intended process for when they introduced the course to benefit claimants, with some saying they had been doing this before randomisation. However, seven to nine months into the trial, and following post-implementation training, it was reported that the intended process was consistently being followed.

Key factors in terms encouraging take-up of the course included:

• Emphasising the difference of the course - particularly with benefit claimants who had attended previous Jobcentre Plus provision and potentially saw little value in going on something that may have superficially sounded very similar.
• Using benefit claimant feedback and international evidence – benefit claimants were receptive to being told about the impact of the course on others, either based on evidence from other countries or about other claimants who had attended.
• Tailoring the explanation to the individual - for example, by picking up on something they raised and using that as a starting point for introducing the course.

The effectiveness of Work Coaches in encouraging participation in the course appeared to be determined by the following main factors:

• The Work Coach training - with Work Coaches drawing on information provided to describe the course, including the international evidence of effectiveness, although not all felt the training gave them a firm grasp of the course’s distinctive features.
• Their exposure to the course - observing a day of the course, or briefing from a Group Leader, gave Work Coaches extra knowledge which made them better able to describe the course, its distinctive features and address any questions raised.
• Feedback from benefit claimants - similarly, Work Coaches reported gaining additional knowledge of the course from benefit claimants attending it and used these as examples of the effectiveness of the course.

3.2.2 Benefit claimants’ reasons for initially accepting or declining the Group Work offer

Based on the process evaluation findings, the main reasons why benefit claimants had accepted the Group Work offer were:

• Desire for help with job situation. This was the most common reason cited, especially amongst Jobseeker’s Allowance (JSA) claimants. Even where individuals said they had received vague or unclear descriptions of the course, this overriding motivation to progress into work had been enough for them to accept.
• Desire for help with general state of mind. Some said they were motivated to accept the offer because they thought it might help with their motivation, self-esteem, or general outlook on the life.
• Desire for mental stimulation. There were also benefit claimants who reported attending the course because it gave them a reason for “getting out of the house”, something to do and an opportunity to meet new people.
• To meet assumed expectations or requirements. Exceptionally, a small number of benefit claimants indicated that they had agreed to attend as they assumed this was expected or required as part of their benefit claim. While they did not say their Work Coach had explicitly told them attendance was mandatory, they had reached the conclusion that their benefits could be affected if they did not accept. Notwithstanding these findings, the voluntary nature of the course was both understood and strongly welcomed by most benefit claimants.

The main reasons that benefit claimants gave for declining the Group Work offer were:

• Perception the course would not help. Some said they had declined the offer because they did not believe it would help them or offer them anything they did not know already. It is possible that how their Work Coach had explained the course, and their ability to differentiate it from other provision, was a factor in this.
• Discomfort with the group setting. Some said they turned down the offer of going on the course out of a simple discomfort with group settings.
• Health reasons. There were benefit claimants who said they declined either because they did not think they were well enough to attend or they had a fixed medical appointment that clashed with the course. Others said they declined as their health condition made it uncomfortable for them to travel for extended periods of time.
• Other personal commitments or crises. These included care responsibilities for sick relatives or children during school holidays which meant benefit claimants felt unable to accept the offer of attending the course.

3.3 Participation in the course

3.3.1 Participation rates

Although 5,355 benefit claimants (45 per cent of those offered the course) agreed to go on Group Work when it was initially offered to them, not all subsequently participated in the course. 4,046 (34 per cent of those offered the course) attended the Initial Reception Meeting (IRM) that preceded the course, and 2,596 (22 per cent of those offered) attended day one of the course itself^{[footnote 25]}.

It is not possible to fully determine the reasons for this attrition in take up and participation. The process evaluation included interviews with benefit claimants who had declined the offer and others who had accepted and went on to attend, but did not capture those who had initially accepted then not attended. The six-month survey included one question for respondents who declined the opportunity to find out why this was, which provided some broad indications of why not all who accepted the offer went on to participate in the course:

• Health or other personal commitments or crises intervened in benefit claimants’ lives after they had initially accepted the course offer and prevented them from attending. Health issues or ‘not feeling well enough’ was the most widely cited reason for not attending the course in the survey; and
• Benefit claimants had found work after initially accepting the course offer, meaning they would not have the time to attend and/or no longer perceived the need to do so. Clashes with paid work was the second most widely cited reason for not attending the course in the survey.

It is also feasible that some benefit claimants had only given a “soft yes” at the point they were initially offered the opportunity and had second thoughts after this point, e.g. on reflection deciding it was not going to help them.

The role of the IRM in supporting participation in the course is not straightforward. Benefit claimants who attended the IRM and participated in the course were overwhelming positive about its value in aiding their understanding of the course and motivating them to attend. However, the provider staff who ran the IRMs confirmed that a minority of benefit claimants decided not to attend the course at that point - typically those who had not understood attendance was voluntary until this had been explained to them at the IRM. Neither participants nor Group Leaders necessarily viewed this as a negative. There was a shared view that participants “had to want to be there” for the group dynamics on the course to function effectively, so while the IRM may have contributed indirectly to lower levels of participation, it may have equally performed a necessary role in reinforcing the fidelity and effectiveness of the trial.

There is also some variation in uptake and subsequent participation in the course among those offered the course by benefit claimant type. Overall, 22 per cent initially agreed to go on the course and subsequently participated in it, but there are statistically significant differences between benefit claimants with different characteristics, including^{[footnote 26]}:

• Gender - 23 per cent of men agreed and participated in the course compared to 20 per cent of women, a small but statistically significant difference;
• Age - 13 per cent of benefit claimants aged 16 to 24 agreed and participated in the course compared to 28 per cent of those aged 50 to 59;
• Benefit receipt - seven per cent of Income Support (IS) and 11 per cent of Employment Support Allowance (ESA) recipients participated in the course compared to 22 per cent of Universal Credit (UC) and 24 per cent of Jobseeker’s Allowance (JSA) recipients;
• When last worked - 20 per cent who had worked in the last six months agreed and participated in the course compared to 31 per cent who had last worked over two years ago. In addition, 21 per cent of course participants reported never having been in paid work;
• Job search self-efficacy - 14 per cent with higher levels of job search self-efficacy agreed and participated in the course compared to 30 per cent with lower levels;
• General self-efficacy - 17 per cent with higher levels of general self-efficacy agreed and participated in the course compared to 27 per cent with lower levels;
• Wellbeing – benefit claimants with lower levels of life satisfaction (23 per cent), happiness (22 per cent), and feeling life is worthwhile (23 per cent) on the ONS-4 wellbeing measure were more likely to agree and participate in the course than those with higher levels of life satisfaction (20 per cent), happiness (21 per cent), and feeling life is worthwhile (21 per cent) – again small but statistically significant differences; and
• Anxiety – benefit claimants with lower levels of anxiety (23 per cent) on the ONS measure were more likely to agree and participate then those with higher levels (21 per cent, a small but statistically significant difference) but there are no statistically significant differences on the GAD-7 anxiety scale between those whose scores suggest caseness or otherwise.

These data indicate that benefit claimants with characteristics associated with the target audience for the JOBS II model (i.e. struggling with job search and/or feeling low or anxious and lacking in confidence about job search) were slightly more likely to agree to go on the course and attend it than others offered the opportunity.

Health reasons are likely to explain the lower proportions of IS and ESA recipients who agreed and participated in the course compared to UC and JSA recipients. It may also be that the requirement for JSA and UC claimants to meet with their Work Coaches more regularly than other benefit regimes meant that there was more chance of them being recognised and referred to the course.

3.3.2 Characteristics of course participants

Table 3.2 provides an overview of the profile of the course participants in the trial^{[footnote 27]}. This is similar to the profile of all benefit claimants recognised as potential beneficiaries but there are differences due to the different take up rates by claimant type. For example, a higher proportion of male and older benefit claimants agreed and attended the course and they therefore represented a larger share of the participant population.  

Table 3.2: Characteristics of Group Work course participants

Gender

Response	Percentage (%)
Male	63
Female	37

Age¹

Response	Percentage (%)
16 to 24	9
25 to 34	18
35 to 49	34
50 to 59	32
60-65	8

Achieved grade C or above for both English and Maths GCSE¹

Response	Percentage (%)
Yes	41
No	52
Not answered	7

Benefit receipt²

Response	Percentage (%)
JSA	82
Universal Credit	12
ESA/DLA/CA	6

Length of time on benefits in last three years²

Response	Percentage (%)
Up to 7 days	4
8 to 31 days	6
1 to 6 months	24
6 to 12 months	15
1 to 2 years	16
Over 2 years	35

When last in work³

Response	Percentage (%)
In the six months before randomisation	9
6 to 12 months	7
1 to 2 years	7
Over 2 years	21
Can’t remember	5
Never in paid work	51

Confidence in finding job¹

Response	Percentage (%)
Confident will find job	51
Not confident will find job	49

Job search self-efficacy³

Response	Percentage (%)
Higher job search self-efficacy	31
Lower job search self-efficacy	69

General self-efficacy³

Response	Percentage (%)
Higher self-efficacy	42
Lower self-efficacy	58

WHO-5 wellbeing³

Response	Percentage (%)
With likely depression/poor wellbeing	54
Other	46

Life satisfaction (ONS measure)¹

Response	Percentage (%)
Satisfied with life	30
Other	70

Life worthwhile (ONS measure)¹

Response	Percentage (%)
Thinking life worthwhile	42
Other	58

Happiness (ONS measure)¹

Response	Percentage (%)
Happy	40
Other	60

Anxiety (ONS measure)¹

Response	Percentage (%)
Anxious	29
Not	71

PHQ-9 depression³

Response	Percentage (%)
Depression suggesting caseness	45
Other	55

GAD-7 anxiety³

Response	Percentage (%)
Anxiety suggesting caseness	49
Other	51

Base: onscreen survey / administrative data: 2,596.

Base: baseline survey: 609.

Source¹: Randomisation survey, Source²: DWP administrative data, Source³: Baseline survey

This data also shows that the population of course participants was still diverse across a range of job search, confidence, wellbeing and mental health indicators. For example, participants were split almost equally between those who were and were not confident about finding a job, and between those whose GAD-7 and PHQ-9 scores suggested they had and had not reached suggested case level anxiety and depression respectively.

There was also variation across the course participants, in terms of the length of time on benefits and the time since they were last in paid work, with over one third (35 per cent) of participants having been on benefits for over two of the previous three years. Half (51 per cent) reported never having been in paid work, while nine percent had been in work in the previous six months. Participants in the UK trial differed to those in several of the JOBS II intervention trials in terms of their duration of unemployment, with, for example, the large US trials featuring participants unemployed for 13 weeks or less, and 28 per cent of the Finnish trial sample being unemployed for 12 months or more.

The implications of this diversity are not clear cut. Some provider staff felt it potentially diluted the impact of the trial. However, there was also a widespread belief amongst provider staff that a mix of participant characteristics (albeit not too much of a mix) was a positive for the group dynamic on the course and that a broad range of people could benefit from what the course offered. This is explored in more detail in Chapter 4.

3.4 Course retention and drop out

Of the 2,596 benefit claimants who started the course, 1,922 (74 per cent) completed it. Participants interviewed for the process evaluation provided three main reasons why they had completed the course:

• Enjoyment of the course - course completers commonly described their experience of the course as being “fun”, “a right laugh”, or “enjoyable”. Although easy to dismiss this as irrelevant to the effectiveness of the course, participants cited enjoyment as one reason why they had continued with it.
• Perceived relevance and value - participants talked about continuing to attend because they found the content relevant to their personal situation, and that they could feel it having an effect on their mind-set and outlook.
• Personal ethos - some participants reported completing the course partly because of a personal commitment to “see things through to the end” - including some who had persevered despite not feeling the course was directly relevant to them.

The reasons given for not completing the course included:

• Health factors and other competing priorities - health issues or not feeling well enough was the most common reason given by participants. Other competing priorities included care responsibilities, a job interview or clashes with paid work.
• Perceived lack of relevance or value - the second most commonly cited factor was that participants ‘didn’t feel like it was doing me any good’. They had reached the conclusion they did not need help with their job situation or more generally.
• Group dynamics – exceptionally, Group Leaders and participants suggested that disrupted group dynamics could be a cause of early exit. For example, one older participant who dropped out of the course described feeling uncomfortable in a group comprising mainly younger members. Some Group Leaders also reported that individuals had dropped out because of a clash or divide in a group.

4. Experiences of the Group Work course

This chapter synthesises findings on the Group Work course itself, its perceived effectiveness, active elements and short-term outcomes. It is based on qualitative evidence collected through the process evaluation and observational research with Jobcentre Plus staff, Group Leaders and participants, plus selected participant survey results.

The findings are structured around the following areas:

• Course delivery;
• Participant reflections (overall and on different aspects of the course);
• The active elements of the course; and
• The short-term outcomes of the course.

4.1 Course delivery

The Group Work course was delivered by a pair of facilitators, termed Group Leaders, who received seven weeks of training and ongoing support to deliver their roles. Course delivery was reported as broadly following the intended model, although in some cases elements were passed over due to time pressures.

The Group Leaders interviewed commonly described the course as challenging, but enjoyable and rewarding to deliver, and all described a strong belief in the course design and the theory that underpinned it.

Group sizes were within the parameters stated, although providers found that group sizes towards the upper limit of 20 could be challenging to manage. In some cases, the scheduling was problematic, with waits to attend being between three and four weeks due to low referral numbers and the group size requirement. In a few cases courses were cancelled at short notice when individuals dropped out at the last minute. Overall, the providers reported being able to meet demand for courses.

4.2 Reflections on the course overall

In the six-month survey after baseline, 92 per cent of participants said they had found the course useful, and this proportion was consistently high across different types of benefit claimant. There are no statistically significant differences by age, gender, benefits claiming, time since last worked or levels of wellbeing and mental health^{[footnote 28]}.

Qualitative findings from the process evaluation suggest a similar but more nuanced picture, in which participants’ overall reflections on the course varied from the overwhelmingly positive, to the mildly appreciative and, much less frequently, the negative. Where participants were on this continuum was partly mediated by the extent to which they perceived they were struggling with their job search, confidence and/or anxiety – with the most positive typically being those who were struggling with their job search and had been anxious or low on confidence.

“It was excellent. For me it kind of did help my self-esteem…for myself, it’s the way you actually saw yourself in the workplace and how, it kind of totally different way of looking for work.” (Completer; Avon, Severn and Thames)

“It was a very good course. You got by the end and you’d actually learned a lot more than you actually thought. It gives you more incentive, more confidence for looking for work.” (Completer; Mercia)

Participants amongst the mildly appreciative included those who felt they weren’t struggling with their job search nor felt particularly anxious or lacking in confidence.

“It was fine. I didn’t really learn that much new, but it reconfirmed other things…it was like a refresher course for me.” (Completer; Merseyside)

“Yeah it was OK. It was useful that it told other people what to do, not necessarily me because I’m not shy, but it was just other people, being able to bring their personality out.” (Completer, Durham and Tees)

Some participants who saw themselves as facing very significant barriers to work (relating to a serious physical or mental health condition) were also relatively lukewarm about the course or occasionally negative. They indicated that it had not been able to fully help them overcome their barriers.

“I still am unemployed but I’m semi-disabled and it didn’t help in that sort of respect.” (Completer; Avon, Severn and Thames)

One or two other participants described personal barriers to engaging with the course (relating to literacy and/or English language skills) which also contributed to their less positive experiences.

4.3 Reflections on different aspects of the course

4.3.1 Facilitation of the course

In common with previous studies of JOBS II (Vinokur et al., 2000; Vuori et al., 2002), the Group Leaders and how they deliver the course content emerged as an important determinant of the perceived success of the Group Work course.

In the six-month survey, over three-quarters of participants agreed that the person or people leading the course when they attended really understood the challenges of finding work (79 per cent) and had personal experience of being unemployed (76 per cent).

In the process evaluation, key aspects underpinning participants’ positive views of the course facilitation included:

• Being treated as an equal - participants frequently contrasted the facilitation of the course with other provision received, with the main difference being that they felt treated as an equal.

“They treat you like adults not like idiots, not like some of the courses do, they ask what you think and how you feel, yeah, it worked.” (Completer; Avon, Severn and Thames)

Self-disclosures by Group Leaders about their own experiences also helped to overcome any teacher-participant divide in the eyes of participants.

• Establishing a participative, safe environment - Group Leaders were widely praised for their ability to encourage participation across the groups, particularly managing group discussions without letting individuals dominate, encouraging quieter individuals to contribute, and asking for and listening to comments.

“It wouldn’t be, yeah, right, well we wouldn’t look for that would we, or that’s a silly thing to say. There was none of that. Everybody’s opinion was valid. And you could see people actually beginning to take part, after the first hour or so, rather than just be told.” (Completer; Midland Shires)

The Group Leaders were also perceived to have played an important role in enabling more nervous or struggling individuals to participate, through the positive reinforcement of their inputs and a willingness to provide additional explanation.

• Responding to individual needs - Group Leaders faced the challenge of drawing out shared experiences across the group, while also helping participants address their individual barriers or challenges. Participants indicated they were generally able to do this effectively. While the course content was scripted and intended to be delivered to the group as a whole, most felt their Group Leaders had been proactive in providing individual support. This responsiveness helped ameliorate some participants’ concerns about the fixed nature of the course by providing more tailored support.

“They were fantastic. If you needed one on one they would, they’d give you one on one.” (Non-Completer; Durham and Tees)

In the six-month survey, 56 per cent of participants agreed that the course had been tailored to their needs and 21 per cent disagreed.

• Role-modelling - part of the theory underpinning the JOBS II model relates to the referent power of the Group Leaders, their ability to motivate or inspire change through personal example. While the participants interviewed did not explicitly describe this happening to them, this is not to say it did not happen – for example benefit claimants responded positively to self-disclosures by the Group Leaders.

“They didn’t come in and say, ‘Right we’ve got a job’, [instead] they were like, ‘We’ve been all through it. We’ve been there and done that and this is what we’ve learned’.” (Completer; Mercia) Participants also described Group Leaders displaying behaviours and traits which they evidently admired – including their professionalism, confidence in addressing the group, and the ability to work well as a team.

4.3.2 The format of the course

The format of the course emerged as a less important factor for participants or provider staff than its facilitation or content. Nonetheless three main characteristics were highlighted as having a bearing on benefit claimants’ experiences of the course.

• The balance between facilitator-led and interactive elements - which was felt to have worked well: “It’s not like going into a class where you’re going to learn one thing and the teacher teaches you. It was a practical session, it was trying to get you to think a certain way, and to look at it in a different way.” (Completer, Mercia)

While some participants described initial concerns over the role play elements, this had not been a barrier to their participation and for some was their favourite part of the course. Group Leaders emphasised the importance of role play in giving participants practice and confidence in the techniques they were taught. Participants also valued the small group elements of the course, with less vocal participants reporting feeling more able to contribute in this environment.

• The written elements of the course - a small number of the participants interviewed had low levels of literacy and/or English language skills and found this a challenge to their participation in the course. Exceptionally, one reported dropping out of the course as they were unable to follow its written content, while others also reported struggling but had persisted with the course and provided positive reflections on it. In these cases, the participative elements of the course provided a means of partly, if not wholly, mitigating this.

• The timing and duration of the course - the four-hour duration of each day of the course was an important enabler for some participant types to attend, and was popular amongst participants in general as it represented a manageable chunk of time that did not leave them tired or drained. Some drew an explicit parallel between attending the course and of going to work, in part due to the benefit of “getting out of the house”, but also about having a structured routine to their day lacking while they were unemployed.

Indeed, extending the course, by a day or an additional week, was suggested by several participants. For some this was primarily due to the enjoyment and sense of purpose it provided, while others thought extending the course would provide additional benefits: “I think we as a group thought the best thing would be to do it for a longer period of time. Not longer each day, but more days. Because there were sections that once you started to feel a bit more hopeful you wanted to go back and do that bit again.” (Completer, Midland Shires)

4.3.3 The group dynamic

In the six-month survey, around two-thirds of participants agreed that working with a group of others on the course had meant they made more progress (62 per cent), felt more supported (65 per cent), and gave them new ideas (68 per cent). Less than a third perceived negatives to working with a group of other people, in terms of discomfort with sharing ideas and experiences (32 per cent), not getting enough individual help (23 per cent) or time being wasted by other participants (20 per cent).

Participants and Group Leaders interviewed in the process evaluation also placed considerable emphasis on the group element, citing the key benefits as including the realisation that there were others in similar situations to themselves, which had enabled them to self-reflect and learn how to address common barriers from others.

“When you’re at home you feel a little isolated and you feel that you’re the only one, and when I realised that there were other people with the same frustrations that I had.” (Completer; Mercia) The observational research indicated that exposure to the group-setting could initially be challenging too for some participants. It could serve to reveal the extent to which they had become isolated while out of work or lead them to compare themselves unfavourably to others in the group. Equally participants acknowledged that this ‘realisation’ or transition was a necessary ‘step’ for them to ‘get out of their situation’, feel better about themselves, and regain a sense of stability and personal control.

It was also reported in the process evaluation interviews that the group dynamic did not always function positively for everyone, with the key determinant being commonality versus difference in the group. Views on this point varied, although there was broad agreement that too much commonality or too much difference was unhealthy, and the ideal was a balance between the two.
Too much commonality was seen to risk a group turning in to “a moaning shop”, within which negative perceptions could be reinforced and limit the scope for participants to benefit from new information, perspectives and strategies.

“If you’ve got the same people with the same thing it tends to be just you’re feeding off the same people. We had different people with different experiences, you know, you learn a bit off that and they learn a bit off you.” (Completer; Mercia)

At the same time, too much difference was perceived as inhibiting the development of a group dynamic or risk isolating individuals. Although “not having a job” was a common denominator, some participants reported not feeling part of a definable group on the course. This included a small number who felt they were not struggling with their job search (and not thinking they faced the same barriers as others), and conversely others who felt they faced different and larger barriers (including some with a significant disability).

Less clear-cut were differences in terms of participants’ age, benefits claimed and their educational or professional background. Exceptionally, some indicated that these had been a dividing line between participants in their group, leading one to suggest that different cohorts of benefit claimants should attend separately, e.g. having a Jobseeker’s Allowance (JSA) and an Employment Support Allowance (ESA) group. Another, younger, benefit claimant thought that “if there were people closer to my age it might have been easier to mix with them”. However, the majority of participants and all Group Leaders reported that having a mix of these characteristics was beneficial.

“It means you get a good mix of opinions, good mix of ideas and contributions if there is that mix of people. We do find that if there is that mix, that can be quite useful for some people who aren’t so quite academic.” (Group Leader).

4.3.4 Course content

The majority of participants reported finding some or all of the content useful and sufficiently relevant to their circumstances. Although addressing familiar topics (e.g. CVs and job interviews), the course brought new insights and ideas that participants thought could be applied to their situation.

“I thought it was very good, because I learnt more, how to prepare yourself for the interviews. I would have never gone into an interview with a notepad and write down what sort of questions to ask.” (Completer; Avon, Severn and Thames)

Others had also recognised and endorsed the intent behind much of the content, in terms of changing the way they perceived themselves and their relationship to work.

“I think the psychology part of it worked for all of us. You’re expecting it to be, kind of, a plug in, this is the psychological bit. It wasn’t like that. It obviously was progressing through the different days. But it did work on us.” (Completer; Midland Shires)

“The key difference between what we normally deliver and Group Work is the ‘thinking part’…the psychological support which is in the course materials and how it’s delivered. The instructors [Group Work Leaders] are there to support people. That…psychological bit is missing in most welfare to work courses’ (Group Leader).

The minority of participants who reported finding the course content less insightful or relevant to them, tended to be those who felt they were not struggling with their job search or those with a disability or long-term health condition who felt the content was not directly relevant to them and/or downplayed the challenges they faced.

4.4 The active elements of the course

The observational research had a specific focus on exploring the active elements and what previous JOBS II studies have termed active ingredients (Vinokur and Schul 1997) within the course, which may lead to changes in health, wellbeing and job search behaviour. The participant observation of the intervention ‘in real time’ provides a more in-depth account of how these operate. Three active elements were identified.

4.4.1 Active element 1: supporting active participation in a group context - access to social contact and social support

From observing the course in action, a collective group change in attitudes and wellbeing was often heard and reflected in participants’ use of language, demeanour and overall wellbeing. On Day 1 participants appeared reserved, and in many cases displayed visible signs of anxiety and worry, including being uncomfortable when asked to speak about themselves in front of other participants. However, over the course of Day 1 and through the sessions in Day 2, which required participants to speak to the group about the challenges they face in job search, one participant commented:

“Getting people to say how they feel about their situation really helps. I thought no one would care about hearing me being shy, demotivated and depressed but I was shocked when [a fellow participant] blurted out that he’s been through the same thing. I didn’t see that coming.” (Participant; Midland Shires).

This process of self-reflection via active participation and interaction with fellow participants was reported to increase feelings of social support and reduced social isolation. Indeed, the Process Evaluation Technical Report (Knight et al., 2020a) found that 66 per cent of course participants reported improved access by Day 5 to the latent benefits that are associated with being in work, such as social support and social contact.

The opportunity to meet people like themselves (i.e. from similar social and economic backgrounds, with similar experiences of the job market and of family or personal health problems) gave them new perspectives on their situation and enabled them to cope better with the challenges they faced.

However, a more nuanced understanding is that a mix of individuals within the group was required for participants to benefit. The most effective groups observed and reported by Group Leaders consisted of an equal gender ratio and a combination of participants with different levels of wellbeing, social support, self-reported mental health and durations of unemployment. It was particularly evident from observations and reports of the Group Leaders that the more confident, less anxious participants would help those experiencing anxiety and reluctant to participate on Day 1.

Participants and Group Leaders often compared the quality of the Group Work learning materials and the delivery setting with existing provision they had either attended or delivered. The common observation was that existing employability courses and interventions were inferior and out of date compared to Group Work.

“The usual courses we have delivered are poorly written and poorly delivered by staff whose focus is on a single important task; to ensure that every customer completes their course folder. Courses are often delivered in hired venues with little to no supporting equipment or staff, and all at the lowest possible cost. The customers know this – walk into any training room and you will see the companies’ attitude towards their customers. Often some of the customers will ask openly; ‘how much are you getting for us on this course?’ That says it all!” (Group Leader)

4.4.2 Active element 2: replicating the time structure and routine of employment

The second significant element through which mental health and wellbeing appeared to change is via the development of a sense of routine and structure to the day. Participants reported that attending the course for four hours a day for five days had got them into (or back into if they had previously worked) a habit of ‘getting up and getting out of the house’, established a constructive daily routine and developed their self-worth.

Group Leaders and participants frequently referred to the intervention emulating or ‘being like’ the experience of having a job, or what they perceived having a job must be like if they had never had one, in terms of structuring a daily routine:

“When I read the course materials you get a sense that whoever designed it was trying to make it like ‘having a job’. You have to get here for 9, talk to people, interact and do tasks…constructive ones. We are giving people an idea of what it’s like to have a job It’s the soft skills which so many people don’t have let alone hard skills and qualifications.” (Group Leader)

Participants often described to the group and in personal conversation how prior to Group Work and while being out of work they would try to adopt a routine or daily structure by extending the time needed to complete simple tasks, such as posting a letter and going to the shops, in order to ‘fill hours in the day’. Participants linked the ability and the feeling that they were constructively organising their time by attending the course, with positive effects on mental health and wellbeing.

4.4.3 Active element 3: Group Leader effectiveness and credibility

The most important intervention element identified across the trial sites, and fundamental to how the other active elements operated and how course materials and sessions were received, was the role and quality of the Group Work Group Leader. From observations across the trial sites, the most effective Group Leaders were those who displayed empathy, relatedness and had experience of being unemployed, particularly those with previous episodes of being out-of-work and of dealing with their own mental health issues or those of relatives. As one Group leader reflected:

“As I said before a lot of the employability courses that we deliver tend to be like traditional classroom type. You stand at the front and read out ‘how to write a CV’, ‘Where to find jobs’. It doesn’t work. It’s passive and people give up listening after a couple of hours. A lot of the people who come on Group Work left school at 14. School didn’t work for them so why make employability courses like school? Group Work is completely different. You’re the catalyst for getting them motivated and activated.” (Group Leader)

The most effective Group Leaders also appeared to be those whose delivery / teaching styles involved them interacting with participants from the start of the intervention. They would adopt the role of facilitator and prompt participants to find their answers to the tasks in each session. Group Leaders would move around the room to listen into group exercises, and ‘act out’ the case-study parts of each session rather than simply reading them from the text. In addition, these Group Leaders did not question people about why they did not have a job or had struggled to find one, which participants reported was a common feature and experience of previous employability provision.

By adopting a more involved role and asking participants about issues they faced outside of job search, Group Leaders aimed to improve confidence and motivation. Some Group Leaders were reported to have gone out of their way after sessions had finished to provide additional support, such as helping participants with disputes with Work Coaches or to access mental health services, which went beyond their official responsibilities as a Group Work Group Leader.

4.5 Short-term outcomes of the course

Detailed evidence is presented on the impacts of the course in Chapter 5, but this section briefly outlines what outcomes were reported while participants were on the course and shortly afterwards.

4.5.1 Outcomes during course participation

The process evaluation, participant observation and data from the Day 1 (baseline) to Day 5 surveys of course participants indicate that the majority of participants experienced a number of positive improvements in their mental health, wellbeing and job search behaviours during the course. Participants talked about, and were observed to experience, improvements in their general mood and positivity as the course progressed. Group Leaders described that by the end of the course some participants would be ‘unrecognisable’ in terms of their confidence and sense of self-esteem. It should also be noted that between 15 per cent and 37 per cent of participants recorded a lower score between Day 1 and Day 5 of the course across the measures used. Although for each measure, the share reporting an improvement exceeded those reporting a lowering, on some measures over one third showed a lower score on Day 5 of the course (namely job search activity on the FIOH scale at 37 per cent and anxiety on the ONS measure at 34 per cent). It was not possible to explore the characteristics of those reporting higher or lower scores from the Day 1 to Day 5 data alone.

Participants’ survey responses between Day 1 and Day 5 of the course are summarised in Table 4.1^{[footnote 29]}.

Table 4.1: Changes in Day 1 to Day 5 survey responses

Measure	% Improved	% Stayed the same	% Lowered	Base: Day 1 to Day 5 surveys
Job search activity (FIOH scale)	48	15	37	919
Job search self-efficacy	80	5	15	1,446
General self-efficacy	52	25	24	1,462
Life satisfaction (ONS measure)	54	25	22	1,661
Life is worthwhile (ONS measure)	50	24	26	1,661
Happiness (ONS measure)	51	22	27	1,658
Anxiety (ONS measure)	42	24	34	1,655
GAD-7 Anxiety	57	20	23	1,579
WHO-5 Wellbeing	66	10	25	1,595
PHQ-9 Depression	63	16	20	1,561
LAMB Psychosocial Deprivation	66	5	29	1,279

This data shows that a half or more participants showed improvements between Day 1 and Day 5 on most job search, wellbeing and mental health measures. In particular, 80 per cent had a higher level of job search self-efficacy; 66 per cent had increased wellbeing (on the WHO-5 scale) and improved access to the latent and manifest benefits (LAMB)^{[footnote 30]} such as social contact and support; and 57 per cent had a lower level of anxiety (on the GAD-7 scale). The lower prevalence of improvements on job search activity (48 per cent) is partly understandable given participants were still on the course at the point they completed the Day 5 survey.

There are statistically significant differences in the reported changes between Day 1 to Day 5 by participant type. Individuals whose scores on the GAD-7 and PHQ-9 scales suggested they were likely to have reached caseness for anxiety and depression respectively on Day 1 of the course, showed improved job search self-efficacy, wellbeing (on the WHO-5 and ONS measures), depression (on the PHQ-9 scale and anxiety (on the GAD-7 scale) on Day 5 compared to other participants.

With the exception of the job search activity results, it is not clear why lower scores were identified for some individuals in their Day 5 responses than at Day 1. However, it may be that the combination of self-reflection facilitated by the course and exposure to other participants led to a negative impact on their wellbeing, as participants came to realise how much their living standards had declined and their distance from the labour market. Such a response is not uncommon, and further emphasised the importance of the Group Leader role in identifying where such a drop had occurred and working with the individual to restore positive momentum.

These results indicate that those with lower baseline levels of mental health were the most responsive to the course while they were on it. This finding is echoed in the results of the impact evaluation.

4.5.2 Outcomes after course participation

Qualitative evidence was collected by following a small cohort of participants in the process evaluation and participant observation research who had completed the course within the preceding three months. Overall 125 semi-structured interviews were completed, with a cohort of 25 participants at five time points between one week and 12 months post course. The aim was to capture the general experiences of how former participants dealt with the transition from Group Work, their job search, and mental health and wellbeing issues.

Former participants finding work after two to three months, attributed their entry into work at least in part to the increased job search activity, skills and self-efficacy gained through the course. Many of the positive wellbeing and mental health outcomes reported during the course had also been sustained – both amongst those entering work and those who had not. Former participants reported that the course had not only improved their sense of resilience and how to cope or deal with setbacks in the job search process but also helped with issues such as dealing with debts. However, after three months, several of those not finding work or losing jobs secured after the course were having difficulties with their mental health, wellbeing, confidence and motivation, and had lost their sense of daily routine. In effect it appeared that many of the gains made had dissipated for this group, and they were back in a similar situation as before the course. Those in work were more likely to report that their mental health and wellbeing gains had been maintained, although this was partly dependent on the nature of this work. Some reported cycling between short-term and/or zero hours jobs and did not report the same sustainment of gains as those in more stable and rewarding employment.

In common with the impact evaluation findings in Chapter 5, one of the conclusions from the process evaluation is that more could possibly have been done to help participants maintain and build on the positive outcomes of the course. As with JOBS II, there were no formal follow-up procedures in Jobcentre Plus for when participants returned from the course, and benefit claimants’ experiences of follow-up discussions with their Work Coach were mixed, with some discussing next steps and how to take these forward, while others reported little consultation on their return. There was broad agreement amongst Group Leader and participants that a more formalised approach to follow up could have only helped to maintain the momentum developed.

5. The impacts of Group Work

5.1 Overview

As described in Chapter 4, the process evaluation, observational research and the analysis of outcomes reported by participants at the start and end of the course, identified positive outcomes for course participants during the course, with some types of participants more likely to report positive experiences than others. However, given these data were collected during the course, they do not provide evidence of whether positive outcomes would be sustained over time. Nor do they provide a measure of the ‘impact’ of Group Work, for which the outcomes of participants need to be compared against those of similar benefit claimants not offered the course.

This chapter provides the findings from the impact evaluation which quantitatively measured the impact of the course over a six and 12 month timeframe, comparing the outcomes of course participants against those of a matched comparison group who were not offered the course. This was mainly based on survey data collected from course participants, supplemented with Department for Work and Pensions (DWP) administrative data.

In line with the design of the trial (see Section 2.2), the original intention had been to measure the impact of Group Work among all those offered the course (an Intention to Treat (ITT) analysis) – that is comparing the combined outcomes of those who attended the course (course participants) and those who declined (course decliners) against those not offered the course (the control group). However, as only 22 per cent of those offered the opportunity ultimately went on the course, the ability to detect an impact of Group Work using this approach is greatly reduced. While findings from the ITT analysis are provided in the impact evaluation technical report, this chapter focuses on the impact of the programme on course participants (an Impact on Participants (IoP) analysis). This analysis, using propensity score matching, compares the outcomes of course participants against those of a matched comparison group from the control group who, at baseline, very closely resembled course participants.

Section 5.1 presents results for all course participants, with Section 5.2 presenting findings for subgroups of participants with lower levels of general self-efficacy and higher levels of anxiety and depression before they went on the course.

The tables divide the outcomes into broad domains, presenting each set of outcomes in the same table format. Each table presents the results for each outcome at baseline or randomisation, six months after baseline and 12 months after baseline. For each survey wave, the tables show the percentage or mean score for those in the Group Work course participant group and for those in the matched comparison group. Where data is not available, this is shown in the table as ‘no data’.

At each wave for each outcome, the p-value^{[footnote 31]} significance level is reported for the difference between the Group Work course participants and matched comparison group. Where the differences between the two groups are statistically significant (that is the p-value is less than 0.05), these are highlighted in red and with an asterisk. The term ‘statistically significant’ is often abbreviated in the text to ‘significant’. The text also includes discussion of impacts which are close to statistical significance using, as a rule of thumb, a p-value of less than 0.10.

A second set of p values labelled ‘p-value for differential impact’ are presented in the participant subgroup tables (Tables 5.7 to 5.9). These are based on a test of whether the impact is significantly different between the two sub-groups^{[footnote 32]}. For example, whether the impact on employment is greater for those starting with higher levels of self-efficacy than for those starting with lower levels of self-efficacy. Where the differences in impact are statistically significant, these are highlighted in blue and asterisked. These p-values are shown for completeness and are not commented on in the text.

P-values are dependent on sample size. For any given observed difference, the smaller the sample size the larger the p-value. Because the survey sample size is larger at six months than at twelve months, the impacts have to be slightly larger at twelve months to reach significance. As a very crude rule of thumb, for outcomes presented as percentages that are around the 50 per cent mark, the difference between the participant and matched comparison group has to be around nine percentage points to reach significance at six months, whereas at twelve months the difference has to be around 10 percentage points.

Although there is no statistically significant evidence of Group Work having an impact on all course participants’ participation in paid work, there are statistically significant positive impacts at six months across a range of mental health (measured by the WHO-5 scale), well-being and self-efficacy measures, as well as on measures of their confidence in finding paid work. Moreover, there is a further pattern of positive but not statistically significant differences between the outcomes of participants and the matched comparison group, including the likelihood of being in paid work, doing more job search, being less likely to have suggested case level anxiety or depression (measured by the GAD-7 and PHQ-9 scales)^{[footnote 33]}. However, despite a continued pattern of positive differences, in the main the statistically significant impacts are no longer evident at 12 months.

Group Work appears to be most effective among those with lower starting positions before they start the course (that is, for those scoring as having lower levels of general self-efficacy, and those with suggested case level anxiety or depression). For these participants, there is a wider range of statistically significant positive impacts, including on employment, at six months which continue to be sustained at 12 months. Importantly, there is no evidence of any negative impacts of attending a Group Work course.

Each section provides results for the different types of outcomes that were examined in the impact evaluation:

• Work-related outcomes;
• Job search-related outcomes;
• Wellbeing outcomes and the Latent and Manifest Benefits of Work (LAMB);
• Mental health outcomes; and
• Wider health outcomes.

An overview of the outcome measures used in each of these domains is provided in Box 1.1, Chapter 1 of this report and more detailed descriptions can be found in the Technical Report of the Impact of the Trial.

5.2 Results for all course participants

5.2.1 Work-related outcomes

Although there are positive differences between course participants and the matched comparison group across the work-related outcomes, the percentage point differences are not sufficiently large to reach statistical significance at either six or 12 months after baseline. In other words, there is no evidence reaching statistically significance that attending the Group Work course has an impact on any of the work-related outcomes.

Table 5.1 shows that six months after baseline, 20 per cent of course participants were in paid work (10 per cent working more than 30 hours per week) compared to 18 per cent of those in the matched comparison group (nine per cent working more than 30 hours per week). Twelve months after baseline 23 per cent of course participants and 20 per cent of the matched comparison group were in paid work (with 11 per cent and seven per cent working for 30 hours or more per week respectively).

There were also no statistically significant impacts on:

• Job satisfaction - with 14 per cent of course participants and 13 per cent of the matched comparison group being in work that satisfied them at six months post-baseline, and 16 percent and 15 per cent at 12 months respectively; and
• Earnings – with nine per cent of both the course participants and the matched comparison group earning £10,000 per year or more at six months, and 11 and eight per cent at 12 months respectively.

Table 5.1: Impact of Group Work on work outcomes: impact on participants

	Participants (baseline)	Comparison group (baseline)	p-value (baseline)	Participants (6-month follow-up)	Comparison group (6-month follow-up)	p-value (6-month follow-up)	Participants (12-month follow-up)	Comparison group (12-month follow-up)	p-value (12-month follow-up)
Working status[footnote 34]
% In paid work	no data	10		20	18	0.442	23	20	0.445
% In paid work 30+ hours a week	no data	2		10	9	0.85	11	7	0.135
Job satisfaction[footnote 35]						0.515			0.573
% In paid work that satisfies me	no data	no data		14	13		16	15
% In paid work that does not satisfy me	no data	no data		6	4		7	5
% Not in paid work	no data	no data		80	82		77	80
Earnings						0.495			0.748
% In paid work earning £10k pa or more	no data	no data		9	9		11	8
% In paid work earning less than £10k pa	no data	no data		6	5		11	10
% In paid work, earnings not given	no data	no data		5	3		1	2
% Not in paid work	no data	no data		80	82		77	80
Base: all	609	533		609	533		510	362

Source: Survey data.

In order to further test the potential impact of Group Work on employment with a larger dataset of course participants than in the survey, DWP administrative data on benefit receipt and monetary value was also examined to explore the impact of participation in the course on benefit receipts as a (necessarily crude) proxy for being in paid work. Six months after randomisation course participants were statistically significantly more likely (85 per cent compared to 83 per cent) to be in receipt of these benefits than those in the matched comparison group, although by 12 months after randomisation this significant difference had disappeared, with 77 per cent of course participants and 76 per cent of the matched comparison group being on Jobseekers’ Allowance, Employment and Support Allowance, Income Support or Universal Credit. There are no significant differences in the amount of these benefits that course participants and their matched comparison group received either after six or 12 months.

However, to reiterate, these are results for the total participant population. Statistically significant impacts on work outcomes are detected for certain subgroups of participants – see Section 5.2.

5.2.2 Job search-related outcomes

Despite the lack of statistically significant impacts on entering paid work, Group Work appears to have equipped participants with some attributes (increased general and job search self-efficacy) and behaviours (submitting CVs to employers) that should increase their likelihood of finding work in the future.

The six and 12-month surveys included a range of measures of job search activity, self-efficacy and confidence that they can find work, as shown in Table 5.2. Group Work appears to have greatest effect on course participants’ self-efficacy and confidence in finding work with a range of statistically significant impacts after six months, with some continuing at 12 months. There is less statistically significant evidence of Group Work having an impact on job search activity, although some non-significant findings also point in a positive direction. In summary:

• Submitting CVs - those attending the Group Work course were statistically significantly more likely to have submitted more CVs in the previous fortnight, at six and 12-months post-baseline, compared to the matched comparison group. At six months, 28 per cent of course participants had submitted 10 or more CVs compared to 16 per cent of the matched comparison group, and at 12 months these figures were 26 per cent and 18 per cent respectively.

• Vacancies applied for and attending training and courses - there is a similar pattern of results in terms of vacancies applied for and attending training and courses, although the differences between the course participants and matched comparison are not statistically significant at six or 12 months.
• Job search activity - using the Finnish Institute of Occupational Health Job Seeking Activity Scale^{[footnote 36]} to categorise benefit claimants by level of job search activity - no statistically significant impact on job search was identified at six or 12 months. In addition to increasing job search activity, the course also sought to increase people’s job search self-efficacy and confidence that they can find work. Six months after baseline (but not at 12 months), the course appeared to provide its participants with a level of confidence about their capacity to find work not apparent among the matched comparison group, with large and statistically significant impacts across a number of measures. In summary:
• General self-efficacy - measured using the General Self Efficacy scale, six months after baseline, 60 per cent of course participants had higher levels of self-efficacy, statistically significantly greater than for the matched comparison group (47 per cent). This suggests that six months after the course participants were more likely to perceive themselves as being able to effectively handle situations than their matched comparison group.
• Job search self-efficacy – was measured using the Job Search Self Efficacy Index - Modified, using both the mean scores and a binary outcome to divide people by higher and lower levels of job search self-efficacy. The proportion of course participants rated as having a higher level of job search self-efficacy rose substantially from 31 per cent at baseline to 58 per cent at six months (vs. 31 per cent and 36 per cent for the comparison group), a statistically significant difference between the groups at six months.
• Value of personal qualities and experience - the percentages of course participants “agreeing strongly” or “agreeing” to two statements about the value of their personal qualities and their experience were substantially and significantly statistically higher six months after baseline than in the matched comparison group. 70 per cent of participants and 59 per cent of the matched comparison group agreed that “my personal qualities make it easy to get a new job” at six months, while 61 per cent compared to 46 per cent agreed that “my experience is in demand in the labour market”.
• Confident that they find work within 13 weeks – course participants were also substantially and statistically significantly more likely to be confident that they will find work within the next 13 weeks. Six months after baseline, 40 per cent of participants were confident compared to 27 per cent of the matched comparison group. However, when asked what they felt plays the greatest role in securing a job, the differences between course participants and the matched comparison group in the proportions who felt that it was mainly down to their own job search effort, fixed effects like education or experience, or things outside of their control (e.g. luck or who you know) were close to, but not reaching, statistical significance.

However, with the exception of levels of job search self-efficacy, by 12 months these statistically significant differences are no longer evident. In the main, the gap between the course participants and the matched comparison narrowed between six and 12 months, largely due to improvements among the matched comparison group. However, for job search self-efficacy, there is still a statistically significant impact at 12 months, with 57 per cent of course participants compared to 45 per cent of the matched comparison group scoring as having higher levels of job search self-efficacy. This suggests that Group Work has a sustained effect on participants’ levels of confidence in their job search abilities.

Table 5.2: Impact on job search activity outcomes: impact on participants

	Participants % (randomisation / baseline)	Comparison group % (randomisation / baseline)	p-value (randomisation / baseline)	Participants % (6-month follow-up)	Comparison group % (6-month follow-up)	p-value (6-month follow-up)	Participants % (12-month follow-up)	Comparison group % (12-month follow-up)	p-value (12-month follow-up)
Job search activity scale in past fortnight						0.437			0.293
Paid work 30 hours or more	no data	no data		10	9		11	7
Higher levels	no data	no data		40	43		36	40
Lower levels	no data	no data		39	33		41	38
No job search	no data	no data		11	15		12	15
Number of vacancies applied for in past fortnight						0.078			0.297
Paid work 30 hours or more	no data	no data		10	9		11	7
10 vacancies or more	no data	no data		37	28		38	34
Fewer than 10 vacancies	no data	no data		29	28		25	29
No vacancies applied for	no data	no data		24	34		26	31
Number of CVs submitted in past fortnight(3)						0.017*			0.031*
% In paid work 30hrs+	no data	no data		10	9		11	7
% 10 CVs or more	no data	no data		28	16		26	18
% Fewer than 10 CVs	no data	no data		29	34		27	27
% No CVs	no data	no data		33	41		36	49
Gaining experience
Attending training/courses	no data	no data		53	45	0.079	42	33	0.083
Voluntary work	no data	no data		26	26	0.994	28	21	0.127
Work placements	no data	no data		13	9	0.12	11	9	0.521
General self-efficacy scale (1 to 5)(2)
Mean score (lower score, higher self-efficacy)	2.6	2.5	0.273	2.3	2.6	0.003*	2.3	2.4	0.381
% With higher self-efficacy	42	46	0.368	60	47	0.005*	59	52	0.521
% With lower self-efficacy	58	54		40	53		41	48
Job search self-efficacy scale (1 to 5)(2)
Mean score (higher score greater self-efficacy)	3.3	3.4	0.759	3.8	3.4	0.000*	3.8	3.5	0.001*
% With higher job search self-efficacy	31	31	0.823	58	36	0.000*	57	45	0.027*
% Agree personal qualities will help get work(1)	49	47	0.529	70	59	0.013*	69	60	0.072
% Agree their experience is in demand (1)	38	35	0.507	61	46	0.001*	58	54	0.421
Confidence in finding job			0.469			0.001*			0.376
% In work including voluntary	n/a	n/a		27	24		30	25
% Confident will find a job	50	54		40	27		33	31
% Not confident will find a job	50	46		33	50		37	44
Factors affecting job search success¹			0.873			0.073			0.205
% Job search effort	23	21		29	20		26	24
% Fixed effects	55	57		42	49		44	52
% Things outside my control	22	22		29	30		30	24
Base: all	609	533		609	533		510	362

Source: Survey data (in the category description 1 denotes the first wave of data comes from the randomisation survey and 2 denotes baseline survey). 3 The participant baseline survey contained high levels of missing data on this question and we are therefore unable to report baseline figures.

5.2.3 Wellbeing outcomes

In addition to examining whether Group Work helped people into work, or moving them towards paid employment, the evaluation also explored whether Group Work improved people’s well-being. This section reports on three relevant measures: the ONS4 Wellbeing questions, the UCLA Loneliness Scale and the Latent and Manifest Benefits (LAMB) scale, the results of which are in Tables 5.3 and 5.4. The scales used are described in Chapter 1, and in more detail in Section 3.4 of the Technical Report of the Impacts of the Trial.

Wellbeing outcomes

Comparing course participants against the matched comparison group, statistically significant impacts were detected on participants’ levels of wellbeing at six months, on all these outcomes except for the ONS anxiety measure. However, with the exception of levels of happiness using the ONS scale, none of these statistically significant impacts are present 12 months after baseline.

Table 5.3: Impact of Group Work on wellbeing outcomes, impact on participants

	Participants (randomisation / baseline)	Comparison group (randomisation / baseline)	p-value (randomisation / baseline)	Participants (6-month follow-up)	Comparison group (6-month follow-up)	p-value (6-month follow-up)	Participants (12-month follow-up)	Comparison group (12-month follow-up)	p-value
ONS measures (0-10)(1)
% satisfied with life	29	27	0.494	48	34	0.002*	49	44	0.315
% life worthwhile	41	43	0.724	54	38	0.001*	54	44	0.051
% happier	40	40	0.904	55	37	0.000*	57	48	0.068
% anxious	28	25	0.447	29	25	0.345	27	34	0.124
Mean scores[footnote 37]
Life satisfaction	5.3 (sd 2.2)	5.1 (sd 2.4)	0.475	6 (sd 2.6)	5.4 (sd 2.4)	0.003*	6.2 (sd 2.5)	6 (sd 2.4)	0.331
Life worthwhile	5.8 (sd 2.3)	6 (sd 2.4)	0.514	6.3 (sd 2.5)	5.7 (sd 2.5)	0.007*	6.4 (sd 2.6)	6.1 (sd 2.4)	0.252
Happiness	5.6 (sd 2.5)	5.6 (sd 2.6)	0.846	6.3 (sd 2.8)	5.4 (sd 2.7)	0.000*	6.5 (sd 2.7)	5.8 (sd 2.7)	0.013*
Anxiety	3.8 (sd 2.9)	3.5 (sd 2.9)	0.304	3.8 (sd 3.1)	3.6 (sd 2.9)	0.387	3.7 (sd 3.0)	3.9 (sd 3.2)	0.576
UCLA measure (3-9)(2)
% lonely	47	50	0.52	46	55	0.041*	48	51	0.484
Mean score (higher=lonelier)	5.5 (sd 1.9)	5.5 (sd 1.8)	0.968	5.4 (sd 2.0)	5.7 (sd 2.0)	0.098	5.4 (sd 2.0)	5.6 (sd 1.9)	0.254
Base: all	609	533		609	533		510	362

Source: Survey data (in the category description 1 denotes the first wave of data comes from the randomisation survey and 2 denotes baseline survey).

There is a pattern of positive statistically significant results six months after baseline across the three ONS wellbeing measures of life satisfaction, feeling worthwhile and being happy:

• Nearly half (48 per cent) of participants reported at six months that they were satisfied with their lives compared to 34 per cent of the matched comparison group;
• Similarly, 54 per cent perceived life as being worthwhile compared to 38 per cent of the matched comparison; and
• The comparable percentages on happiness were 55 and 37 per cent.

However, the positive differences in the percentages of course participants and the matched comparison group feeling satisfied, worthwhile and happy are no longer statistically significant 12 months after baseline, although the differences between the two groups in terms of the proportions feeling happy and feeling life is worthwhile are close to significance. The gap between the two groups reduces, largely through improvements in the matched comparison group. However, the statistically significant mean score difference on the happiness scale is still evident 12 months after baseline, with participants having a mean score of 6.5 against 5.8 among the matched comparison group.

Six months after baseline, course participants were also statistically significantly less likely than the matched comparison group to rate as being lonely on the UCLA scale. Forty-six per cent of course participants scored as lonely compared to 55 per cent.

Latent and Manifest Benefits

The Latent and Manifest Benefits (LAMB) scale measures the perceived psychosocial benefits to someone of being in employment (that is, how they perceive their psychosocial environment, such as social support, activity, time structure and routine), Table 5.4 shows the overall LAMB scores for the Group Work and matched comparison groups, with their scores on two sub-scales which measure individuals’ levels of psychosocial deprivation and their level of financial strain.

There is a statistically significant difference at six months on the overall LAMB score measuring people’s perceptions of the benefits of work. Looking at the standard four-category LAMB outcome (where a lower score denotes a better LAMB score), 15 per cent of course participants scored in the lowest (best) category compared to seven per cent of the matched comparison group, although the mean score difference between the two groups is not statistically significant. This is likely to be due to the movement being mainly between the first two categories rather than across the whole scale. In other words, participants appear to show a stronger belief in the psychological and financial benefits of work than the matched comparison group. A similar pattern is found 12 months after baseline, but smaller and not statistically significant.

Although there is no statistically significant evidence that Group Work has an impact on people’s levels of psychosocial deprivation and financial strain, using the two separate sub-scales the differences between course participants and the matched comparison on the groupings for the psychological deprivation score (indicating someone’s perceived psychological benefits of work) are close to statistical significance. However, the picture is mixed, with course participants being more likely than the matched comparison group to be both in the lowest (i.e. best) and highest (i.e. worst) scoring groups.

Table 5.4: Impact of Group Work on the Latent and Manifest Benefits scale: impact on participants

	Participants % (baseline)	Comparison group % (baseline)	p-value (baseline)	Participants % (6-month follow-up)	Comparison group % (6-month follow-up)	p-value (6-month follow-up)	Participants % (12-month follow-up)	Comparison group % (12-month follow-up)	p-value (12-month follow-up)
Overall scale (0 to 60, lower score better)
Mean score	31.5 (sd 8.9)	31.5 (sd 9.7)	0.964	30.5 (sd 12.4)	30.4 (sd 10.7)	0.968	30.1 (sd 12.4)	30.4 (sd 10.9)	0.781
Score 0 to 14	3	3	0.981	15	7	0.019*	14	11	0.622
Score 15 to 29	38	38		27	37		28	33
Score 30 to 44	52	51		47	45		48	47
Score 45 to 60	7	7		12	11		9	10
Psychosocial deprivation scale (0 to 50, lower score better)
Mean score	24.9 (sd 9.0)	25.2 (sd 9.7)	0.739	24.3 (sd 12.0)	24.2 (sd 10.3)	0.875	24 (sd 12.1)	24.2 (sd 10.8)	0.858
Low	27	30	0.658	33	30	0.098	35	33	0.541
Medium	58	54		45	54		45	51
High	14	16		21	15		20	17
Financial strain score ( 0 to 10,lower score better)
Mean score	6.7 (sd 2.8)	6.7 (sd 3.1)	0.875	6.3 (sd 3.5)	6.4 (sd 3.1)	0.696	6.3 (sd 3.4)	6.4 (sd 3.2)	0.784
Low	14	14	0.768	23	23	0.815	23	21	0.918
Medium	42	39		29	32		32	32
High	44	47		47	45		45	46
Base: all	609	533		609	533		510	362

Source: Survey data.

5.2.4 Mental health outcomes

The evaluation also examined whether Group Work had a positive impact in terms of improving people’s mental health, either by addressing anxieties and concerns about job search or by helping them enter paid work (with its known associations with improved mental wellbeing). The measures used are described in Chapter 1, and in more detail in Section 3.5 of the Technical Report of the Impact of the Trial, and the findings shown in Table 5.5.

Six months after baseline, course participants were statistically significantly less likely than the matched comparison group to score as having likely depression or poor wellbeing on the WHO-5 scale (49 per cent compared to 59 per cent). At 12 months after baseline there is still a positive percentage point difference in those with likely depression or poor wellbeing, but it is smaller (50 per cent compared to 55 per cent) and no longer statistically significant.

Table 5.5: Impact of Group Work on mental health outcomes: impact on participants

	Participants (baseline)	Comparison Group (baseline)	p-value (baseline)	Participants (6-month follow-up)	Comparison group (6-month follow-up)	p-value (6-month follow-up)	Participants (12-month follow-up)	Comparison group (12-month follow-up)	p-value (12-month follow-up)
WHO-5 wellbeing (score 0-25) (2)
Mean score (higher=better)	11.7 (sd 5.8)	12.1 (sd 6.3)	0.505	12.7 (sd 6.7)	11.3 (sd 6.4)	0.016*	12.6 (sd 6.7)	11.3 (sd 7.1)	0.094
% with likely depression /impaired wellbeing	54	59	0.33	49	59	0.029*	50	55	0.318
WHO-5 wellbeing categories(2)	%	%	0.481	%	%	0.089	%	%	0.591
Likely depression	31	31		33	40		35	39
Poor wellbeing	23	28		15	19		14	16
Good wellbeing	46	41		51	41		50	45
PHQ-9 depression scale (score 0 to 27, lower score better)
Mean score	9.6 (sd 7.1)	9.7 (sd 7.5)	0.907	7.7 (sd 7.6)	8.4 (sd 7.1)	0.26	7.9 (sd 7.4)	8.3 (sd 7.6)	0.577
% depression level suggesting caseness	44	45	0.928	32	36	0.428	33	35	0.684
PHQ-9 depression categories	%	%	0.971	%	%	0.153	%	%	0.576
None	31	30		48	38		43	42
Mild	25	25		20	27		24	23
Moderate	19	17		12	15		10	13
Moderately severe	13	14		10	10		12	9
Severe	12	14		10	10		11	13
GAD-7 anxiety scale (score 0 to 21, lower score better)
Mean score	8.1	8.5 (sd 6.3)	0.564	7 (sd 6.7)	7.8 (sd 6.3)	0.168	7 (sd 6.6)	7.8 (sd 6.6)	0.233
	%	%		%	%		%	%
% anxiety levels suggesting caseness	49	50	0.771	39	48	0.051	40	45	0.347
GAD-7 anxiety categories	%	%	0.812	%	%	0.293	%	%	0.628
None	32	33		47	40		47	43
Mild	29	25		21	27		21	19
Moderate	23	23		13	15		14	18
Severe	16	19		18	18		19	20
Base: all	609	533		609	533		510	362

Source: Survey data.

Whilst there is the same pattern of positive results for the PHQ-9 measure of depression, compared with the WHO-5 findings the differences between course participants and the matched comparison group are not as large and not statistically significant at six or 12 months after baseline. This may be because while WHO-5 and PHQ-9 have both been shown to be valid and reliable screening tools for depression (Levis et al., 2019), the items in the shorter WHO-5 are phrased positively or neutrally whereas in PHQ-9 they are presented as problems (with negative phrasings and associated connotations). This may influence how individuals respond to the items, with research (Henkel et al., 2003) suggesting that the WHO-5 is a better screening tool for depression in primary care settings.

Six months after baseline, 39 per cent of course participants and 48 per cent of the matched comparison group reported anxiety levels on the GAD-7 scale which suggested caseness (i.e. their GAD-7 score suggests they would exceed the ‘caseness thresholds’ used by NHS England’s Improving Access to Psychological Therapies service)^{[footnote 38]}. This substantial difference is very close to, but just above the ceiling of, statistical significance. The mean score difference at six months between the two groups is positive but not statistically significant, nor are the positive, but smaller, differences observed after 12 months.

5.2.5 Wider health outcomes

Despite the positive impacts of the course on wellbeing, no statistically significant impacts were detected on participants’ self-reported overall heath (measured using the EQ-5D-3L and EQVAS scales) or their use of health services after six or 12 months. Table 5.6 presents the findings for the EQ-5D and EQVAS measures, and on participant’s use of health services compared to the matched comparison group.  

Table 5.6: Impact of Group Work on wider health outcomes: impact on participants

	Participants (baseline / randomisation)	Comparison group (baseline / randomisation)	p-value (baseline / randomisation)	Participants (6-month follow-up)	Comparison group (6-month follow-up)	p-value (6-month follow-up)	Participants (12-month follow-up)	Comparison group (12-month follow-up	p-value (12-month follow-up
EQ-5D health (2)
EQ Value	0.7 (sd 0.3)	0.7 (sd 0.3)	0.959	0.7 (sd 0.3)	0.7 (sd 0.3)	0.531	0.7 (sd 0.3)	0.7 (sd 0.3)	0.563
EQVAS mean score (higher score better) [footnote 39]	54.2 (sd 27.1)	63.1 (sd 25.1)	0.000* (footnote 39)	65.6 (sd 24.5)	61.6 (sd 25.3)	0.099	64.9 (sd 25.9)	62.1 (sd 27.0)	0.411
	%	%		%	%		%	%
Use of health services (1)
% to GP	27	25	0.748	25	19	0.121	25	23	0.634
% to Casualty or outpatients	19	17	0.491	16	20	0.195	23	17	0.125
Base: all	609	533		609	533		510	362

Source: Survey data (in the category description 1 denotes the first wave of data comes from the randomisation survey and 2 denotes baseline survey).

The EQ-5D provides an overall measure of someone’s health status, while the EQVAS is a self-rated health measure, with people being asked to rate their health from 0 to 100 (see Section 3.6 for more detail).

While no statistically significant impacts were detected under either measure, there were positive differences in the EQVAS mean scores for course participants and the matched comparison group (65.6 versus 61.6 out of 100), which at six months comes close to statistical significance (p=0.099). When people were asked about GP visits within the past two weeks or casualty or hospital outpatient visits in the past three months, no statistically significant impacts were detected.

5.3 Results for participant subgroups

As reported in Chapter 3, the benefit claimants who participated in the Group Work course were diverse both demographically and in terms of their baseline levels of confidence, self-efficacy, wellbeing and mental health. The impact evaluation included extensive subgroup analysis to examine the impacts of the course on groups of participants with these different characteristics. Details of the subgroups examined and approach to the analysis are provided in the Impact Evaluation Technical Report, but in summary for each sub-group propensity score matching was used to create a matched comparison group of individuals with the same baseline characteristics and, as with the main analysis, outcomes at six and 12 months were compared.

From among all the subgroup analyses, a clear pattern emerged across the range of outcome measures, namely that, broadly in line with the international evidence, Group Work had the greatest impact on participants:

• With lower levels of general self-efficacy at baseline;
• With suggested case level anxiety on the GAD-7 scale at baseline; and
• With suggested case level depression on the PHQ-9 scale at baseline.

For participants with low levels of general self-efficacy or suggested case level anxiety at baseline, statistically significant positive impacts were detected at six months for being in paid work, general and job search self-efficacy, mental health outcomes, and sustained for work and self-efficacy at 12 months. The mental health outcome was also sustained at the 12-month point for those with low general self-efficacy at baseline, but not for those with suggested case level anxiety. A similar, but not so pronounced, pattern of statistically significant impacts was detected among those with suggested case level depression at baseline.

5.3.1 Lower levels of general self-efficacy at baseline

After dividing course participants and the matched comparison group into those with higher and lower levels of general self-efficacy at baseline, course participants with lower baseline general self-efficacy had statistically significantly better outcomes than their matched comparison group at six and 12 months after baseline (see Table 5.7).

After six months, in comparison to the matched comparison group, participants with lower baseline general self-efficacy were:

• Nearly twice as likely to be in paid work (21 per cent compared to 11 per cent), and four times as likely to be in paid work of 30 hours a week or more (eight per cent compared to two per cent);
• More than twice as likely to have higher levels of general and job search self-efficacy, 46 per cent compared to 18 per cent and 46 per cent versus 19 per cent respectively; and
• Less likely to score as having likely depression or poor wellbeing on the WHO-5 scale (57 per cent compared to 83 per cent) or suggested case level anxiety on the GAD-7 (46 per cent versus to 67 per cent).

A similar pattern of results is sustained 12 months after baseline, with continued statistically significant impacts for all but being in paid work (although being in paid work of 30 hours or more remained statistically significant).

Apart from the work outcomes, both the course participants and the matched control group with higher levels of baseline general self-efficacy had better six and 12-month outcomes than those with lower baseline levels - although in contrast to those with lower baseline general self-efficacy Group Work appeared to have very little impact on this group. The only six-month outcome where a statistically significant impact was detected for this group is job search self-efficacy, where 73 per cent of the course participants and 58 per cent of the matched comparison group scored as having higher levels.

No statistically significant impacts were detected among those with higher or lower levels of baseline general self-efficacy on levels of depression measured by the PHQ-9 or on the LAMB scales, although the percentage point differences are positive. Although it is not clear why impact on depression was detected using the WHO-5 scale and not PHQ-9, Section 5.2.4 of this report, and also Section 3.5 of the Impact Evaluation Technical Report, suggest that the positive or neutral phrasing of the items in WHO-5 may influence how individuals respond to them.

Table 5.7: Impact of Group Work on outcomes according to level of general self-efficacy at baseline: Impacts on Participants

	Higher general self-efficacy	Higher general self-efficacy	Higher general self-efficacy	Lower general self-efficacy	Lower general self-efficacy	Lower general self-efficacy		Higher general self-efficacy	Higher general self-efficacy	Higher general self-efficacy	Lower general self-efficacy	Lower general self-efficacy	Lower general self-efficacy
	Participants (six month follow up)	Comp’n group (six month follow up)	p-value (six month follow up)	Participants (six month follow up)	Comp’n group (six month follow up)	p-value (six month follow up)	p-value for differential impact	Participants (12 month follow up)	Comp’n group (12 month follow up)	p-value (12 month follow up)	Participants (12 month follow up)	Comp’n group (12 month follow up)	p-value (12 month follow up)	p-value for differential impact (12 month follow up)
	%	%		%	%			%	%		%	%
A higher % among participants denotes a positive outcome:
% in paid work	19	21	0.72	21	11	0.044*	0.128	29	29	0.981	18	12	0.207	0.002*
% in paid work 30 hours or more	12	14	0.71	8	2	0.030*	0.002*	16	11	0.351	7	2	0.024*	<0.001*
% with higher general self-efficacy	79	82	0.592	46	18	<.001*	0.001*	82	85	0.632	41	19	0.002*	0.012*
% with higher job search self-efficacy	73	58	0.024*	46	19	<.001*	<.001*	69	71	0.82	46	18	<0.001*	<0.001*
% lower LAMB score	51	61	0.23	35	34	0.98	0.019*	53	64	0.163	36	32	0.605	0.001*
% low LAMB psychosocial deprivation score	41	49	0.29	28	19	0.164	0.025*	46	47	0.972	26	18	0.286	0.001*
% low financial LAMB deprivation score	22	28	0.344	24	19	0.436	0.485	28	21	0.39	20	20	0.943	0.037*
A lower % among participants denotes a positive outcome:
% likely depression/poor wellbeing (WHO-5)	37	30	0.38	57	83	<.001*	<.001*	37	31	0.401	59	75	0.040*	0.001*
% depression levels suggesting caseness	21	19	0.668	41	50	0.222	<.001*	23	19	0.585	41	51	0.188	0.023*
% anxiety levels suggesting caseness	29	29	0.96	46	67	0.003*	0.002*	33	22	0.101	45	67	0.007*	0.002*
Base: all	251	282		349	236			215	192		285	159

Source: Survey data.

5.3.2 Suggested case level anxiety at baseline

Participants with suggested case level anxiety at baseline had statistically significantly better outcomes than their matched comparison group six months after baseline (see Table 5.8):

• One in five (20 per cent) were in paid work compared to 10 per cent of the matched comparison group (with nine per cent and three per cent respectively being in work of 30 hours a week or more);
• They had twice the likelihood of having higher levels of general self-efficacy than their matched comparison group (49 per cent compared to 24 per cent) and higher levels of job search self-efficacy (46 per cent versus 27 per cent); and
• They were less likely than the matched comparison group to score as having likely depression or poor wellbeing on the WHO-5 scale (64 per cent compared to 84 per cent) or suggested case level anxiety on the GAD-7 (60 per cent compared to 79 per cent).

For those with suggested case level anxiety at baseline, although the percentage point differences continue to be positive (and indeed larger than at six months), the impact falls just beneath statistical significance on being in any paid work 12 months after baseline. The impacts on mental health and wellbeing are also not sustained. However, 12 months after baseline, course participants with suggested case level anxiety were significantly more likely to be in paid work of 30 hours or more (12 per cent versus five percent for the matched comparison group) and to have higher levels of general (50 per cent versus 33 percent) and job search self-efficacy (48 per cent versus 27 per cent).

With the exception of the work outcomes, those with lower levels of baseline anxiety had better six and 12-month outcomes than those with suggested case level baseline anxiety, for both course participants and those in the matched comparison group. However, among this sub-group and in contrast to those with suggested case level anxiety levels at baseline, Group Work appeared to have very little impact. As with the higher general self-efficacy group, the only six-month outcome showing a statistically significant impact among those with lower levels of baseline anxiety is job search self-efficacy (69 per cent of course participants and 44 per cent of the matched comparison group scored as having higher levels).

Again, although the percentage point differences between course participants and the matched comparison group are positive, there are no statistically significant impacts either among those with and without suggested case level anxiety at baseline on levels of depression measured by the PHQ-9 or on the LAMB scales.

Table 5.8: Impact of Group Work on outcomes according to level of anxiety at baseline: Impacts on Participants

	Case level anxiety	Case level anxiety	Case level anxiety	Not case level anxiety	Not case level anxiety	Not case level anxiety		Case level anxiety	Case level anxiety	Case level anxiety	Not case level anxiety	Not case level anxiety	Not case level anxiety
	Participants (six-month follow up)	Comp’n group (six-month follow up)	p-value (six-month follow up)	Participants (six-month follow up)	Comp’n group (six-month follow up)	p-value (six-month follow up)	p-value for differ’ial impact (six-month follow up)	Participants (12 month follow up)	Comp’n group (12 month follow up)	p-value (12 month follow up)	Participants (12 month follow up)	Comp’n group (12 month follow up)	p-value (12 month follow up)	p-value for differ’ial impact (12 month follow up)
	%	%		%	%			%	%		%	%
A higher % among participants denotes a positive outcome
% in paid work	20	10	0.023*	21	23	0.641	0.030*	24	13	0.054	22	25	0.561	0.13
% in paid work 30 hours or more	9	3	0.023*	10	14	0.394	0.007*	12	5	0.050*	10	8	0.575	0.646
% with higher general self-efficacy	49	24	<.001*	70	65	0.505	<.001*	50	33	0.017*	67	58	0.272	0.134
% with higher job search self-efficacy	46	27	0.004*	69	44	0.001*	<.001*	48	27	0.004*	66	59	0.401	<0.001*
% lower LAMB score	27	34	0.366	56	62	0.405	<.001*	34	33	0.888	50	58	0.391	0.037*
% low LAMB psychosocial deprivation score	22	23	0.863	45	40	0.521	0.005*	25	25	0.961	44	37	0.434	0.039*
% low financial LAMB deprivation score	20	20	0.97	27	25	0.758	0.224	17	17	0.89	29	24	0.413	0.005*
A lower % among participants denotes a positive outcome
% likely depression/poor wellbeing (WHO-5)	64	84	<.001*	33	32	0.89	<.001*	63	74	0.123	36	36	0.988	0.006*
% depression levels suggesting caseness	51	59	0.254	14	11	0.433	0.001*	50	58	0.298	16	13	0.453	0.021*
% anxiety levels suggesting caseness	60	79	0.001*	19	15	0.442	0.005*	59	72	0.069	22	16	0.284	0.045*
Base: all	289	290		300	230			247	198		247	156

Source: Survey data.

5.3.3 Suggested case level depression at baseline

When course participants and the matched control group are divided into those whose baseline scores on the PHQ-9 suggest that they have or do not have suggested case level depression, there was little statistically significant evidence of Group Work having a differential impact on whether course participants were in paid work across those who did or did not have suggested case level depression at baseline (see Table 5.9).

There were no statistically significant impacts six months after baseline or on the overall measure of ‘being in paid work’ after 12 months. Being in paid work of 30 hours or more a week was the one outcome for which there was a statistically significant impact among those with suggested case level baseline depression 12 months after baseline, with 12 percent of former participants working 30 or more hours a week compared to three per cent of the comparison group.

With the exception of impact on paid work, the pattern of statistically significant results across those who do or do not have suggested case level baseline depression is very similar to those with higher and lower levels of self-efficacy and anxiety – unsurprising given the overlaps between the groups. Among those with suggested case level depression at baseline statistically significant impacts – at six and 12 months after baseline – were detected on their levels of general and job search self-efficacy, depression/wellbeing (as measured by the WHO-5 scale) and anxiety (GAD-7). When participants with suggested case level depression at baseline are compared to those in the matched comparison group:

• Twice as many scored higher levels of general self-efficacy after six months (52 per cent compared to 22 per cent) and 12 months (50 per cent compared to 32 per cent) as those in the matched comparison group.
• Almost half (47 per cent) had higher levels of job search self-efficacy after six months compared to 20 per cent of the matched comparison group, with the percentages after 12 months close to identical to those at six months.
• Two thirds (65 per cent) scored as having higher depression/poor wellbeing after six months compared to 86 per cent of the matched comparison group, with similarly statistically significant results after 12 months.
• 60 per cent scored as having suggested case level anxiety after six months, versus 77 per cent of the matched comparison group, again with statistically significant impacts sustained after 12 months.

As with the comparison between those with higher and lower levels of self-efficacy and anxiety, with the exception of the work outcomes, those with lower levels of baseline depression had better six and 12-month outcomes than those with suggested case level baseline depression (reflecting their baseline differences), whether a course participant or in the matched comparison group.

Table 5.9: Impact of Group Work on outcomes according to level of depression at baseline: Impacts on Participants

	Case level depression	Case level depression	Case level depression	Not case level depression	Not case level depression	Not case level depression		Case level depression	Case level depression	Case level depression	Not case level depression	Not case level depression	Not case level depression
	Participants (six month follow up)	Comp’n group (six month follow up)	p-value (six month follow up)	Participants (six month follow up)	Comp’n group (six month follow up)	p-value (six month follow up)	p-value for differ’ial impact (six month follow up)	Participants (12 month follow up)	Comp’n group (12 month follow up)	p-value (12 month follow up)	Participants (12 month follow up)	Comp’n group (12 month follow up)	p-value (12 month follow up)	p-value for differ’ial impact (12 month follow up)
	%	%		%	%			%	%		%	%
A higher % among participants denotes a positive outcome
% in paid work	20	13	0.181	20	20	0.977	0.398	21	13	0.133	24	26	0.767	0.116
% in paid work 30 hours or more	10	5	0.178	9	12	0.592	0.22	12	3	0.016*	11	9	0.669	0.231
% with higher general self-efficacy	52	21	<.001*	70	62	0.231	<.001*	50	32	0.021*	69	55	0.118	0.028*
% with higher job search self-efficacy	47	20	<.001*	69	49	0.005*	<.001*	45	20	<0.001*	67	63	0.587	<0.001*
% lower LAMB score	28	36	0.337	52	57	0.528	0.007	32	29	0.745	52	56	0.635	0.001*
% low LAMB psychosocial deprivation score	24	22	0.777	42	39	0.686	0.221	26	22	0.632	44	38	0.438	0.007*
% low financial LAMB deprivation score	20	16	0.591	26	26	0.96	0.194	22	14	0.185	25	22	0.642	0.316
A lower % among participants denotes a positive outcome
% likely depression / poor wellbeing (WHO-5)	65	86	<.001*	34	36	0.773	<.001*	64	79	0.037*	37	36	0.836	<0.001*
% depression levels suggesting caseness	55	61	0.475	14	15	0.728	0.822	54	65	0.155	14	11	0.31	0.086
% anxiety levels suggesting caseness	60	77	0.007*	22	22	0.967	0.001*	57	74	0.045*	23	17	0.273	<0.001*
Base: all	258	245		319	260			277	167		255	178

Source: Survey data.

However, again mirroring the findings from the other two groups, Group Work appeared to have very little impact on those who do not exhibit suggested case level baseline depression. The only six-month outcome on which there is a statistically significant impact of Group Work among those with lower levels of baseline depression is job search self-efficacy, where 69 per cent of the participants and 49 per cent of the matched comparison group scored as having higher levels. There are no statistically significant differences 12 months after baseline.

Again, there is no evidence of statistically significant impacts either among those with and without suggested case level depression at baseline on levels of depression measured by the PHQ-9 or on the LAMB scales.

6. The costs and benefits of Group Work

This chapter summarises the findings from the cost benefit analysis (CBA) of Group Work, which aimed to examine the costs and the benefits of the course, quantify them in monetary terms as far as possible, and establish the balance between the costs and the benefits identified. The full findings are detailed in the CBA technical report. The CBA drew primarily on cost data provided by the Department for Work and Pensions (DWP) and the impact evaluation findings reported in Chapter 5.

The chapter is structured around the following sections:

• The costs of delivering Group Work;
• The monetised benefits of Group Work; and
• Comparison of costs and monetised benefits (for all participants and subgroups).

6.1 Costs of delivering Group Work

6.1.1 Total costs

The total cost of delivering Group Work was projected to amount to £3.4 million in the Business Case prepared in 2016. The actual costs are estimated to be £3.3 million. These costs are roughly evenly split between intervention-specific costs (£1.7 million) and trial-specific costs (£1.6 million).

Table 6.1 provides a breakdown of the estimated costs. It includes all known costs relating to developing, delivering and trialling Group Work, but excludes policy development costs not directly related to the programme. The staff costs include pensions, National Insurance and overheads. All costs are assumed to be additional costs relative to a Business as Usual scenario.

Table 6.1: Estimated costs of delivering Group Work

Cost item	Delivery cost	Trial cost
Staffing costs
DWP and Jobcentre Plus staff	£660,280	£713,352
Contractors
Intervention contract	£1,042,000
Evaluation contract		£896,670
Total contractor costs	£1,042,000	£896,670
Total costs	£1,702,280	£1,610,022

6.1.2 Unit costs per participant

If trial-specific costs are excluded, and only the costs of administering and delivering the intervention are considered, the unit cost of the Group Work course is £656 per participant starting the course, and £886 per participant completing it.

It is assumed that trial-specific costs would not be incurred in the event of any future delivery of Group Work. The following findings in this chapter are therefore based on delivery costs only.

6.2 The monetised benefits of Group Work

Benefits of Group Work were expected to include:

• Improvements in participants’ mental health and wellbeing as a direct result of participating in Group Work; and
• Benefits to participants, employers, the Exchequer and the economy where participation in Group Work leads to employment outcomes.

A central assumption of the CBA was that the intervention delivered benefits only if these were found to be statistically significant at the five per cent level.

6.2.1 The value of mental health and wellbeing benefits

As reported in Chapter 5, some statistically significant impacts on participants’ mental health and wellbeing were detected six and/or twelve months after their participation on the course. These included increases in wellbeing as measured on the WHO-5 scale and on some of the ONS wellbeing measures. The impact evaluation also found some benefits for job search related outcomes such as participant confidence in finding work and general and job-search self-efficacy, suggesting it had helped to move participants closer to the labour market, though these effects were stronger at 6 months than at 12 months after completion of the trial.

However, there is currently no evidence-base to enable the value of improvements on these measures to be quantified robustly in monetary terms.

No statistically significant impacts were detected on the two health indicators for which it is possible to quantify benefits robustly in money terms: overall health status (as measured through EQ-5D scores) and health service usage.

Therefore, the health benefits delivered could not be valued in monetary terms.

6.2.2 The value of employment benefits

A slightly higher proportion of Group Work participants were in work six months and 12 months after completing the course compared to members of the comparison group. However, these differences were not statistically significant.

An upper estimate can be made by valuing the observed differences in employment between Group Work participants and the comparison group, assuming that these are real differences even though they are not statistically significant.

Table 6.3 estimates the additional effects in terms of the additional days that each participant spent in work per financial year, and the value of this additional employment in terms of enhanced output. A lower estimate was made based on an assumption that these effects would last up to 18 months following the course, and an upper estimate that these effects would last for up to 24 months.

Table 6.3: Estimated effects of Group Work on days in work and value of output

Estimated additional days in employment per participant per year

Financial year	Lower estimate	Upper estimate
2017/18	1.7	1.7
2018/19	8.1	10.1
2019/20	0	1.4

Estimated present value of additional output per participant

Financial year	Lower estimate	Upper estimate
2017/18	£390	£528
2018/19
2019/20

^{[footnote 40]}

The impact evaluation found that the weekly benefits paid to Group Work participants were an average of £0.10 lower than those to the comparison group six months after completion of the course, and £2.30 lower at the twelve-month stage. However, these differences were not statistically significant.

As with employment outcomes, an upper estimate can be made assuming that these are real differences even though they are not statistically significant. A lower estimate was made based on an assumption that the effects on benefits payments would last up to 18 months following the course, and an upper estimate that these effects would last for up to 24 months, as shown in Table 6.4.

Table 6.4: Estimated effects of Group Work on reductions in benefit payments

Estimated reduction in benefits payments per participant per year

Financial year	Lower estimate	Upper estimate
2017/18	£1.52	£1.52
2018/19	£60.88	£78.33
2019/20	£0	£12.46

6.3 Comparison of the costs and monetised benefits of Group Work

6.3.1 All participants

The estimated costs and monetised benefits of Group Work were inputted into the DWP Social Cost Benefit Analysis (SCBA) model in order to assess the overall balance of costs and benefits for all participants in the course. The model provides an analysis of costs and benefits at four levels: society; participants; participants’ employers; and the Exchequer.

The assessment was conducted at three levels:

• Main assessment - this applies the best estimates of costs and benefits, assuming no benefit for employment (on the basis that no statistically significant impact was found);
• Positive employment effect - this assessment applies the best estimate of costs but assumes a positive effect on employment (i.e. applying estimates of increased numbers of days in work, even though these were not statistically significant); and
• Positive employment effect, lower cost estimate - this assessment assumes a positive effect on employment and a reduced estimate of costs from the sensitivity analysis. It therefore presents a best-case scenario for both benefits and costs.

In each case the analysis focused on the costs of the Group Work programme and excluded the trial-related costs. Tables 6.5 to 6.8 provide an overview of the assessment results in terms of costs and benefits for society, participants, participants’ employers and the Exchequer respectively. All figures are presented as a present value using a discount rate of 3.5 per cent specified in the HMT Green Book (HM Treasury, 2018) and applied by the DWP SCBA model. The Benefit cost ratio shows the potential return for each £1 invested.

Table 6.5: Society cost benefit analysis for Group Work

	Main assessment	Positive employment effect	Positive employment effect, lower cost estimate
Total benefits	£0	£1,042,929	£1,042,929
Total costs	£1,646,463	£1,695,337	£1,564,766
Net benefit	-£1,646,463	-£652,408	-£521,837
Benefit Cost Ratio	0	0.62	0.67

Table 6.6: Cost benefit analysis for participants

	Main assessment	Positive employment effect	Positive employment effect, lower cost estimate
Total benefits	£0	£1,026,086	£1,026,086
Total costs	£0	£820,148	£820,148
Net benefit	£0	£205,938	£205,938
Benefit Cost Ratio	0	1.25	1.25

^{[footnote 41]}

	Main assessment	Positive employment effect	Positive employment effect, lower cost estimate
Total benefits	£0	£1,026,086	£1,026,086
Total costs	£0	£820,148	£820,148
Net benefit	£0	£205,938	£205,938
Benefit Cost Ratio	0	1.25	1.25

Table 6.7: Cost benefit analysis for employers of Group Work participants

	Main assessment	Positive employment effect	Positive employment effect, lower cost estimate
Total benefits	£0	£1,013,850	£1,013,850
Total costs	£0	£1,013,850	£1,013,850
Net benefit	£0	£0	£0
Benefit Cost Ratio	0.00	1.00	1.00

Table 6.8: Exchequer cost benefit analysis

	Main assessment	Positive employment effect	Positive employment effect, lower cost estimate
Total benefits	£0	£958,523	£958,523
Total costs	£1,646,463	£1,722,542	£1,591,971
Net benefit	-£1,646,463	-£764,019	-£633,448
Benefit Cost Ratio	0.00	0.55	0.60

These results show that, overall and for society as a whole, the costs of delivering Group Work outweighed the value of the monetised benefits, even employing best case assumptions for employment effects and costs. However, to reiterate, this analysis does not reflect the statistically significant benefits to mental health, wellbeing, confidence or self-efficacy detected in the impact evaluation, which could not be valued in money terms.

6.3.2 Subgroup analysis

Although the impact evaluation did not detect statistically significant employment benefits for the total population of Group Work participants, these were detected for two subgroups:

• Those with low general self-efficacy at baseline; and
• Those with suggested case level anxiety (on the GAD-7 scale) at baseline^{[footnote 42]}.

Table 6.9 provides the estimated costs and benefits to society per participant for the each of the groups. It is assumed that the average costs per participant in these groups is the same as those per participant in Group Work overall.

Table 6.9: Society cost benefit analysis of Group Work per participant

	Low general self-efficacy at baseline	Suggested case level anxiety at baseline
Total benefits	£1,271	£1,647
Total costs	£916	£934
Net benefit	£355	£712
Benefit Cost Ratio	1.39	1.76

This shows that the monetised benefits were found to exceed the intervention costs for both groups. A sensitivity analysis showed that this finding was unchanged even if higher estimates of costs and a shorter duration of benefits were assumed. This suggests that the intervention could deliver net benefits to society if targeted at these groups. This finding relies on the (strong) assumption that the costs of delivery to these groups can be held at the average intervention cost of Group Work, and that similar levels of benefits can be maintained. It also does not reflect the statistically significant benefits to mental health, wellbeing and job search related outcomes detected by the impact evaluation.

The impact evaluation also examined the effects on a third sub-group – those with higher levels of depression at baseline – but did not find a significant effect on the numbers in paid work at six or twelve months. The CBA did not therefore estimate the benefits for this sub-group.

7. Conclusions and recommendations

This chapter presents the conclusions and recommendations of the evaluation, and the lessons learnt for wider provision, drawn from across the three strands of research.

7.1 Conclusions

Our conclusions are structured by the research aims and key areas of investigation as follows:

• Did Group Work improve benefit claimants’ employment rates and wellbeing?
• For whom was the course most effective and why?
• Was the support cost effective?
• Was the Group Work delivered as intended, and what worked well and less well in its delivery?
• What were the experiences of provider staff delivering the course and Work Coaches making referrals to it?
• What were the experiences of benefit claimants participating in the course, the reasons why some declined to attend or did not complete the course?

7.1.1 The impact of Group Work

Looking across all course participants, while no statistically significant evidence was found of Group Work having an impact on participation in paid work, there are statistically significant positive impacts at six months after course participation across a range of mental health, wellbeing and self-efficacy measures, including confidence in finding paid work. There is also pattern of positive but not statistically significant difference between the outcomes for participants and the matched comparison group, although for the most part the statistically significant differences at six months are no longer statistically significant at 12 months after course completion.

Importantly, there is no evidence of any negative impacts resulting for individuals attending a Group Work course, with the observed outcomes for participants almost always being the same or better, on average, than the outcomes for the matched comparison group.

Results for all participants

The findings on the impact of the course across all participants are summarised as:

Work-related outcomes – while positive differences were observed between participants and the matched comparison group, they were not sufficient to reach statistical significance at the six or 12-month points. Positive but not statistically significant change was observed across a range of outcomes, including being in paid work (20 per cent vs 18 per cent in the matched comparison group at six months, and 23 per cent and 20 per cent respectively at 12 months), participants’ job satisfaction and their earnings.

Job search related outcomes – Group Work appeared to have equipped participants with certain job search attributes and behaviours which should increase their likelihood of finding work. The course was found to have the greatest effect on participants’ general and job search self-efficacy and confidence in finding work at six, and in some cases 12, months post-baseline. While there is less statistically significant evidence that Group Work has an impact on job search activity, positive differences with the matched comparison group included the numbers of CVs submitted (where a statistically significant difference was found) and vacancies applied for. Positive results were also found for job search self-efficacy (statistically significant at six and 12 months), and confidence in finding work and the value of their qualities and experience (both statistically significant at six but not 12 months).

Wellbeing outcomes – statistically significant positive impacts were detected on participants’ wellbeing at six months on a range of outcomes (including the ONS wellbeing measures of life satisfaction, feeling worthwhile and happiness, and the UCLA scale for loneliness), although only the ONS happiness mean score remained statistically significant at the 12-month point. The differences between the participants and matched control group reduced largely due to improvements in the matched comparison group. There was also a statistically significant difference at six months on the overall Latent and Manifest Benefits (LAMB) score on perceptions of the benefits of work, although this difference was smaller, and so not significant, at 12 months.

Mental health outcomes – six months after baseline participants were statistically significantly less likely to score as having likely depression or poor wellbeing on the WHO-5 scale, although the difference was smaller at 12-months. Positive but not significant differences between participants and the comparison group were also identified using the PHQ-9 measure of depression – with the difference between the two outcomes possibly being due to the wording of the measures influencing participant responses (WHO-5 being phrased neutrally whereas PHQ-9 items are presented as problems). There was also a positive but not statistically significant difference between participants and the comparison group for suggested case level anxiety (as measured by GAD-7).

Wider health outcomes – while positive impacts were identified for wellbeing, no statistically significant impacts were detected on participants’ self-reported overall health or their use of health services at six or 12 months.

Results for participant sub-groups

Group Work had the greatest impacts on participants with:

• Lower levels of general self-efficacy at baseline;
• Suggested case level anxiety on the GAD-7 scale at baseline; and
• Suggested case level depression on the PHQ-9 scale at baseline.

For those with low levels of general self-efficacy or suggested case level anxiety at baseline, statistically significant positive impacts were detected at six months for being in paid work, general and job search self-efficacy, mental health, and sustained for work and self-efficacy at 12 months. The mental health outcomes were also sustained at the 12-month point for those with low general self-efficacy at baseline but not for those with suggested case level anxiety. A similar, but not so pronounced, pattern of statistically significant impacts was detected among those with suggested case level depression at baseline.

Analysis of participant survey responses at the start and at the end of the course and evidence from the qualitative process evaluation and observational research suggests that a range of perceived employment, health and wellbeing were achieved by course participants during their time on the course. This suggests that amongst a wider group of participants the course was having at least a perceived positive effect, which had the potential to be sustained and built upon in future.

Finally, the trial showed that mental health and wellbeing outcomes can be achieved outside of a clinical setting by an employment intervention. Importantly, no negative effects on participants were identified across the measures used.

The learning from the trial has the potential to be used with some of the Department’s most vulnerable clients, and the Group Work/JOBS II intervention could be tailored to address the needs of specific groups, e.g. the long-term unemployed, those with health and disability issues, and the outcomes monitored accordingly. While elements of the intervention can be applied across a range of Department for Work and Pensions (DWP) interventions, the highly specified nature of the Group Work model means the same or similar impacts with other groups cannot be guaranteed.

7.1.2 Was the course cost effective? – cost benefit analysis

The cost benefit analysis provided estimates of the cost of delivering Group Work and of the (monetised) benefits resulting.

At the ‘all participant’ level the cost benefit analysis (CBA) found that the costs of delivering Group Work outweighed the value of the benefits that could be valued in money terms. However, it is also important to note that the mental health, wellbeing and job search-related benefits could not be monetised, and so are not reflected in the analysis.

When the costs and benefits are examined at the subgroup level, i.e. focussing solely on those with high levels of anxiety and with lower levels of general self-efficacy at baseline, the monetised benefits for both groups were found to exceed the costs (producing benefit cost ratios of 1.76 for those with high levels of anxiety and 1.39 for those with low self-efficacy at baseline).

This suggests that if the intervention were targeted at these specific subgroups net benefits could be achieved – although this assumes that the per-participant costs of delivering the course to these groups would be the same as in the trial (i.e. that there would be no extra costs in limiting and targeting the intervention to these groups), and that groups comprising solely benefit claimants from the two subgroups would result in the same levels of benefit.

7.1.3 The process evaluation and observational research – participant recognition, delivery and active elements

The process research explored the delivery of Group Work from the Work Coach, Group Leader and participant perspectives, to identify what worked well and what less well, what influenced participants’ reasons for attending (or declining, as well as not completing the course), and to identify the fidelity of the implementation of the trial.

Participant recognition and response

While the participant recognition and recruitment processes were found to work well overall, the latter stages of the trial showed a change in the profile of participants, notably in terms of an increase in the share reporting being confident of finding work within 13 weeks and fewer who considered themselves to be struggling with their job search. While Work Coach discretion may explain why some participants who did not consider themselves to be struggling were recruited, it is likely that this would have affected their response to the course.

Participants made the decision to attend the course for a range of reasons, including to help them find work but also to improve their state of mind, receive mental stimulation and, for a few, because they assumed attendance was expected. Conversely those declining the course offer most commonly did so because they felt they had little to learn from it, were uncomfortable about previous experiences of working in a group, had existing personal commitments or felt they were not sufficiently well to attend.

Benefit claimants initially accepting the offer to attend the course, but not subsequently attending it, did so for a range of reasons. Most commonly these included health issues, other personal crises or finding work, although some attending the Initial Reception Meeting (IRM) reportedly found that the course was not mandatory and so did not continue.

Course completion rates were high at 76 per cent, with participants reporting a combination of their enjoyment of the course, its perceived relevance and value, and a personal determination to ‘finish what I started’ as key to their continued attendance. Those leaving the course early did so most commonly for health reasons, other competing priorities for their time (including job interviews and hospital appointments), and in a few cases what they felt were negative or disruptive group environments.

In conclusion, the implementation of the recognition and recruitment process in live delivery led to benefit claimants with a diverse range of characteristics being recruited, including some who may not have met the intended criteria, and is likely to have influenced the share of participants able to benefit most.

Course delivery

Overall, the version of the Group Work developed for the UK trial was found to be delivered broadly as intended. Group sizes were within the stated parameters, although those closer to the upper limit of 20 could be challenging, and scheduling issues arose when last minute drop-outs caused courses to be cancelled. On occasion, where sessions ran out of time, topics were skipped, but in the main the course covered its intended topics. The Group Leaders reported finding Group Work both challenging and rewarding to deliver, and expressed a strong understanding of, and belief in, the theory underpinning the course.

The vast majority of participants reported finding the course enjoyable and useful, with the extent to which they were struggling with their job search, confidence or anxiety influencing their overall perceptions of it (those reporting most positively typically being individuals who were struggling with their job search and who had been anxious or low on confidence).

Participants’ experiences of the course were also influenced by the Group Leaders, including their ability to understand the challenges of finding work often based on their personal experience. Their facilitation skills were also important, and participants’ positive experiences were underpinned by the Group Leaders’ ability to treat the participants as equals, respond to their needs, establish a safe and participatory environment, and model behaviours and traits the participants admired.

The group environment and the group dynamic established were also important factors, and while group working could be challenging for those new to or previously uncomfortable in this environment its importance was recognised by both participants and Group Leaders. The six-month survey found that around two thirds of participants felt that the group environment had helped them make more progress, feel more supported and gave them new ideas and insights. The focus of group activities on constructive and goal orientated tasks, such as mock interviews, role play or job search networking was also important to the ‘active learning’ approach followed. The group environment also fostered self-reflection, based on the realisation that others were in similar situations to themselves, and allowed participants to learn how to address common barriers from others. However, in some cases this self-realisation could lead individuals to compare themselves unfavourably to others on the course.

Views on the most appropriate group composition varied between the Group Leaders and participants, with the key determinant being the degree of commonality across the group members. Too much commonality risked reinforcing participants’ negative perceptions and reduced the opportunity for learning from others, while too little commonality inhibited the development of a group dynamic and risked isolating some participants – with the ideal being a balance between commonality and difference. Age, educational level and professional background were also influences on the group dynamic, where again having a mix of characteristics was seen as most beneficial. This further emphasised the importance of the participant selection process to effective delivery.

Finally, views on the usefulness and relevance of the course content also varied depending on the participants’ individual characteristics and needs. While the majority found the content useful and relevant, and a source of new insights and ideas, the minority finding it less useful or relevant tended to be those not struggling with their job search or with a long-term health condition or disability. This further emphasised the importance of the participant selection process to effective delivery. Both course participants and Group Leaders considered that without the course content, and the goal-orientated activities, the social contact and support element would not be as strong, with many participants commenting it would ‘just be like the other courses or provision’ which resembled a traditional school classroom approach.

Short-term outcomes of the course

The process evaluation and observational research, along with survey data collected, showed that a range of short-term outcomes were experienced by course participants during the course and shortly afterwards. These included increased positivity and improvements to participants’ general mood, confidence and sense of self-esteem, with over half the participants showing improvements between Days 1 to 5 in most job search, wellbeing and health measures. Statistically significant differences were detected between Day 1 to 5 for participants whose scores on the GAD-7 and PHQ-9 scales suggest caseness for anxiety and depression in terms of job search self-efficacy, wellbeing, depression and anxiety measures, further indicating that those with lower levels of wellbeing and mental health are most responsive to the course while they were on it.

Post-course qualitative interviewees conducted as part of the observational research with a small sample of former participants one week, and one, two, three, six and 12-months post-participation provided insights into the duration of course outcomes. Participants finding work two to three months after completion attributed this at least in part to the course, and along with those not finding work reported that many of the positive wellbeing and mental health outcomes had been sustained at this point. However, from the three-month point on, those not finding work or who had lost jobs secured since the course were struggling with their mental health, wellbeing, confidence and motivation, and had lost the initial benefits gained. Conversely, those in work at three months on were more likely to report that these benefits were maintained, although the type of work engaged in was important and those moving between short-term or zero hours contracts did not report the same levels of sustained benefit.

The research suggests that the opportunity exists to help participants maintain and build on the positive outcomes of the course afterwards. In the absence of formal follow-up procedures, there was broad agreement amongst Group Leaders and participants that a more formalised approach to post-course support could have helped to maintain the initial benefits of the course and the momentum developed during it.

7.1.4 The active elements of Group Work

The observation research and interviews over the five days of the course identified three ‘active elements’ of Group Work which may lead to changes in health, wellbeing and job search behaviour. These were:

• Active element 1: support active participation in a group context - the combination of active participation and the group dynamic established was important, with the balance between Group Leader led and the more interactive elements being considered to have worked well by the individuals interviewed. Active participation was facilitated by the Group Work learning materials and sessions such as role playing, mock interviews and group feedback sessions at the beginning and end-of-each day. Where positive, the group dynamic could lead to benefits including realising that others were in the same position as themselves, the fostering of self-reflection, learning from others on how to address shared barriers, and establishing new friendships and social networks. The access to social support was particularly pertinent and was reported to lead to reduced feelings of loneliness and social isolation with associated self-reported improvements in mental health and wellbeing.
• Active element 2: replicating the time structure and routine of employment - the development of a routine and structure to the day was also an important element, alongside providing participants with constructive activities and a change to what were often described as monotonous daily routines they experienced while being out-of-work. A structured daily format of four hours per day was reported by Group Leaders and participants to replicate or emulate the experience of being in work which many reported provided a sense of stability and improvements in wellbeing.
• Active element 3: Group Leader quality and credibility – the role of the Group Leaders was perhaps the most important of the three elements and acted as a catalyst for the other active elements. The course observations found the most effective Group Leaders were those who: demonstrated empathy; experience of being unemployed or knowledge of mental health issues; the ability to relate this to the participants; and provided positive reinforcement so participants felt valued and supported. In some cases, participants reported these characteristics as being absent in employability provision they had attended previously.

These active elements should also be supported by good quality learning materials, like those offered through the Group Work course.

Both Group Leaders and participants considered that if other employability provision contained and was based on these three elements they could have positive impacts on participants.

7.1.5 The overall implementation of the trial

The research explored the fidelity of the trial from an evaluation perspective. While there was evidence that Work Coaches had followed increasingly broad interpretations of who would benefit from the course (see 7.1.3), and the referral of benefit claimants not struggling with their job search raised questions around value for money and effectiveness, overall we conclude that the implementation of the trial was robust – the random allocation process worked effectively and the referral criteria were broadly followed.

Conversations with Group Leaders as part of the observation work suggest that the course may also be appropriate as a means of speeding the return to work for individuals recently losing their jobs (as in the original JOBS II model in the United States), and to help avoid slipping into negative routines. However, where benefit claimants are referred early in their claims this should be on the basis that they are, or are likely to be, struggling with their job search.

Learning from the research for future trials in similar policy areas, included:

• Ensuring any introductory training for Work Coaches and others involved in the trial is delivered in sufficient time to prepare them for the start of the trial;
• Considering the timing of the trial start – to avoid holiday periods or times of the year when pressures on Work Coaches may be greatest;
• Responding appropriately to lower than expected numbers of recruits – to ensure the numbers recognised to participate increase while maintaining fidelity with the trial participation criteria;
• Monitoring recruitment and responding if necessary - for example, monitoring the characteristics of those recruited as well as the numbers, to ensure the correct benefit claimants are being targeted, and responding if this is not the case; and
• Considering whether the Zelen model (described in detail in the Technical Report on the Impacts of the Trial) should be applied in future – while this approach helped limit Work Coach concerns over the randomisation process (i.e. removing the need to inform benefit claimants of the opportunity to attend the course only for them to be subsequently randomised out), this must be set against the implications for evaluation (i.e. ensuring that sufficient numbers of benefit claimants agree to participate in the intervention to allow analysis).

7.2 Recommendations

The evaluation findings point to opportunities to build on the lessons learnt from the trial, for further exploration of the Group Work/JOBS II model and applying the learning in wider Department for Work and Pensions/Jobcentre Plus provision.

7.2.1 Recommendations for Group Work

Recommendation 1 - If Group Work was trialled again, consideration could be given to targeting the sub-groups found to benefit most (those with low levels of general self-efficacy, and/or higher levels of anxiety, for whom the course was shown to be cost effective), building on lessons from this evaluation.

Key areas of interest in testing a revised model could include:

• Whether Group Work is effective and replicates the results for the sub-groups identified in the study when provided on a more targeted basis. The aim would be to test delivery of the course with mixed benefit claimant groups (with the subgroups in the majority), to explore the effect on this more targeted approach on group dynamics.
• The extent to which the three ‘active elements’ identified continue to apply to the target subgroups, or whether there are other factors specific to these groups which should be considered.
• Whether the inclusion of formalised post-course follow-up by the benefit claimant’s Work Coach on completion, including action planning for job acquisition to maintain the momentum generated, has a material effect on the outcomes (and impacts) achieved. Using the Group Leaders to undertake follow-up meetings could also be considered, perhaps by reconvening groups for a half-day session three months after completion, and involving former participants who have found work to share their experiences.
• Whether there is value in facilitating the continuation of the social groupings established on the course.

Recommendation 2 - If Group Work is adopted in the future, consideration should be given to how Work Coaches or others can be trained and helped to accurately recognise benefit claimants whose behaviour and psychological resources indicate they are likely to benefit most.

Recommendation 3 – If there was further use of the Group Work model in the UK labour market, the recognition, recruitment and other processes associated with the intervention should be monitored closely, to ensure the eligibility criteria are being adhered to and the appropriate people are recruited.

Recommendation 4 – If Group Work was used in the future, it should seek to replicate the content and structure of the UK Group Work trial which was found to be effective and ensure it continues to be delivered by suitably trained and experienced facilitators. Lessons from the process evaluation to also consider, include:

• Locating provision within easy reach of participants;
• Providing comprehensive training in advance to Work Coaches and providers; and
• Offering Work Coaches the opportunity to observe the course in action (or at least receive comprehensive briefings from Group Leaders).

While the trial was delivered by third party contractors, any future development could consider whether Jobcentre Plus staff could deliver the course and achieve the same impact. It should be noted, however, that having the course delivered at a neutral location, and quality and credibility of the Group Leaders, were found to be key factors reported by participants. Equally, the issue of Group Leader supervision and the monitoring of Group Leader delivery, are important issues for future consideration.

7.2.2 Lessons learned for wider provision

The learning from the trial also has the potential to be applied across wider Jobcentre Plus provision, although the extent to which similar effects would result from their application in different contexts would need to be monitored.

Lesson 1 - Explore how each of the ‘active elements’ of Group Work identified in the trial may be applicable to wider provision. More specifically, consideration could be given to:

• How provision can be structured to more closely emulate work?
• How more active participation in labour market interventions can be achieved?
• How the Facilitator role and the characteristics of those Facilitators found to be more effective in the trial can be applied to other provision?

Lesson 2 - As the intervention had some positive effects on mental health and wellbeing, especially in helping to improve participants’ self-efficacy and sense of confidence, consideration could be given to the inclusion of mental health and wellbeing measures such as self-efficacy, ONS-4 and WHO-5 in other interventions and provision, as indicators of the intervention’s effectiveness in addition to the routine measures such as rates of job entry.

 

References

Ayala, L., and Rodríguez, M. (2013) Health-related effects of welfare-to-work policies. Social Science & Medicine 93: 103–112.

Callanan, M., Moke, T.M. and Edovald, T. (2015) Evaluation of the Group Work Psychological Wellbeing and Work Feasibility Pilot. DWP Research Report No 905. London.

Coutts, A. (2009) ‘Active labour market programmes and health: an evidence base’, Report for World Health Organisation and Strategic Review of Health Inequalities in England Post 2010 (Marmot Review 2010)

Coutts, A., Stuckler, D. and Cann, D. (2014) The health and wellbeing effects of active labour market programs. In: Huppert FA and Cooper C (eds), Interventions and Policies to Enhance Wellbeing, London: Wiley, pp. 465–482.

Crost, B. (2016) Can workfare programs offset the negative effect of unemployment on subjective well-being? Economics Letters 140: 42–47.

Henkel, V., Mergl, R., Kohnen R., Maier W., Möller H-J., and Hegerl, U. (2003). Identifying depression in primary care: A comparison of different methods in a prospective cohort study. BMJ (Clinical research ed.). 326. 200-1.

Kim T.J., and Von dem Knesebeck, O. (2015) Is an insecure job better for health than having no job at all? A systematic review of studies investigating the health-related risks of both job insecurity and unemployment. BMC public health, 15(1), 985.

Knight, T., Lloyd, R., Downing, C., Svanaes, S. and Coutts, A. (2021a) Group Work/JOBS II: Process Evaluation Technical Report, DWP Research Report No 989. London.

Levis, B., Benedetti, A., and Thombs, B. (2019) Accuracy of Patient Health Questionnaire-9 (PHQ-9) for screening to detect major depression: individual participant data meta-analysis, BMJ; 365.

McKee-Ryan, M.F., Song, Z., Wanberg, C.R. and Kinicki, A.J. (2005) ‘Psychological and Physical Well-Being During Unemployment: A Meta-Analytic Study’, Journal of Applied Psychology. 90, (1), pp. 53–76.

McManus S., Bebbington P., Jenkins R., and Brugha, T. (eds.) (2016) Mental health and wellbeing in England: Adult Psychiatric Morbidity Survey 2014. Leeds: NHS Digital.

McManus, S., Meltze, M., Brugha T., Bebbington, P., and Jenkins, R. (2009) Adult psychiatric morbidity in England (2007): Results of a household survey. Leeds: The NHS Information Centre for Health and Social Care.

McManus, S., Mowlam, A., Dorsett, R., Stansfeld, S., Clark, C., Brown V., Wollny, I., Rahim, N., Morrell, G., Graham, J., Whalley, R., Lee, L., and Meltzer, H. (2012) Mental health in context: the national study of work-search and wellbeing. DWP Research Report No. 810. London.

Meehan, M., Birkin, R., Ruby, K., and Moore-Purvis, H. (Eds.) (2015) UK JOBS II: A Manual for Teaching People Successful Job Search Strategies. London: DWP. (The UK edition is a revision of Curran. J., Wishart, P., Gingrich, J. (1999) JOBS: A Manual for Teaching People Successful Job Search Strategies. Ann Arbor, MI: University of Michigan).

Paul K., and Moser K. (2009) Unemployment impairs mental health: Meta-analyses, Journal of Vocational Behaviour 74, 3, pp. 264-282.

Pearlin, L., Menaghan, E., Lieberman, M., and Mullan, J. (1981). The stress process. Journal of Health and Social Behavior, 22, 337-356.

Puig-Barrachina, V., Malmusi, D., Artazcoz, L. Bartoll, X., Clot, E., Cortes-Franch, I., Ventura, L., Daban, F., Diez, E., and Borrell, C. (2020). How does a targeted active labour market program impact on the well-being of the unemployed? A concept mapping study on Barcelona “Employment in the Neighbourhoods”. BMC Public Health 20, 345.

Purdon, S., and Bryson, C. (2021) Group Work/JOBS II: Technical Report on the Impacts of the Trial, DWP Research Report No 988. London.

Rayment, M., Knight, T., Lloyd, R., Purdon, S., Bryson, C. and McKay, S. (2021) Group Work/JOBS II: Cost Benefit Analysis Technical Report, DWP Research Report No 990. London.

Sage, D. (2015a) Do active labour market policies promote the subjective well-being of the unemployed? Evidence from the UK National Well-Being Programme. Journal of Happiness Studies 16(5): 1281–1298.

Steadman, K., and Taskila, T. (2015) Symptoms of depression and their effects on employment. The Work Foundation.

Van Hooft, E.E., Wanberg, C.R., and Van Hoye, G. (2012) Moving beyond job search quantity: Towards a conceptualization and self-regulatory framework of job search quality. Organizational Psychology Review.

Van Stolk, C., Hofman, J., Hafner, M., and Janta, B. (2014) Psychological wellbeing and work: Improving service provision and outcomes. Department for Work and Pensions and Department of Health: London, UK.

Vinokur, A.D., Price, R.H., and Schul, Y. (1995). Impact of the JOBS intervention on unemployed workers varying in risk for depression. American Journal of Community Psychology, 23, 39-74.

Vinokur, A.D., and Schul Y. (1997) Mastery and inoculation against setbacks as active ingredients in the JOBS intervention for the unemployed. Journal of Consulting and Clinical Psychology.;65(5):867‐877.

Vinokur, A.D., and Schul Y. (2002) The web of coping resources and pathways to reemployment following a job loss. Journal of Occupational Health Psychology, 7(1), 68–83.

Vinokur, A.D., Schul, Y., Vuori, J., and Price, R. (2000) Two years after a job loss: Long-term impact of the JOBS program on reemployment and mental health. Journal of Occupational Health Psychology 5(1): 32–47.

Vuori, J., Silvonen, J., Vinokur, A. D., and Price, R.H. (2002) The Tyohon Job Search Program in Finland: Benefits for the Unemployed with Risk of Depression or Discouragement. Journal of Occupational Health Psychology 2002 Vol 7, No. 1, 5-19.

Vuori, J., and Vinokur, A.D. (2005) Job-search preparedness as a mediator of the effects of the Työhön Job Search Intervention on re-employment and mental health. Journal of Organizational Behavior 26(3): 275–291.

Waters, L., and Moore, K. (2002) Reducing latent deprivation during unemployment: The role of meaningful leisure activity. Journal of Occupational and Organizational Psychology, 75, 15–32. Wulfgramm, M. (2014) Life satisfaction effects of unemployment in Europe: The moderating influence of labour market policy. Journal of European Social Policy 24(3): 258–272.

 

1. For some outcomes, the baseline measure was collected at the point of randomisation. For others, they were collected for course participants on day 1 of the course and for course decliners and the control group in a survey collected some months after the participant baseline. ↩

2. Findings from an ‘intention to treat’ analysis can be found in Chapter 5 of the Technical Report on the Impacts of the Trial. Chapters 6 and 7 of that report covers the findings reported on here. ↩

3. Job search self-efficacy is the strength of an individual’s belief that they have the skills to undertake a range of job search tasks. ↩

4. General self-efficacy is the strength of an individual’s belief that they are effective in handling life situations. ↩

5. A person is described as having suggested case level anxiety if their score on the GAD-7 scale suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnosis of anxiety would be based on a clinical interview and would take account of additional evidence, to which the GAD score may contribute. ↩

6. A person is described as having suggested case level depression if their score on the PHQ-9 scale suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnosis of depression would be based on a clinical interview and would take account of additional evidence, to which the PHQ score may contribute. ↩

7. Further information on the course is available in the Process Evaluation Technical Report, including the Desk Aid for recognising potential participants provided to Work Coaches in the training materials. ↩

8. See section 3.3.2 for details of the characteristics of course participants. ↩

9. See the Technical Report on the Impact of the Trial for the detail of the indicators used and further details on the characteristics of the participants. ↩

10. A person is described as having suggested case level anxiety or depression if their scores on the GAD-7 or PHQ-9 scales suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnosis of anxiety and depression respectively would be based on a clinical interview and would take account of additional evidence, to which the GAD and PHQ scores may contribute. ↩

11. This does not take account of benefits for mental health, wellbeing and job search-related outcomes, which could not be valued in monetary terms. ↩

12. Active elements are the components within an intervention or course (such as the nature of the learning materials, the quality of the Group Leader and whether a course offers or facilitates social support and interaction with other participants) which interact with the background characteristics of participants and may result in changes in their health, wellbeing and behaviour. ↩

13. See Tables 3.2. and 3.3 ↩

14. See Employment and Support Allowance Caseload: DWP Stats Xplore, November 2019 ↩

15. Active Labour Market Policies (ALMPs) aim to increase the employment opportunities for job seekers and improve matching between jobs (vacancies) and workers (i.e. the unemployed). In so doing ALMPs may contribute to reducing unemployment and benefit receipt via increased rates of employment and economic growth. ↩

16. See for example in Germany (Crost 2016, Wulfgramm 2014), Spain (Ayala and Rodriguez 2013), US (Vinokur et al 2000) and Finland (Vuori and Vinokur 2005). See also Puig-Barrachina et al (2020) and Sage (2015). ↩

17. Sample size: 1,801 (Experimental: 1,249 Control: 552). ↩

18. The mastery outcome was a composite measure taking into account scores on job search self-efficacy, self-esteem and locus of control indexes. It was designed to be a measure of someone’s emotional and practical ability to cope and take on particular situations. ↩

19. Sample size: 1,261 (Experimental: 629, Control: 632). ↩

20. Job stability was based on the respondents’ report of whether their new job was a stable job or just a temporary job. In the Finnish labour markets, a stable job is generally desirable and secure and has a higher status compared with temporary jobs. ↩

21. Psychological distress is a term used to describe unpleasant feelings or emotions that impact an individual’s level of functioning, i.e. psychological discomfort that interferes with daily activities and can result in negative views of the environment, others, and of self. Sadness, anxiety, distraction, and symptoms of mental illness are manifestations of psychological distress. ↩

22. It should be noted that the onscreen survey was specific to the trial and would not feature in any wider roll-out of Group Work. ↩

23. Full data tables are provided in the Technical Report on the Impacts of the Trial. ↩

24. A person is described as having suggested case level anxiety or depression if their scores on the GAD-7 and PHQ-9 scales suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnosis of anxiety and depression respectively would be based on a clinical interview and would take account of additional evidence, to which the GAD and PHQ scores may contribute. People with scores below the respective caseness thresholds are referred to as ‘other’. For additional detail please see Chapter 3, Section 3.5 of the Technical Report on the Impacts of the Trial. ↩

25. All benefit claimants who attended day one of the course were classed as “participants” in the trial. ↩

26. Full data tables are provided in the Technical Report on the Impacts of the Trial ↩

27. Full data tables are provided in the Technical Report on the Impacts of the Trial ↩

28. Full data tables provided in the Process Evaluation Technical Report ↩

29. Full data tables are provided in the Process Evaluation Technical Report ↩

30. Latent and Manifest Benefits (LAMB) are material and psychosocial benefits associated with being in work such as social interaction, social support, activity, identity, collective purpose, self-worth (Latent benefits) and the absence or lessening of financial strain (Manifest). ↩

31. The p-value is the probability of an observed difference being due to chance alone, rather than being a real underlying difference for the population. ↩

32. A test of a significant interaction. ↩

33. A person is described as having suggested case level anxiety or depression if their scores on the GAD-7 or PHQ-9 scales suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnosis of anxiety and depression respectively would be based on a clinical interview and would take account of additional evidence, to which the GAD and PHQ scores may contribute. ↩

34. Participants were not asked if they were doing any paid work at the baseline. ↩

35. Not included baseline comparison data on work satisfaction and earnings given lack of data for participants. ↩

36. See Section 3.3 of the Technical Report of the Impact of the Trial for more details. ↩

37. For life satisfaction, feeling worthwhile and happiness, a higher mean score denotes a more positive outcome while for anxiety, a higher score denotes greater anxiety. ↩

38. It is important to note a clinical diagnosis of anxiety is based on a clinical interview and would take account of additional evidence, to which the GAD-7 score may contribute. ↩

39. We believe that this statistically significant difference at baseline is an anomaly cause by differences in the data collection mode for course participants and the comparison group at baseline. It is not in line with other similar measures such as ONS satisfaction levels asked at randomisation. ↩

40. The difference between the lower and upper estimates reflects the assumed duration of employment effects (18 months in the lower estimate, 24 months in the upper estimate). The lower and upper estimates are therefore the same in the first year. ↩

41. The “lower cost scenario” relates to the costs of the programme, borne by the Government. Participant costs (which relate to benefits, taxes, travel and childcare) and employer costs (wages and National Insurance) are not therefore reduced in this scenario. ↩

42. Their score on GAD-7 scale suggests they would probably be clinically diagnosed as having generalised anxiety. Although it is important to note a diagnosis of anxiety would be based on a clinical interview and would take account of additional evidence, to which the GAD-7 score may contribute. ↩