Research and analysis

Process evaluation technical report

Updated 17 May 2022

Applies to England

DWP Research Report no. 989

A research report carried out by research consortium led by ICF on behalf of Department for Work and Pensions/Department of Health and Social Care (Employers, Health and Inclusive Employment directorate).

Crown copyright 2021.

You may re-use this information (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence

Or write to:

Information Policy Team
The National Archives
Kew
London
TW9 4DU

Email: [email protected]

This document/publication is also available on our GOV.UK website

If you would like to know more about DWP research, please email: [email protected]

First published 2021.

ISBN: 978-1-78659-261-3

Views expressed in this report are not necessarily those of the Department for Work and Pensions or any other government department.

Acknowledgements

This study was commissioned by the joint Department for Work and Pensions and Department of Health and Social Care Work and Health Unit. We are particularly grateful to Pontus Ljungberg, Sian Moley, Lyndon Clews, Sarah Honeywell, Caroline Floyd, Rachel Shanahan, David Johnson, Mark Langdon, Anna Bee, Craig Lindsay and members of the DWP Policy Psychology Division for their guidance and support throughout the study.

Dr Adam P. Coutts would like to thank the Health Foundation for the three-year fellowship (Grant ID – 1273834) which enabled him to conduct the research. Many thanks to Liz Cairncross at the Health Foundation who provided support and advice throughout the fellowship. We would also like to thank the Jobcentre Plus staff, Group Leaders, provider representatives and individual benefit claimants who gave their time to participate in the fieldwork.

Views expressed in this report are not necessarily those of the Department for Work and Pensions, Department of Health and Social Care, or any other government department.

Author’s credits

This report was prepared by Tim Knight and Richard Lloyd of ICF Consulting Services Ltd, Christabel Downing and Siv Svanaes of IFF Research, and Dr Adam P Coutts of the University of Cambridge.

Glossary of terms

Term	Definition
Active elements	Components and features embedded within an intervention or course, such as the nature of the learning materials, the quality of the Group Leader and whether a course offers or facilitates social support and interaction with other participants. These combine with the background social, economic and psychosocial characteristics of participants which may result in changes in their health, wellbeing and job search behaviour.
Active Labour Market Policy	Active Labour Market Policies (ALMPs) aim to increase the employment opportunities for job seekers and improve matching between jobs (vacancies) and workers (i.e. the unemployed). In so doing ALMPs may contribute to reducing unemployment and benefit receipt via increased rates of employment and economic growth.
Active learning techniques	Active learning techniques are based on actively involving participants in a learning activity rather than just requiring them to passively listen.
Carer’s Allowance	Carer’s Allowance (CA) is the main welfare benefit for carers and was formerly known as the Invalid Care Allowance.
Caseness	A person is described as having suggested case level anxiety or depression if their scores on the Generalised Anxiety Disorder (GAD-7) and Patient Health Questionnaire (PHQ-9) scales suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnoses of anxiety or depression respectively would be based on a clinical interview and would take account of additional evidence, to which the GAD or PHQ scores may contribute.
Cost Benefit Analysis	A cost benefit analysis (CBA) examines all the costs and benefits of the intervention and quantifies them in monetary terms as far as possible, in order to examine the balance of costs and benefits.
Disability Employment Advisor	Disability Employment Advisors (DEAs) are people employed by Jobcentre Plus to support and upskill Work Coaches and other members of jobcentre staff to deliver tailored advisory services to disabled people.
Employment and Support Allowance	Employment and Support Allowance (ESA) is a benefit for people who have an illness, health condition or disability that affects how much they can work. ESA offers financial support if you are unable to work, and personalised help so that you can work if you’re able to.
Financial strain	Financial strain refers to when an individual’s financial outgoings start to exceed their income to a degree that psychologically threatens their sense of self, identity, relationships and/or self-esteem.
General self-efficacy	General self-efficacy is the strength of an individual’s belief that they are effective in handling life situations.
Group Leader	Group Leaders are the individuals who delivered the Group Work course, using active learning techniques, to participants.
Group Work	Group Work is a job search course designed to also enhance self-efficacy, self-esteem and social assertiveness among those looking for paid work. It aims to prevent the potential negative mental health effects of unemployment and help unemployed people back into work. The course, is the application of JOBS II model, originally developed by the University of Michigan, in the UK labour market.
Impact on Participants	Impact on Participants (IoP) refers to the analysis of the impact of an intervention based on comparing outcomes for individuals who participated in the intervention with a matched comparison group of individuals who did not.
Income Support	Income Support (IS) is an income-related benefit for people who have no income or are on a low income, and who cannot actively seek work. It is mainly for people who cannot seek work due to childcare responsibilities.
Initial Reception Meeting	All Group Work participants were invited to an Initial Reception Meeting (IRM) which preceded the course itself. The IRM was designed as an opportunity for participants to meet the Group Leaders who would deliver their course and learn more about what it would involve.
Intention to Treat	Intention to Treat (ITT) refers to the analysis of the impact of an intervention based on comparing outcomes for all individuals who were offered the opportunity to participate in the intervention with a control group of individuals who were not offered this opportunity.
Jobcentre Plus	Jobcentre Plus (JCP) is a brand under which the DWP offers working-age support services, such as employment advisory services. In the context of this report, ‘jobcentre’ refers to the physical premises in which Jobcentre Plus services are offered.
Job-search self-efficacy	Job-search self-efficacy is the strength of an individual’s belief that they have the skills to undertake a range of job-search tasks.
JOBS II	JOBS II is the course originally designed by the University of Michigan. Group Work course is the application of JOBS II in the UK.
Jobseeker’s Allowance	Jobseeker’s Allowance (JSA) is an unemployment benefit for people who are actively looking for work.
Latent and Manifest Benefits	Latent and Manifest Benefits (LAMB) are material and psychosocial benefits associated with being in work such as social interaction, social support, activity, identity, collective purpose, self-worth (Latent benefits) and income (Manifest).
Learning and Development Officers	Individuals responsible for delivering training was provided to Work Coaches at the participating Jobcentre Plus offices.
Mastery	The mastery outcome was a composite measure taking into account scores on job search self-efficacy, self-esteem and locus of control indexes. It was designed to be a measure of someone’s emotional and practical ability to cope and take on particular situations.
Mental Health Issue(s)	Mental Health Issue is a broad term that includes those who have: deteriorating mental health (for example, related to the experience of unemployment); elevated but not clinical levels of a symptom; mental health conditions; or are post-treatment; have symptoms but may not recognise they have a condition; or are aware of their condition/ situation but choose not to disclose. Many individuals with Mental Health Issues are found to struggle with their job search.
Psychosocial	Psychosocial indicators concern psychological and social factors that can influence health and wellbeing outcomes. Typical examples of such indicators include social support, employment status, job quality, poverty and marital status.
Self-efficacy	Self-efficacy is the strength of an individual’s belief that they have the skills to undertake a task and achieve an outcome.
Single Point of Contact	Single Points of Contact (SPoCs) were the designated point of contact in each of the Jobcentre Plus districts in which Group Work was trialed, involved in monitoring volumes, training and delivery.
Statistically significant	A statistic derived from a study, such as the difference between two groups, is said to be statistically significant if the size of that statistic has only a low probability of arising by chance alone. The probability of a statistic of that size occurring by chance alone is termed the ‘p-value’. By convention, if the p-value is less than 0.05 then it is stated that the statistic is ‘significant’.
Trial Integrity and Support Officers	Trial Integrity and Support Officers were designated DWP staff responsible for monitoring and supporting the fidelity of the DWP input to the Group Work trial.
Universal Credit	Universal Credit (UC) is an in and out of work benefit designed to support people with their living costs. Most new claims by people with a health condition or disability are now made to UC.
Well-being	Well-being is an individual’s self-report as to whether they feel they have meaning and purpose in their life, and includes their emotions (happiness and anxiety) during a particular period.
Work Coach	Work Coaches are frontline Jobcentre Plus staff based in jobcentres. Their role is to support benefit claimants into work through work-focused interviews.
Work and Health Unit	The Work and Health Unit (WHU) is a joint unit between the Department for Work and Pensions and Department of Health and Social Care. It leads on the Government’s strategy to support working-age disabled people or those with long-term health conditions, to access and retain good quality employment.
Zelen design	The Zelen design is randomised control trial methodology in which randomisation is applied before any potential beneficiaries are informed of the possibility of participating in the intervention being trialed. Only those randomised into the experiment group are informed of the opportunity of participating.

Executive summary

Introduction

ICF, in partnership with IFF Research, Bryson Purdon Social Research, Professor Steve McKay of the University of Lincoln and Dr Clara Mukuria of the University of Sheffield, were commissioned by the Department for Work and Pensions to undertake a programme of research to evaluate the Group Work trial (Group Work being the UK version of the JOBS II course). This report provides findings from the process evaluation conducted as part of the research. Dr Adam Coutts of the University of Cambridge provided background theoretical and empirical evidence from the international literature regarding the links between Active Labour Market Policies (ALMPs)s, health and wellbeing and findings from observational research and participant observation of the Group Work intervention.

Overview of Group Work/JOBS II

Since 2013 the Department for Work and Pensions (DWP) and the Department of Health and Social Care (DHSC) have been working together to explore joined-up policy approaches and interventions to address the links between unemployment and mental health issues. The Improving Lives: The Work, Health and Disability Green Paper included a commitment to a “Group Work” trial to test whether the JOBS II model improves employment prospects and wellbeing when delivered in the UK context. A Randomised Control Trial (RCT) was undertaken to test the potential effectiveness of the course in a UK labour market context.^{[footnote 1]} The trial ran in five Jobcentre Plus districts between January 2017 and March 2018. It was targeted at benefit claimants who were struggling with their job search and/or were feeling low or anxious and lacking in confidence about aspects of their job search.

The Group Work course delivered in the trial was based on the JOBS II model, originally developed in the United States by the University of Michigan. It comprised five four-hour sessions, delivered over one working week, in a group environment by trained facilitators encouraging participation through active learning techniques. While the course focused on job search skills, the way in which it was delivered was intended to enhance individual self-efficacy, self-esteem and social assertiveness. Participation was voluntary.

Research methodology

This report is based on evidence collected through two main pieces of research:

• A qualitative process evaluation conducted by ICF - a total of 140 interviews were conducted with Group Work provider leads and Group Leaders (15), Jobcentre Plus staff (45), and benefit claimants (80). Of the benefit claimants interviewed, 40 had completed the Group Work course, 20 started but did not complete, and 20 who declined the opportunity to attend.
• Observational research conducted by Dr Adam Coutts - this included interviews with DWP and Jobcentre Plus staff, over 300 hours of participant observation of 17 Group Work sessions, one-day workshop with DWP policy psychologists (2) and Group Leaders (9), and semi-structured interviews with participants (100). Post Group Work health and labour market experiences were explored through semi-structured interviews with a cohort of 25 participants from across the trial sites. Interviews were conducted with this cohort (the same individuals) at five time points (one week, one, three, six and 12 months) post participation.

Quantitative management information and survey data collected through the trial has also been drawn on in some instances to provide further evidence.

Findings on the Jobcentre Plus elements of delivery

Jobcentre Plus Work Coaches were responsible for recognising benefit claimants who would potentially benefit from Group Work and for offering potential beneficiaries the opportunity to participate.

Training to support trial delivery

Training and development activities for Work Coaches, developed and delivered by the DWP’s Policy Psychology Division, took place in two main waves: prior to the trial start in October to December 2016 and after the trial had commenced from March 2017 onwards. Work Coaches were generally positive about the quality, coverage and appropriateness of the training, and reported particular benefits to additional activities such as briefings by Group Work Group Leaders and opportunities to observe delivery. However, it was felt that the timing of the pre-implementation training, and the level of detail provided about the course and what differentiates it from other provision, could have been improved – with less of a gap between the training and the start of the intervention, and with more information to help promote the course to potential participants.

Recognising potential beneficiaries of Group Work

An initial expectation set at the start of the trial was for Work Coaches to recognise 26,000 benefit claimants as potential Group Work beneficiaries, whereas a total of 16,193 benefit claimants were actually recognised by the end of the trial. The main factors (in addition to the training) influencing this were:

• Time constraints - Work Coaches reported that the time required to complete the onscreen survey to collect baseline measures from benefit claimants was the main barrier to them recognising more potential beneficiaries (this survey was specific to the trial and would not feature in any wider roll-out of Group Work).
• Benefit claimant feedback - positive feedback from benefit claimants attending the course encouraged Work Coaches to recognise more potential beneficiaries.
• Interpretations of the criteria for recognising potential beneficiaries - Work Coaches had often understood from the initial training that the course was aimed at those with, or at risk of developing, mental health issues. They thought a broader interpretation of the criteria had been communicated subsequently, which led to them recognising more potential beneficiaries.
• Drives to increase numbers - Work Coaches across the trial districts also reported that they had increasingly been encouraged to consider more and/or more wide-ranging types of benefit claimants as potential beneficiaries.
• Underlying Work Coach attitudes - a minority were said by the jobcentre managers and Group Work leads interviewed to be unenthusiastic about any new initiatives introduced in their jobcentre, not specifically Group Work, and had identified few or no potential beneficiaries.

During the trial the profile of benefit claimants being recognised as potential beneficiaries also changed, with those who were more confident about their job prospects increasingly being identified. Some Group Work provider staff felt this potentially diluted the impact of the trial but equally there was a belief that a mix of participant characteristics was a positive for the group dynamic on the course.

Introducing and explaining Group Work

Evidence from the process evaluation suggests that not all Work Coaches had initially followed the intended process for when they introduced the course to benefit claimants. However, six to nine months into the trial, when the study was conducted, and after further post-implementation training, this process was consistently being followed in most cases.

Overall, 45 per cent of benefit claimants who were offered the opportunity to go on Group Work by their Work Coach accepted (although not all subsequently attended the course). What appeared to most effective in terms of encouraging take up was:

• Emphasising the difference of the course - particularly with benefit claimants who had been on previous Jobcentre Plus provision and potentially saw little value in going on something that superficially sounded very similar.
• Using benefit claimant feedback and international evidence - benefit claimants were receptive to being told about the impact of the course on others, either based on evidence from other countries or about other benefit claimants who had attended.
• Tailoring the explanation to the benefit claimant - for example by picking up on something they had said and using that as a way into introducing the course.

Reasons for accepting or declining the Group Work offer

Reasons for course take-up were most commonly to help find work (including improving job search skills or CV presentation); increase motivation and self-esteem; and, less often, to meet assumed Work Coach expectations or requirements.

Conversely, reasons for declining included the view that the course would not offer them anything new (mainly amongst older and long-term benefit claimants); discomfort with the group setting; health reasons (including medical appointments clashing with the timing of the course); concerns about the physical challenges of travelling to the course venue; and personal commitments such as childcare.

The Group Work course

Prior to starting, benefit claimants attended an Initial Reception Meeting (IRM) to find out more about the course, visit the venue and meet their Group Leaders. Benefit claimants interviewed felt the IRM was well delivered, useful, and commonly confirmed or re-enforced their motivation to attend the course.

Delivery of the Group Work course

Group Leaders, employed by the third party providers, expressed a strong belief in the course, and had attended seven weeks of training which they felt had prepared them sufficiently. They said the course was delivered in accordance with the UK JOBS II manual, although elements had been passed over quickly or occasionally omitted due to time pressures. It was felt there was ‘a lot to cram in’ to the time available, but that sessions could, and should, run over time to allow participants to ask questions.

Some issues were reported with course scheduling, with some benefit claimants having to wait until the minimum cohort size of ten was reached, and on occasion courses were cancelled due to last minute drop-outs. In most cases benefit claimants and Group Leaders reported group sizes of between 12 and 15, in line with the international evidence which suggests group sizes of between 10 and 15.

Benefit claimants’ experiences of the course

Benefit claimants’ experiences ranged from the overwhelmingly positive to the mildly appreciative and, exceptionally, the negative, and appeared to reflect:

• Benefit claimant needs - with the ‘overwhelmingly positive’ being typically those struggling with their job search and either anxious, worried or low in confidence, and who indicated that the course had a positive impact in both these domains.
• How benefit claimants responded to the course - while the vast majority described their experiences positively, the few negative reflections typically related to an aspect of course delivery rather than design.

Looking at experiences of specific elements of the course in more detail, key factors underpinning participant experiences included:

• The facilitation of the course - with the most positive participants highlighting the role of the Group Leaders and their style of delivery, which included: being treated as an equal; establishing an environment which encouraged active participation; the sharing of experiences; and role-modelling (through the Group Leaders’ referent power to motivate or inspire by example).
• Course content - which the majority of participants described as useful and relevant, and provided them with new job search tactics, tools or techniques.
• Course format - where the balance between facilitator-led and interactive elements; the course timing/duration (seen as manageable and an enabler for attendance), and the provision of routine and structure were important factors.
• Group dynamics - which most participants found positive and supportive, underpinned by the realisation that others were in similar circumstances, and allowing experiences of addressing common barriers to be shared.

Course retention and drop-out

Interviews with benefit claimants completing the course, found that the majority of completers had enjoyed attending, due to a combination of the facilitation, group dynamic and resulting mental stimulation; and the perceived relevance and value of the course. Some benefit claimants who left the course early reported in interviews doing so due to competing commitments, such as hospital appointments or emergencies, while others had concluded early in the course that it had little to offer them.

Reported short-term outcomes

Data was collected by the course providers to capture self-reported outcomes for participants at Days 1 and 5 of the course. This showed that a range of positive job search, wellbeing and mental health outcomes were realised over the five days of the course (including 80 per cent who reported an improvement in job search self-efficacy, 66 per cent who showed higher levels of wellbeing/reduced likelihood of depression under the WHO-5 scale, and 63 per cent who scored lower on the PHQ-9 scale for depressive symptoms).

In the follow-up surveys undertaken with a sample of Group Work participants for the quantitative impact evaluation (detailed in the Technical Report on the Impacts of the Trial), most (92 per cent) respondents to the six-month survey who had been on the course reported finding the course useful to them. Over two thirds reported positive effects in terms of their motivation and confidence (including 71 per cent whose motivation to find work had increased, and 62 per cent who felt they were more confident), and over half reported positive effects on different aspects of their job search (from 55 per cent considering the course had helped them move towards work, up to 72 per cent who reported a better understanding of what is needed to find work).

The process evaluation identified five groups of immediate outcomes resulting from the course, commonly reported in tandem, and applied to the majority of completers:

• Increased confidence/self-efficacy - the most widely reported outcome, resulting from the positive reinforcement from Group Leaders, the group dynamics, and the skills, knowledge and techniques acquired.
• Increased motivation - commonly reported, particularly amongst older benefit claimants out of work for some time and those most negative about their job prospects.
• Increased mental health and wellbeing - with the minority of interviewees who disclosed a mental health condition often reporting positive benefits.
• Changes in job search behaviour - including increased volume/intensity of job search; and more creative, reflective, informed and structured approaches.
• Progress into and towards work – a handful of interviewees found work after the course, while others reported taking steps to move them towards employment (e.g. undertaking training and work placements) which they attributed to the increased motivation, confidence and new ideas from the course.

The observational research also included consultations with a cohort of 25 participants at five time points (1 week, and 1,3, 6 and 12 months) post-participation. (totalling 125 interviews). Although the sample size was small), and the findings should be treated as illustrative, the follow-up found:

• Between one week and three months post-participation, most participants felt that their mental health and wellbeing had improved as a result of attending Group Work and these improvements were maintained up to three months post participation regardless of whether they were employed, unemployed or engaged in voluntary work. Overall participants felt that that the resilience and inoculation against set-backs developed on the course had given them ability to deal with the stressors of job search such as application rejections.
• Between three months to a year, those who struggled to find work or lost jobs reported declines in mental health, wellbeing, confidence and motivation. Those who did not enter employment explained how they had lost the sense of a daily routine that had been developed through participation in the course. In general, those who moved into or remained in work reported that that their mental health and wellbeing had remained positive, although this appeared to be partly dependent on the nature of this work. Some reported moving between short-term and/or zero hours jobs and did not appear to experience the same health and wellbeing gains as those in more stable and rewarding employment.

Post-course follow-up

Although the Work Coach training emphasised the importance of discussing benefit claimants’ experience of the course on their return, the absence of formal follow-up procedures in Jobcentre Plus meant that opportunities for reflection, and maintaining the positive momentum generated, were uncommon. Benefit claimants provided mixed experiences of Work Coach follow-up, with some discussing next steps and receiving support to take these forward, while others reported little consultation on their return. There was broad agreement across the Jobcentre Plus staff interviewed in the field that a more formalised approach to follow up was required to help maintain the momentum developed.

The active elements of Group Work

The process evaluation, observational fieldwork and participant observation in each district provided insights into the active elements which appear to lead to changes in participant health, wellbeing and job search behaviour. These are:

• Active element 1: active participation in a group context - the combination of active participation and the group dynamic established was key, with the balance between Group Leader led and the more interactive elements being considered to have worked well by the individuals interviewed. Active participation was facilitated by the Group Work learning materials and sessions such as role playing, mock interviews and group feedback sessions at the beginning and end of each day. Where positive, the group dynamic could lead to benefits including realising that others were in the same position as themselves, the fostering of self-reflection, learning from others on how to address shared barriers, and establishing new friendships and social networks. The access to social support was particularly pertinent and was reported to lead to reduced feelings of loneliness and social isolation with associated self-reported improvements in mental health and wellbeing.
• Active element 2: replicating the time structure and routine of employment - the development of a routine and structure to the day was also a key element, alongside providing participants with constructive activities and a change to what were often described as monotonous daily routines they experienced while being out-of-work. A structured daily format of four hours per day was reported by Group Leaders and participants to replicate or emulate the experience of being in work. This helped participants get back into a more positively focused routine.
• Active element 3: Group Leaders effectiveness and credibility - the role and quality of the Group Leaders was a key element, and they acted as a catalyst for the other active elements of the course. Group Leaders’ influence as role models, and their ability to display behaviours and traits admired by participants, was evident through their ability to motivate by example. Those able to recount recent experiences of unemployment and mental health issues were most able to empathise and appear credible to the course participants.

The findings from the observational research would suggest that in order for the active elements to generate changes in participants’ mental health and wellbeing, they should be supported by the existing Group Work learning materials and sessions, which encourage and facilitate the direct and active participation of job seekers in the course.

Lessons Learned

The evaluation concluded that the implementation of the course overall, and the fidelity of the trial, was methodologically sound. It also identified a number of learnings that could be applied to any future Group Work intervention. These include:

• Continuing to make course participation voluntary as wanting to be on the course and actively choosing to do so appeared to be important.
• Ensuring Work Coach training is aligned with the start of delivery, and with updates being provided on an ongoing basis.
• Emphasising when communicating with staff involved in the referral process, of the importance of recognising participants who are struggling with job search and could therefore benefit from participation.
• Ensuring messaging about the course to benefit claimants is tailored to them and provides detail on the content, coverage and benefits of attending the course, emphasising how it is different from other provision, is aimed at helping them find work and could help address person-specific challenges (using examples from the current trial).
• Exploring whether ongoing medical appointments can be negotiated to allow benefit claimants with long-term health conditions to participate.
• Where provider venues are some distance from or difficult to reach by public transport, exploring the feasibility of using venues closer to or within for example the specific jobcentre catchment areas.
• Exploring the possibility of allowing courses sizes to drop below the minimum, for example, in the case of last minute withdrawals.
• Consideration given to whether the composition of the participants would aid the dynamics of the group.
• Considering how a more reliable and efficient follow-up process could be put in place, which seeks to maintain and build upon the wellbeing gains and positive momentum developed in the course.
• In any wider application of Group Work, and in employability provision more widely, consideration to be given in the course or intervention design to the three active elements identified to be important in the Group Work Trial. This includes considering:
  • How the provision can be structured to more closely emulate work?
  • How more active participation in labour market interventions can be achieved?
  • How the Facilitator role and the characteristics of those Facilitators found to be more effective in the trial can be applied to other provision?

However, for these elements to effectively work they should be accompanied by good quality learning materials.

1. Introduction

ICF, in partnership with IFF Research, Bryson Purdon Social Research, Professor Steve McKay of the University of Lincoln, Dr Clara Mukuria of the University of Sheffield were commissioned by the Department for Work and Pensions in January 2017 to undertake a programme of research to evaluate the trial of Group Work, which is the UK version of the JOBS II programme. Dr Adam Coutts of the University of Cambridge undertook a research placement between July 2016 and June 2020 within the Work and Health Unit in order to carry out his independent research. This report provides technical information and findings from one element of this research: the process evaluation.

The report is one of four reports that have been produced to present the evidence from each element of the research. The other reports are:

• An impact evaluation technical report;
• A cost benefit analysis technical report; and
• A synthesis report.

This chapter provides an initial overview of Group Work, the aims and objectives of the process evaluation, and details of the evaluation methodology.

1.1. Overview of Group Work/JOBS II

Group Work was a trial conducted to test and evaluate the JOBS II model, originally developed in the United States by the University of Michigan, in the UK labour market context. A Randomised Control Trial (RCT), employing a Zelen design, was undertaken to test the potential effectiveness of Group Work/JOBS II in the UK labour market context.^{[footnote 2]} JOBS II is one of several interventions being trialled by the Department for Work and Pensions (DWP) and Department of Health and Social Care (DHSC) Joint Work and Health Unit to build a strong evidence base of what interventions work best to help those with health issues move into or retain work.

The Group Work trial was targeted at benefit claimants of Jobseeker’s Allowance, Employment Support Allowance, Universal Credit Full Service and Income Support (Lone Parents with child(ren) aged three and over) who were struggling with their job search and/or feeling low, anxious or lacking in confidence with regard to their job search. Participation in Group Work was voluntary, and no sanctions applied if a benefit claimant decided not to attend or if they withdrew part way through. The trial started in January 2017 and finished in March 2018, having achieved 2,596 starts.

The trial operated in five Jobcentre Plus districts – Durham and Tees, Merseyside, Midland Shires, Mercia, and Avon, Severn and Thames, with one or two centrally located provider hubs (where the Group Work course was delivered) and a number of participating jobcentres in each district. The participating jobcentres were responsible for recognising benefit claimants that may benefit from Group Work. Jobcentre Plus Work Coaches administered an onscreen survey with all benefit claimants they thought may benefit from Group Work. On completion of the survey, a randomisation filter was applied. Amongst those randomly assigned to the treatment group and offered the opportunity to go on the course, 45 per cent initially accepted the offer and 22 per cent progressed to start the course.

The Group Work course was the application of the JOBS II model developed by the University of Michigan, but adapted for the UK labour market by DWP policy psychologists. Participants were invited to take part in a group-based course delivered in five half-day sessions, averaging four hours a day, over the period of a working week. Although the content of the course was focused on job search skills, the underlying processes by which it was delivered were also designed to enhance the self-efficacy, self-esteem and social assertiveness of the participants. The course was led by trained facilitators using active learning techniques.

1.2. Process evaluation aims and objectives

The process evaluation was undertaken as part of a wider programme of research into Group Work, which also included an impact evaluation and a cost benefit analysis. The Statement of Requirements for the wider programme of research set out the following aims:

“The central research question of this research is to examine:

• “What works to improve employment and health outcomes for people who are out of work and struggling with their job search?”

The primary research questions to be addressed are:

• Did Group Work improve benefit claimants’ employment rates and wellbeing?
• For whom is this support most effective and why?
• Is the support cost effective?”

The specific objectives of the process evaluation, as defined in the Statement of Requirements, were to:

“Assess how the trial processes operate, what are the active elements of the intervention that cause behavioural and health changes and explore key issues which would otherwise not be possible through quantitative techniques.”

Specific areas for investigation for the process evaluation included exploring:

• The operational processes involved in delivering Group Work – referrals, the delivery of the course and relationships between Work Coaches and providers – to identify what has worked well and what less so, and the fidelity of delivery against the prescribed model;
• The experiences of provider staff delivering the course and Work Coaches making referrals to it; and
• The experiences of benefit claimants participating in the course and the benefits resulting for them, and for those declining to attend or not completing the course to explore the reasons for this.

1.3. Methodology

The findings presented in this report are based on two main methodological components:

• Qualitative interviews conducted for the process evaluation with provider staff, Jobcentre Plus staff and benefit claimants, conducted by ICF; and
• A programme of observational and follow-up research conducted by Dr Adam Coutts.

The report also draws upon the findings from the Day 1 to Day 5 survey completed with participants and delivered by Group Leaders. Figure 1.1 provides an overview of the key outcome measures that were used in the evaluation, with additional detail being available in the Impact Evaluation Technical Report.

Figure 1.1: Key measures used in the Group Work evaluation

Work-related measures

* Currently being in paid work.
* Currently being in paid work of 30 or more hours a week (i.e. in full-time work).
* Currently being in paid work that they are satisfied with.
* Currently earning above or below £10,000 per annum.
* Benefit receipt and value of benefit payments.

Job search-related measures

* FIOH (Finnish Institute of Occupational Health) Job Seeking Activity Scale Revised: A measure of how frequently individuals undertake job search activities, and, following revisions, includes two items that measure the number of vacancies applied for and the number of CVs submitted in past two weeks.
* Gaining relevant skills or experience: Measured by whether someone has (a) attended training or courses, (b) done voluntary work, or (c) attended work placements in the previous six months.
* JSSE (Job Search Self Efficacy) Index - Modified: A measure of an individual’s belief that they have the skills to undertake a range of job search tasks.
* GSE (General Self Efficacy) Scale: A broader measure of the strength of an individual’s belief that they are effective in handling life situations.
* Confidence in finding a job: A measure of how confident individuals are of finding a job within 13 weeks.
* Additional job search-related measures used in the evaluation included: their perceived ability to influence the likelihood of finding work, and the influence of personal qualities and experience.

Wellbeing and mental health measures

* WHO-5 (World Health Organisation) Wellbeing Index: A measure of an individual’s wellbeing based on particular feelings experienced in the last two weeks. The WHO-5 can also be used to indicate likely depression.
* ONS4 (Office for National Statistics) Subjective Wellbeing: Four related items measuring individual’s wellbeing based on their subjective happiness, life satisfaction, feeling that life is worthwhile, and anxiety.
* UCLA Loneliness Scale: A measure of an individual’s loneliness.
* LAMB (Latent and Manifest Benefits): A measure of benefits associated with work that can also be used to measure psychosocial deprivation and financial strain.
* PHQ-9 (Patient Health Questionnaire): A measure designed to facilitate the recognition of the most common mental disorders, notably depression.
* GAD-7 (General Anxiety Disorder) Assessment: A measure designed primarily to facilitate the recognition of generalised anxiety.

Wider health measures

* EQ5D-3L (EuroQol Group): A standardised measure of an individual’s overall health.
* EQVAS (EuroQol Group): A measure of an individual’s subjective overall health on that day.
* Visits to a GP in the last two weeks and use of Casualty and Outpatients services in the past three months were also used as measures of overall health.

Details of the two main components of the methodology are provided in the following two sections.

1.3.1. Qualitative interviews with provider and Jobcentre Plus staff and participants - ICF

The process evaluation was based on evidence collected through a programme of in-depth qualitative face-to-face and telephone interviews (undertaken by ICF between July 2017 and January 2018) with a sample of jobcentre and provider staff, and benefit claimants, provided by DWP. The interviewees comprised:

• 15 Provider staff (all the local provider leads and Group Leaders in the trial districts);
• 45 Jobcentre Plus staff (including office managers, office leads/champions, Work Coaches and Disability Employment Advisors in a sample of nine participating jobcentres, with one to two in each trial district); and
• 80 benefit claimants (40 who had completed Group Work, 20 who had started the course but had not completed it, and 20 who had been offered the opportunity to go on the course but declined). The sample included benefit claimants from each of the trial districts and the majority of participating jobcentres within these.

The characteristics of the interview samples are provided in Tables 1.1 to 1.3.

Table 1.1: Provider staff interviewed for the evaluation of the Group Work trial

Districts	Local leads	Group Leaders
Durham and Tees	1	3
Merseyside	1	4
Midland Shires	1	2
Mercia	1	2
Avon, Severn and Thames	1	2
Total	5	13

*The figures in this table total more than the number of interviews conducted because some respondents were working across more than one of the Group Work trial districts.

Table 1.2: Jobcentre Plus staff interviewed for the evaluation of the Group Work trial

Districts	Office managers	Group Work leads/ champions	Work Coaches	DEAs
Durham and Tees	2	2	8	0
Merseyside	0	1	4	0
Midland Shires	0	1	3	0
Mercia	2	0	9	0
Avon, Severn and Thames	2	1	7	2
Total	6	5	31	2

Table 1.3: Benefit claimants interviewed for the evaluation of the Group Work trial

Characteristic	Count
Respondent Type: Completer	40
Non-Completer	20
Decliner	20
District: Avon, Severn and Thames	28
Durham and Tees	11
Mercia	15
Merseyside	13
Midland Shires	13
Benefit: JSA	61
ESA	12
IS	3
N/A	4
Age: 18-30	15
31-49	27
50+	38
Total	80

The interviews were conducted with the aid of topic guides (provided in Appendix A) that were tailored to each group and respondent type. Each interview was digitally recorded, with the respondents’ informed consent, and subsequently transcribed. The interview transcripts were then analysed using NVivo to identify key themes, subthemes, groupings and associations in the evidence.

In addition to the interviews with benefit claimants, provider staff and jobcentre staff, eight less formalised discussions and interviews were also conducted with:

• Each DWP compliance manager (who have been involved in the delivery of Group Work training to jobcentre staff and with an ongoing remit to monitor and advise jobcentres on the delivery of the trial); and
• The Jobcentre Plus leads in each of the five districts.

These were conducted after the interviews with benefit claimants, provider staff and jobcentre staff, to feedback the findings from the interviews, collect any reflections on them, and thereby inform the subsequent analysis and reporting of the findings.^{[footnote 3]}

1.3.2. Observational research and follow-up interviews – Dr Coutts

Dr Coutts began a placement as a research analyst in the DWP/DHSC Work and Health Unit in July 2016, and was directly involved in the design and commissioning of the evaluation, the training of Group Work leaders and Work Coaches, and DWP live-trial support forums and trial team teleconferences. Dr Coutts observed the whole participant journey through the field trial from initial attendance at work focused interviews within jobcentres (at the point of where the survey tool was administered and randomisation), reception interviews with Group Leaders, to participation in the course (Day 1 to 5) and after completion.

Dr Coutts also conducted site visits to all districts with trial compliance managers and staff, which comprised semi-structured interviews with: DWP officials, Work Coaches, Single Points of Contact (SPoCs) and Jobcentre Plus managers (total 40); and Group Work Group Leaders (n=14). These interviews helped capture issues with the delivery and implementation of the trial as well as local labour market and socio-economic conditions. A one-day feedback workshop was held with Group Leaders (n=9) and DWP policy psychologists (n=2) in June 2018. This provided an opportunity to test insights and validate observations of Group Work to those directly involved with its implementation.

Ethical approval for the research described in the observation and follow up chapter (Chapter 6) was obtained from the Department of Sociology, University of Cambridge. Given the sensitivity and ‘live’ setting of the trial, interviews were not recorded and field notes taken instead.

Course observations

The field observations began in April 2017, allowing the intervention to bed in (the trial started in January 2017) prior to any external research. A total of 17 week-long Group Work sessions across all five jobcentre districts were observed, representing over 300 hours of observations between April 2017 and April 2018.^{[footnote 4]} On average between 12 and 15 people attended each weekly session, amounting to the observation of over 200 participants in total.

Table 1.4: General characteristics of Group Work participants interviewed within participant observations

Characteristic	Count
Respondent Type: Course participants	100
District: Avon, Severn and Thames	20
Durham and Tees	20
Mercia	16
Merseyside	22
Midland Shires	22
Benefit: JSA	55
ESA	22
IS	23
Age: 18 to 30	21
31 to 49	46
50+	33

While conducting course observations, semi-structured interviews took place with a convenience sample of 100 participants. Table 1.4 sets out the key characteristics of the participants interviewed, which shows that the sample was broadly distributed by gender, age, duration of unemployment, and reporting or not reporting issues with mental health and wellbeing.

On arrival at each Group Work venue the purpose of the research was explained to the Group Leaders, and then to participants, before consent was sought for the observations on the understanding this was on an anonymous basis.

The qualitative topic guides for participants and Day 1 to 5 survey questionnaires which contain a range of health and wellbeing measures provided an overall framework for conversations with participants and stakeholders. These were used to elicit participants’ views on a range of topics from the links between work, unemployment, health and wellbeing, the latent and manifest benefit measures and how participants felt their health and wellbeing had changed as a result of attending the course. New themes and issues emerged out of these interviews and were further explored with participants across each area. While the topic guides and measures of health and wellbeing were used initially, after a few weeks these were reduced to a series of prompt questions and themes which had developed. The absence of lengthy topic guides also helped to reduce participants’ perceptions that the participant observation was ‘assessing them’ for DWP.

The observations were recorded in writing, with the semi-structured interviews with participants being conducted during break times and before and following the sessions. A typical interview lasted between 30 minutes and one hour, depending on whether it was conducted before or after the course. At the end of each day notes were prepared, including key observations, and with the consent of the Group Leaders and participants, photographs were taken during each session and session materials (such as flipcharts and notes taken by participants) were collected.

The qualitative information gathered from the participant observations and interviews was analysed using the theoretical framework developed and presented in Chapter 2. All materials were reviewed to develop an in-depth understanding of key themes and codes. Themes such as mental health, wellbeing, social isolation, loneliness and the psychosocial characteristics embodied in the Latent and Manifest Benefit (LAMB) scale[^5 ](routine, social support and time structure), were used to code and group quotations and insights from participants, Group Leaders and Jobcentre Plus staff.

Post Group Work interviews

In addition to the course observations,125 semi-structured longitudinal follow-up interviews also took place between April 2017 and April 2019. The cohort is described in Table 1.5 (with the employment status at different points shown as Table 1.6). This was a convenience sample of 72 participants who had completed the course and voluntarily provided their contact details at the end of the Day 5 observations. Due to sample attrition a cohort of the same 25 participants emerged by the end of the fieldwork. These 25 people were each interviewed at one week, one month, three months, six months and 12-months post-participation in order to provide insight into how participants experienced the transition from Group Work into the local labour market and how this affected their mental health and wellbeing. Themes and topics which had emerged during the course observations were explored as well as those covered in the six and 12 months survey questionnaires.^{[footnote 6]}

Table 1.5: Follow up component: Cohort of 25 participants interviewed at one week and one, three, six and 12-months post Group Work participation

Characteristic	Count
District: Avon, Severn and Thames	7
Durham and Tees	4
Mercia	2
Merseyside	7
Midland Shires	5
Benefit: JSA	14
ESA	6
IS	5
Age: 18 to 30	7
31 to 49	12
50+	6
Total	25

Table 1.6: Employment status of follow-up cohort following Group Work participation

Employment status	One week to month 3	Month 3 to month 12
In full-time work	12	8
Temporary / voluntary positions	5	7
Unemployed	8	10

1.4. The structure of this report

The remainder of this report is structured as follows:

• Chapter 2 sets out the policy and evidence context for the trial, and describes the Group Work course as applied in the UK;
• Chapter 3 provides findings on the Jobcentre Plus elements of the trial;
• Chapter 4 provides findings on the delivery of the Group Work course;
• Chapter 5 provides the findings from the quantitative survey on participants’ experiences of the course;
• Chapter 6 provides findings from the observational research and the active elements of the Group Work course;
• Chapter 7 presents our conclusions and lessons learnt from the trial.

The report also features one appendix which contains the topic guides used in the qualitative ICF fieldwork.

2. Policy and evidence context

This chapter sets out the policy and evidence context to the UK JOBS II trial. It summarises pre-existing evidence on the links between unemployment and mental health and on the effectiveness of active labour market policies (ALMPs) in terms of generating employment, health and wellbeing outcomes.^{[footnote 7]} It then provides evidence from international studies of the JOBS II model, in terms of how and for whom it has been found to be effective.

2.1 The ‘genesis’ of Group Work: tackling the relationship between unemployment and mental health

Unemployment and mental health concerns are at the forefront of the current national policy agenda in the UK (Department for Work and Pensions, 2016 and 2017). Since 2013 the Department for Work and Pensions (DWP) and the Department of Health and Social Care (DHSC) have been working in conjunction to explore more joined-up policy approaches and interventions to address the links between unemployment and mental health issues. A key aspect of this approach, has involved using the best available evidence on interventions and designing and testing field trials of interventions that can help people get back to work, promote wellbeing and help reduce mental health issues. These have bolstered policy and academic interest into how mental health and wellbeing are affected by non-health sector policy interventions such as employability provision, ALMPS and back-to-work programmes.

The Government’s disability, health and employment strategy highlighted the prevalence of mental health issues in the population (DWP, 2013). Research evidences shows that at any given time around one in six people has a common mental health condition, such as anxiety or depression (McManus et al., 2016). People with mental health issues are also shown to fare worse in terms of labour market participation:

• The Annual Population Survey (April 2018 to March 2019) showed the employment rate of disabled people aged 16 to 64 who had any mental health condition as their main health condition was 43.6 per cent, compared to 51.4 per cent of all disabled people and 81 per cent of non-disabled population;^{[footnote 8]} and
• Quarterly data to February 2020 shows that 50 per cent of Employment and Support Allowance claimants have a mental and behavioural disorder as their primary registered condition.^{[footnote 9]}

More widely a robust evidence base exists which shows that unemployed individuals are more likely to experience common mental health conditions such as anxiety and depression than those in employment (Kim and Von dem Knesebeck, 2015; Murphy and Athasanou, 1999; Paul and Moser, 2009). In addition, these conditions can themselves hinder attempts to re-enter the labour market after a period of unemployment. Research has found that mental health conditions and low self-esteem can adversely affect job search behaviour and motivation via reduced confidence, making it less likely people will enter employment (Waters and Moore, 2002; Zabkiewicz and Schmidt, 2007; Vinokur and Schul, 2002; Van Hooft et al., 2012). High quality and appropriate re-employment have been shown to counter some of the negative mental health and wellbeing effects of unemployment (Coutts et al., 2014; McKee-Ryan et al., 2005).

2.2. ALMPs, health and wellbeing: an evidence base

Empirical evidence on labour market status, mental health and wellbeing has largely focused on the effects of employment, redundancy, long-term unemployment, or ‘bad’ jobs^{[footnote 10]} and working conditions. However, the evidence is unclear concerning the health and behavioural impacts of the transition between unemployment and employment resulting from participation in employability provision and ALMPs, why these impacts occur, what are the active elements^{[footnote 11]} involved, and who is most responsive in terms of improved mental health, wellbeing and job outcomes (Coutts, 2009; Coutts et al., 2014; Sage, 2014).

A range of ALMPs exist. Their overall purpose is to increase employability and reduce the risk of further unemployment. This is achieved via interventions which offer job search assistance, basic skills and vocational training, as well as wage and employment subsidies. They aim to enhance human capital, labour supply and the general functioning of local labour markets (Coutts, 2009; Kluve et al., 2017). The Organisation for Economic Co-operation and Development (OECD) define ALMPs as ‘..all social expenditure (other than education) which is aimed at the improvement of the beneficiaries’ prospect of finding gainful employment or to otherwise increase their earnings capacity. This category includes spending on public employment services and administration, labour market training, special programmes for youth when in transition from school to work, labour market programmes to provide or promote employment for unemployed and other persons (excluding young and disabled persons) and special programmes for the disabled.’^{[footnote 12]}

Most studies which examine the impacts of ALMPs and the employability provision interventions used to deliver them focus on what are traditionally considered the more tangible economic impacts or outcomes such as job entry rates and time off benefits (Coutts, 2009). Major systematic reviews of hundreds of ALMPs conducted across social and economic contexts have found mixed and small effect sizes in terms of job outcomes (Card et al., 2018; Kluve et al., 2017). Specifically on average ALMPs have relatively small effects in the short-term (less than a year post-programme), but larger positive effects in the medium run (1-2 years post-programme) and longer run (2+ years).^{[footnote 13]} In a meta-analysis of over 800 observational and experimental studies of ALMPs for the general unemployed populations, Card et al. (2018) found short-term (less than a year after the intervention) effects of skills training on an individual’s employment. As noted this effect has been found to multiply over time, leading to around six to seven percentage point increase in employment outcomes over two years.

A major gap identified by Coutts (2009) and Kluve et al. (2017) is the limited reporting of the health and wellbeing impacts of these interventions and what are commonly referred to as the theories of change or active elements embedded within a policy intervention, i.e., what is going on within an intervention that influences health, wellbeing and behaviour changes among those who participate. The small body of international evidence that exists suggests that participation in employability provision and ALMPs can have positive effects on mental health and subjective wellbeing (such as on individual confidence, motivation and self-efficacy).^{[footnote 14]} A recent Cochrane systematic review of health-improving interventions for obtaining employment in unemployed job seekers (Hult et al., 2020) found that out of 15 trials reviewed, ALMPs or ‘therapeutic interventions’ as the authors define them ‘may increase employment, but the evidence is very uncertain’. Further they report ‘there may be no differences in the effects on mental health and on general health. Combined interventions that included therapeutic methods and job-search training probably slightly increase employment compared to no intervention’ (Hult et al., p 2). The review did not report or explore what active elements or intervention processes may be responsible for these effects.

There is also little evidence from the United Kingdom (See Coutts 2009; Andersen, 2008; Sage, 2015b, 2015a; Wang, et al., forthcoming for exceptions), and no major studies involving randomised control trial (RCT) designs. The evidence base in general provides little indication as to what are the active elements embedded within interventions which can generate a change in job search behaviours, health and wellbeing, and subsequently enhance an individual’s employability and therefore generate employment outcomes. Given that hundreds of thousands of people move through employability provision every year in the UK there is a need to understand when ALMPs work, why and for whom.

2.3. How ALMPs may affect health, wellbeing and job search behaviour

In order to understand how ALMPs may affect the health, wellbeing and behaviour of participants, Coutts (2009) proposed a framework combining theories and evidence from work psychology, self-efficacy and the importance of social support, networks and social isolation in maintaining health and wellbeing.^{[footnote 15]} It follows that the experience of unemployment may reduce a person’s access or exposure to a set of psychosocial^{[footnote 16]} characteristics of employment. These are referred to as the latent and manifest benefits of employment, which are important for maintaining and protecting mental health and wellbeing (Kovacs et al., 2019; Waters and Moore, 2002). According to Jahoda (1982) examples of latent benefits include time structure, routine, social activity, social contact and support, collective purpose, regular and purposeful activity and identity (Coutts, 2009). The manifest benefits refer to income and wages.

It is proposed that ALMPs may be able to replicate these psychosocial characteristics of the employment experience, which potentially influence changes in health and wellbeing. As Coutts (2009) notes ‘people are neither employed nor unemployed but occupy an intermediate stage in terms of their labour market status. Within ALMPs, participants are subject to both the negative psychosocial characteristics of being unemployed (such as low income) and positive psychosocial aspects such as a routine and activity, as well as the ALMP active elements such as the type of provision and course content that are used to ‘help them into the labour market’.

For instance, unemployed job seekers frequently report feeling socially isolated and lonely. As the existing evidence base demonstrates, limited social support and social isolation is linked to a range of negative health and wellbeing outcomes (Berkman et al., 2000). The employability provision or ALMP environment provides a context in which the unemployed job seeker is able to meet people and increase social contact, which can reduce feelings of social isolation and loneliness.

Further, Coutts (2009) and Price and Vinokur (2014) propose that interventions themselves are embedded with specific active elements which generate behavioural, health and wellbeing changes. Firstly, ‘process effects’ (Helliwell, 2011) such as the quality of the ALMP trainer or leader in terms of their skill levels and overall credibility has a significant impact on encouraging and supporting participants to develop their own skills and actively participate which can lead to increased confidence and motivation. Secondly, the type of intervention content and material such as whether it involves basic skills, vocational training, role play, job search and networking. Each of these approaches varies in the degree to which they are able to facilitate active participation or promote the more instructive or teacher-led approach which traditionally dominates employability provision. Thirdly, the characteristics of the participants themselves may function to encourage people to support each other and share challenges they face in job search or their daily lives, such as whether they share similar backgrounds and experiences, similar levels of mental health and wellbeing and duration of unemployment. This may have a group psychotherapeutic effect and encourage participants to form bonds and social support which can help them identify ways to overcome challenges, deal with personal issues, setbacks and rejections in job search and reduce feelings of social isolation and loneliness.

2.4. The policy response: test and learn

In order to look at how to improve employment and health prospects for people with common mental health issues the Department for Work and Pensions (DWP) and the Department of Health (DH), now Department for Health and Social Care (DHSC), jointly commissioned Psychological Wellbeing and Work: Improving Service Provision and Outcomes (Van Stolk et al., 2014).

The report concluded that the interaction between mental health and employment is complex and no ‘one size fits all’ approach is appropriate. It argued for more integration between existing treatment and employment services, timely access to coordinated treatment and employment support and application of evidence-based models of support. Four models were proposed for further investigation:

1. Using group work in employment services to build self-efficacy and resilience to setbacks that benefit claimants face when job seeking, based on the JOBS II programme, and which had been advocated by the DWP Policy Psychology Division.

2. Embedding vocational support, based on the Individual Placement and Support model (IPS), in the Improving Access to Psychological Therapies programme (IAPT) or other suitable psychological therapy services.

3. Using Jobcentre Plus-commissioned, third-party provision of combined telephone-based psychological and employment related support for the JSA group or ESA group before they enter the Work Programme.

4. Providing access to online mental health and work assessments and support.

Based on the first three recommendations, a series of small-scale feasibility pilots were established to examine the most effective design of the pilot interventions, plus the most effective delivery mode. Group Work, which is based on the JOBS II model, was a week-long intervention aimed at JSA claimants who were struggling with their job search. It was piloted between August and December 2014 in the two Jobcentre Plus districts of Thames Valley and Gloucester and the West of England.

Findings from the evaluation of the pilot (Callanan et al., 2015) indicated that the intervention had some impacts on participants’ mental health, wellbeing and employment, and should be scaled up in order to robustly test the intervention across different labour market contexts. The pilot study recommended certain changes which were implemented in the final trial. This included training for Work Coaches to enhance their understanding of who was suitable for the intervention, changes to the documentation to reflect language and cultural differences in the UK, and avoiding the use of the term ‘psychological’ support.

2.5. JOBS II intervention model: Evidence from international studies

The JOBS II studies developed in the United States and replicated in a range of other nations are currently the best available evidence on the impact of employability provision on health and wellbeing, and provide the foundation for the establishment of the Group Work intervention.

Developed at the University of Michigan in the United States, the JOBS programme is a theory and research founded intervention originally designed to meet the employment needs of those who had recently experienced job loss and to prevent the onset of mental health issues, particularly depression for those at risk. Alongside specialised job search skills training, the JOBS programme included a range of elements designed to promote self-efficacy, coping strategies and provide ‘inoculation against setbacks’ in the job search process. The programme was tested in two large randomised trials (JOBS I and JOBS II) in the United States and has since been adapted and implemented in trials and pilots across five countries (United States, Ireland, China, the Netherlands and Finland) with positive results in terms of health, wellbeing and job outcomes for some participants across different economic and welfare settings.

JOBS I and JOBS II has created a series of academic publications over the past twenty plus years exploring the impacts and experiences of the intervention across a range of country contexts. These studies have examined the effects of the intervention on a range of mental health, wellbeing, behavioural and employment outcomes. In addition to the initial evaluation (Vinokur et al., 1995), two seminal papers are: Vinokur et al. (2000) in the United States and Vuori et al. (2005) in Finland.^{[footnote 17]} The US study followed up data on 1,801 participants from the earlier 1995 randomised field experiment (JOBS) to assess long-term effects (after two years). The Finnish study assessed outcomes (at six months) of a similar randomised field study of 1,261 unemployed jobseekers. The main differences between the studies, as shown in Table 2.1, were recruitment and eligibility, in particular, the length of time the sample had been unemployed (United States: mean = 4.11 weeks since job loss and no longer than 13 weeks, Finland: 28 per cent of the sample had been unemployed for more than 12 months) and the targeting of specific groups (US: those recently becoming unemployed, and Finland: aimed at previously employed office workers).

Key findings from the US and Finland studies

The US JOBS II study by Vinokur et al. (2000) found that after two years post-programme the experimental group had significantly higher paying jobs and higher quality jobs. They were working more hours and earning greater levels of income, although there was no improvement in the “stability” of the jobs obtained.

In terms of mental health and wellbeing, participants who were assigned to the programme experienced a small but significant overall reduction in depressive symptoms. They also reported improvements in a sense of control and feelings of mastery, which are proposed to be protective for mental health^{[footnote 18]} (Vinokur et al., 2000). Those who participated in the US JOBS II were significantly less likely to experience episodes of depression and mental health issues.

In the Finnish trial of JOBS II (Vuori et al., 2005), participants were more likely to be in stable employment^{[footnote 19]} post-programme at six months and two years later. Participants experienced reduced psychological distress levels compared to the control group. Those who had been unemployed for a moderate length of time (three to 12 months) appeared to benefit the most; they were more likely to have entered a stable job compared to both long-term and recently unemployed groups.

In both studies those at greatest risk of depression and reporting mental health issues at baseline benefited the most (in terms of health and wellbeing) from participating.

Table 2.1: Details of previous JOBS II evidence, methodologies and sample sizes

Methodology	United States (US) Vinokur et al. 1995 (JOBS)	United States (US) Vinokur et al. 2000, 2002 (JOBS II)	Finland Vuori et al. 2002; Vuori and Silvonen, 2005	Group Work Trial UK
Sample characteristics	Eligible participants were those who were unemployed on average for 13 weeks, from Michigan Employment Security Commission in the US and shown no preference to treatment or control programme. Mean age = 35.9 years Proportion of men = 46 per cent Data was collected in four waves: t1: A pretest questionnaire. t2: 1 month after the intervention. t3: 4 months after the intervention. t4: 32 months after the intervention.	Eligible respondents were those unemployed for less than 13 weeks, still looking to find a job, and not expecting to retire within the next two years or to be recalled back to their former jobs in four offices in Michigan Employment Security Commission in the US. Mean age = 36.2 years Proportion of men = 45 per cent Data was collected in four waves t1: pre-test questionnaire. t2: 2 months after the intervention. t3: 6 months after the intervention. t4: 2 years after the intervention.	Eligible participants were unemployed job seekers or had received a termination notice and were searching for a job between September 1996 and June 1997. Implemented during a financial crisis in Finland and high unemployment rates. Mean age = 37 years Proportion of men = 22.2 per cent Duration of unemployment: 28 per cent had been unemployed for over 12 months. Characteristics: sample is higher educated and contains more females than the average unemployed workers. Data was collected in four waves: t1: A pre-test questionnaire. t2: 2 weeks after entry, t3: 6 months after completion t4: 2 years after completion	Eligible participants were those in receipt of unemployment benefits, struggling with their job search in five jobcentre districts in the UK. 51 per cent of the treatment group reported ‘never worked’, and 21 per cent were last in work over two years ago. Mean age = 40.9 years Proportion of men = 63 per cent Data was collected in four waves: t1: Day 1: start of intervention t2: Day 5: end of intervention t3: 6 months after the intervention t4: 12 months after the intervention
Sample size at baseline	Treatment group = 752 Control group = 379	Treatment group = 1249 Control group = 552	Treatment group = 629 Control group = 632	Treatment group = 11,900 Control group = 4,293
Outcome measures	Past job and its quality, attitudes toward work and job seeking, job seeking intentions and behaviour, job-search self-efficacy, mental health and wellbeing, social support and personality dispositions (self-efficacy, locus of control and self-esteem), reemployment, wage rate and job quality.	Depressive symptoms, role and emotional functioning, financial strain, mastery (job-search efficacy, locus of control and self-esteem), sense of self-worth/social undermining, job-search motivation, reemployment.	Re-employment and labour market engagement, job satisfaction, depressive symptoms measured via a Finnish scale based on the Hopkins Checklist and self-esteem measured with a 10-item measure.	WHO-5 wellbeing index, psychological stress (PHQ-9 depression assessment, GAD-7 anxiety assessment), ONS 4, LAMB-12, UCLA loneliness index, general self-efficacy, EQ-5D-3L and EQ-VAS health index. Job search motivation, job search self-efficacy, employment status, full- or part-time, salary, satisfaction with paid work, benefit receipts.
Main findings	(1) One-month follow up of intervention, 33 per cent of participants were re-employed, compared with 26 per cent in control group. Four-month follow up of intervention, 59 per were re-employed, compared with 51 per cent in control group. (2) At one and four months follow up the experimental group reported significantly higher levels of self-efficacy in their job seeking. (3) At one and four months follow up reemployed scored significantly lower on anxiety, depression and anger. (4) At the 32-months follow up, the experimental group had higher earnings compared to the control. In addition, they worked more hours and had more stable work. (5) During the 32-months follow up, the intervention had a preventive impact on both incidence and prevalence of depressive symptoms especially among the high risk people (i.e. those suffering from severe depressive symptoms): 25 per cent for experimental groups compared with 39 per cent for control groups.	(1) Two years after the JOBS workshop, the intervention participants had significantly higher levels of reemployment and monthly income, lower levels of depressive symptoms, lower likelihood of experiencing a major depressive episode, and better role and emotional functioning compared with the control group. (2) Baseline job-search motivation and sense of mastery had both direct and interactive effects on reemployment and mental health outcomes of participants. (3) It was found that those who had initial low levels of job-search motivation and mastery benefited the most. Additional follow-up work at the two-year point with participants from the treatment group recorded lower levels of depressive symptoms and better mental health, and higher levels of working hours, more months working 35 plus hours compared with control group.	(1) Participation in the programme significantly decreased psychological distress at six months, and both decreased symptoms of depression and increased the self-esteem of the participants during the two years of follow-up. (2) Compared to the control group, the participants of the treatment group were engaged significantly more often in the labour market,either employed, or participating in vocational training. At the six months follow up, 23.8 per cent were re-employed, 25.3 per cent in subsidised work, 4.4 per cent in other categories, 39 per cent were unemployed. Reemployment in the experimental and control group was, respectively, 34 per cent and 31.9 per cent. (3) Treatment group participants were significantly more likely to have a stable job than the control group. (4) At two years: 70.4 per cent of the treatment group were re-employed, in a subsidized job or in vocational training, compared to 64.1 per cent in the control group.	See Technical Report on the Impacts of the Trial for UK findings.

2.6. Why JOBS II works and for whom – evidence from international studies

A retrospective analysis of the JOBS II studies by Price and Vinokur (2014), identifies three main components which shaped the experience of intervention participants:

1. The programme itself, the social and psychological components (or active elements to which participants are exposed such as teaching of job search skills, networking and motivational training);

2. The initial motivation and job search intensity of the participants; and

3. Participants’ baseline levels of financial strain, depression, and emotional and social functioning.

As summarised in Figure 2.1, previous JOBS II interventions were found to enhance participants’ mental health, wellbeing and employment outcomes because they improve the self-efficacy of participants which includes enhancing their confidence, sense of control and motivation (Choi et al., 2003; Vuori et al., 2005). This is achieved via participation in group activities, which provide access to social support and teaches participants how to improve their job search skills and cope with job application rejections. In order for the interventions to work a number of enabling conditions are crucial, which the authors state involves ‘creating an atmosphere of trust, an open group climate for sharing ideas, encouragement and…an emphasis on support and absence of criticism…’ (Price and Vinokur, 2014, shown as Figure 2.1).

Figure 2.1: How JOBS II works - What Participants Gain from JOBS II

Source: Price and Vinokur, 2014

2.7. The Group Work/JOBS II model in the UK

This section provides information about the Group Work/JOBS II model in the UK. It includes details of the Group Work trial, the design and delivery of the Group Work course, and the role of the DWP policy psychology division in supporting this.

2.7.1. The Group Work trial

The UK Group Work trial aimed to test and evaluate the JOBS II model in the UK labour market context and with a wider general unemployed population. As detailed in the Technical Report on the Impacts of the Trial (see Section 4.1.2), a larger share of participants in the UK trial were either long-term unemployed, or had never worked, than in many of the previous studies.

The trial was targeted at JSA, Universal Credit (UC), ESA and IS (Lone Parents with child(ren) three years of age and above) claimants struggling with their job search and/or feeling low or anxious and lacking in confidence in relation to their job search. It was not solely focused on those with mental health conditions or issues. The eligibility criteria for the trial were based on the following evidence:

• Research has found that benefit claimants who struggle with their job search or work- related activity include those with deteriorating health, mental health conditions or are in a post treatment situation (see, for example, McManus et al., 2012); and
• Helping unemployed people to actively job search can help them secure appropriate work, which has been shown to be beneficial for their wellbeing (see, for example, McManus et al., 2012).

Participation in Group Work was on an entirely voluntary basis, and no sanctions (reductions in benefit amounts if certain mandatory provision is refused) were applied if benefit claimants decided not to attend or withdrew part way through.

The trial started in January 2017 and expected to support 3,000 starts. In late 2017 the criteria for participation in the project were broadened to include Universal Credit Live Service claimants. As recruitment to the trial was lower than expected, the trial period was extended to the end of March 2018 to allow as many benefit claimants to participate as possible. The trial operated in five Jobcentre Plus districts, with one or two centrally located provider hubs (where the Group Work course is delivered) and a number of participating jobcentres in each district, as summarised in Table 2.2.

Table 2.2: Group Work trial districts, provider hubs and participating jobcentres

Districts	Provider hubs	Jobcentres
Durham and Tees	Middlesbrough	Darlington, Hartlepool, Thornaby, Stockton, Billingham, Loftus, Middlesbrough Central, Middlesbrough East, Eston, Redcar, Guisborough
Merseyside	Liverpool	Aintree, Belle Vale, Bootle, Crosby, Edge Hill, Everton, Garston, Huyton, Kirkby, Norris Green, Toxteth, Wavertree, West Derby, Williamson Sq, Birkenhead
Midland Shires	Derby	Derby Forester House, Derby Normanton
Mercia	Coventry	Coventry Cofa Court, Coventry Tile Hill, Rugby, Leamington Spa, Nuneaton, Bedworth
Avon, Severn and Thames	Bristol	Bristol Central, Bedminster, Bishopsworth, Easton, Horfield, Bath, Shirehampton, Yate, Clevedon, Kingswood, Weston Super Mare
Avon, Severn and Thames	Gloucester	Gloucester, Stroud, Cheltenham, Coleford, Tewkesbury, Cirencester, Cinderford

The Group Work course was delivered by two providers: one covering Durham and Tees and Merseyside districts; and in the other covering the Midland Shires, Mercia and Avon, Severn and Thames districts.

The trial process required Work Coaches in participating jobcentres to recognise suitable benefit claimants based on specific criteria, and administer a survey to collect background data on them. A randomisation tool was used to place individuals into either the intervention or control groups, with the intervention group being invited to join the course and the control group receiving Jobcentre Plus ‘business as usual’ support. Responses to the on-screen survey had no bearing on allocations to the Group Work or control groups, and as the survey and the randomisation process were specific to the trial and would not be included in any wider roll out.

Work Coaches were instructed to introduce the course to benefit claimants after completing the survey and their randomisation into the treatment group, to reduce the likelihood of disappointment and challenge amongst those allocated to the control group. If the benefit claimant accepted the offer to attend the course, a referral was then made to attend an Initial Reception Meeting with the provider serving their District.

2.7.2. Design and delivery of the Group Work course

The Group Work course delivered in the trial is the application of the JOBS II model developed by the University of Michigan and adapted for the UK labour market by DWP policy psychologists. Although the content of the course is focused on job search skills, the underlying processes by which it is delivered are also designed to enhance the self-efficacy, self-esteem and social assertiveness of the participants:

“The job-search skill content is used as a vehicle for helping participants feel competent and confident. It is this confidence that will be the true source of their success.” UK edition of JOBS II Manual (Meehan, Birkin, Ruby and Moore-Purvis, 2015)

The course was led by trained facilitators (known as Group Leaders) using active learning techniques and social support to encourage participation. Benefit claimants referred onto the course were first invited to attend an Initial Reception Meeting (IRM), at which they could meet the Group Leaders and other participants and find out more about what the course will involve. Both the IRM and the full course were delivered at non-Jobcentre Plus venues by a third-party provider, with service level agreements being in place with DWP that benefit claimants would attend an Initial Reception Meeting within five days of a referral and start the course within 15 days.

The course was delivered through five half-day modules, averaging four hours a day, over the course of a working week, with the modules covering:

1. Discovering your job skills;

2. Dealing with obstacles to employment;

3. Finding job openings;

4. CVs, contacts and interviewing; and

5. The complete interview and planning for setbacks.

The job skills content of the course was partly used as a vehicle for achieving its more fundamental aims of increasing participants’ self-efficacy, self-esteem and social resources. This is reflected in the “essential components” of the course, as described in the UK edition of the JOBS II manual:

• Job search skill training - participants are invited to acquire and rehearse job search skills in a safe and supportive environment.
• Active teaching and learning methods - the learning process is almost entirely active, with an emphasis on engagement with the course content, participation by all participants, and collaborative working, and uses the knowledge and skills of the participants themselves, elicited through small and large group discussions, brainstorming and other activities.
• Inoculation against setbacks - the process involves identification of specific problems, generation of possible behavioural or cognitive responses, evaluation of responses, skill acquisition, behaviour rehearsal, try-out and re-evaluation.
• Trainer referent power - when a participant sees and believes that a trainer can be trusted to enhance his or her self-esteem, that trainer will be able to play a key role in promoting positive changes. That is, they will be seen as an authentic and credible source of motivation and influence by participants.
• Social support from trainers and the group – where trainers model and reinforce supportive behaviour, and group exercises provide opportunities for the participants to demonstrate supportive behaviour toward each other.

The UK edition of the JOBS II manual recommend that there are 10 to 20 participants per course, with the groups being led by trained facilitators (preferably, one male and one female, who have some experience or understanding of unemployment), and using active learning techniques to encourage participation. Group Leaders were directed to closely follow the guidance provided in the UK edition of the JOBS II manual in delivering the course (see section 4.1.2).

2.7.3. Involvement in the trial – consort diagram

Figure 2.2 provides an overview of the numbers of benefit claimants involved in the different steps in the trial, from initial recognition and allocation to the treatment and control groups to participation in and completion of the course.

Figure 2.2: Consort steps for Group Work trial

Step 1: Recognised as potential beneficiary and randomisation applied (n=16,193).

Go to step 2 or 3

Step 2: Allocated to treatment group (n=4,293)

Step 3: Allocated to treatment group and offered opportunity to participate (n=11,900).

Go to step 4 or 5

Step 4: Declined opportunity to participate in GW/JOBS II (n=6,545)

Step 5: Initially accepted offer to participate in GW/JOBS II (n=5,355).

Go to step 6

Step 6: Attended Initial Reception Meeting (n= 4,046).

Go to step 7

Step 7: Started GW/JOBS II course, atttended Day 1 (n=2,596).

Go to step 8

Step 8: Completed the course (n=1,922)

2.7.4. The role of the DWP policy psychology division

DWP’s Policy Psychology Division played an important part in the preparation and delivery of the Group Work course, and in ensuring fidelity to the model. The team:

• Revised and updated the JOBS II manual with approval from Professor Richard Price, the JOBS II project leader;  Developed and delivered bespoke training for Work Coaches, Work Coach team leaders, Work Coach trainers, and Trial Integrity and Support Officers;
• Led on the monitoring of provider performance throughout the trial to ensure fidelity to the JOBS II manual, highlight good practice and note areas for development; and
• Undertook a review of the development needs of the Group Leaders across the trial districts and provided training and developmental inputs.

3. Findings on the Jobcentre Plus elements of the trial

This chapter focuses on the elements of the Group Work trial performed by Jobcentre Plus. The main elements were:

• Recognising benefit claimants who would potentially benefit from participating in the Group Work course;
• The administration of an on-screen survey and randomisation with potential beneficiaries;^{[footnote 20]} and
• Introducing, explaining and offering potential beneficiaries who were randomised into the Group Work arm of the trial the opportunity to go on the course.

The chapter also incorporates findings on benefit claimants’ reasons for either accepting or declining the opportunity to go on Group Work when this was offered to them by their Work Coach.

The findings in the chapter are primarily based on evidence from interviews with Jobcentre Plus staff, benefit claimants and providers conducted for the process evaluation. Department for Work and Pensions (DWP) management information from the onscreen survey conducted with benefit claimants prior to randomisation have also been included. Interviews also took place with DWP compliance managers, who were involved in delivering training to jobcentre staff, monitoring and advising jobcentres on delivery throughout the trial and, in its later stages, playing a role in increasing the numbers of benefit claimants recognised. These followed jobcentre and provider staff interviews to triangulate findings and collect additional reflections.

An important caveat concerning the qualitative evidence is that most of the interviews for the process evaluation were conducted between July and September 2017 –seven to nine months into the Group Work trial. As such, the evidence offers a detailed picture of the trial up to this point but not beyond it.

3.1. Training to support delivery

Preparatory training and materials, designed and delivered by the Policy Psychology Division at DWP, was provided to Work Coaches and other Jobcentre Plus staff both as part of preparations for the trial launch and following its implementation.

The initial ‘pre-implementation’ training, delivered between October and December 2016, comprised a series of workshops for jobcentre managers and Group Work leads or champions, followed by half or one day courses for Work Coaches delivered by jobcentre staff in each district.

The aim of the training was that only benefit claimants who were likely to benefit were put forward for the Group Work programme. So an important element of the training was to ensure that Work Coaches were able to recognise those benefit claimants who were likely to benefit. The training was therefore designed to teach Work Coaches to identify and distinguish between evidence of behaviours that indicated a benefit claimant:

• Would be able to job search independently; or,
• Would potentially benefit from Group Work; or,
• Was not currently ready for Group Work, and may benefit from other help (e.g. action with regard to accommodation) before being reconsidered for Group Work.

This required that the training should help Work Coaches to consider the evidence in terms of the overarching criterion - those struggling with their job search and/or were feeling low or anxious and lacking in confidence about aspects of their job search. Using struggling with job search as the referral criteria avoided Work Coaches having to make health decisions or using screening instruments.

As part of the training, Work Coaches were introduced to a pro-forma decision aid that helped with the gathering and recording of behaviours under the headings “work history,” “job search ability” and “interpersonal, communication and wellbeing.” A second pro forma then helped the Work Coaches align the behaviour with whether it would indicate the claimant was capable of “independent job search”, “benefit from Group Work” or “not yet ready”. To help with this process, Work Coaches were provided with a summary desk aid (See Figure 3.1) to assist recognition of individuals who were struggling with job search and would benefit from attending Group Work.

Figure 3.1: Group Work recognition desk aid

Step 1: Does claimant have a new/existing claim for JSA/ESA/UCLS/IS*?

Yes - Go to step 2. No - Go to step 4

Step 2: Is the claimant struggling with their job search?

Yes - Go to step 5. No - Go to step 3

Step 3: Claimant is able to job search independently.

Go to step 4

Step 4: BAU/JobcentrePlus offer/other provision

Step 5: Does the claimant describe/do you observe one or more of these:

• Concerned or worried about completing aspects of job search e.g. what to include in an application form
  
• Feelings of being overwhelmed by the process of looking for work
  
• Expressing a marked lack of confidence about their job search skills
  
• Difficulty managing job search and life events (e.g. relationships, money, etc), at the same time
  
• Worried about how potential employers will perceive them
  
• Responding negatively to job search setbacks (including being despondent) and having difficulty coping with their emotions
  
• Comparing themselves negatively to other jobseekers or their previous job search
  
• Anxious or doubtful about what they have to offer an employer (low self-appraisal of their employment strengths)
  
• Worried that the job search steps are too big/too difficult to cope with
  
• Nervous about travelling to interviews
  
• Anxious about attending interviews
  
• Worried about performing a job like other workers
  
• Worried about building relationships with colleagues or managers
  
• Unable to think ahead and prepare for job search or work setbacks and how to respond
  

Yes - Go to step 6. No - Go to step 4

Step 6: Does the claimant describe/do you observe any of these:

• Issues with aggressive/disruptive behaviour
  
• Issues with being able in a group (e.g. claimant reports feelings of being highly stressed in group environment, able to give example)
  
• Very slow disorganised, incoherent responses
  
• Reports of ongoing drug/alcohol abuse/smells of alcohol
  
• Disruptive behaviour during interview (impulsive/intrusive/inappropriate questions)
  
• Difficulty in attending to/keeping track of conversation or questions in an interview
  

Yes - Go to step 7. No - Go to step 8

Step 7: Not ready for Group Work Programme

Go to step 4

Step 8: Work Coach completes Survey Tool with claimant (including consent section).

Go to step 9

Step 9: Claimant automatically allocated by Random Allocation Tool.

Go to step 10 or 12

Step 10: Claimant allocated to Business as Usual (control).

Go to step 11

Step 11: Inform claimant of JCP offer and other suppport.

Step 12: Claimant allocated to Group Work (JOBS II).

Go to step 13

Step 13: Explain benefits of attending Group Work (JOBS II).

Go to step 14

Step 14: Obtain claimant consent (Consent parts A and B to be completed).

Go to step 15

Step 15: Work Coach action - update LMS/inform SPOC/provider, Data opt-out slip

Source: DWP Group Work/JOBS II Recognition & Response Skills Training Pack

Post-implementation or refresher training also took place from March 2017 onwards. This was reported to be less formally structured than the initial training and followed different approaches across the districts. These included visits from district leads for Work Coach briefings, and in at least one district, included one-to-one coaching on how to make referrals to the course.

The post-implementation training was paralleled with awareness raising activities, including email circulars and other internal communications, discussions at team meetings, and posters. In some cases, interviewees also reported receiving visits from the Group Work provider in their district, and in others Work Coaches had observed the course being delivered, although this exposure to the course was not uniform across the participating districts.

3.2. Recognising potential beneficiaries

This section considers the effectiveness of this element of the trial from two main perspectives:

• The number of potential beneficiaries of Group Work recognised; and
• The characteristics of the potential beneficiaries recognised.

3.2.1. The number of potential beneficiaries recognised

The target of the trial was for a total of 26,000 benefit claimants to be recognised as a potential beneficiary of Group Work and randomisation applied. At the end of March 2018 when the trial had finished, 62 per cent of this target had been achieved, with a total of 16,193 benefit claimants recognised by Work Coaches as potential beneficiaries. Figure 3.2 shows the share of recognitions in each month of the trial.

Figure 3.2: Percentage of total potential Group Work beneficiaries recognised by trial month

Trial month	Percentage
Jan-17	1.4
Feb-17	4.1
Mar-17	6.9
Apr-17	6.4
May-17	6.1
Jun-17	6.7
Jul-17	9.7
Aug-17	9.1
Sep-17	9.8
Oct-17	7.2
Nov-17	9.0
Dec-17	5.6
Jan-18	6.2
Feb-18	7.2
Mar-18	4.4

Source: DWP management information

The month-by-month fluctuations in recognitions during the first seven to nine months of the trial, when the evaluation fieldwork took place, and the overall shortfall in numbers are consistent with the interview evidence from the process evaluation.

Work Coaches interviewed in all the trial areas reported recognising few potential beneficiaries in the first two months of the trial, but due to a combination of factors, most said this had increased from then onwards. Equally, some challenges were reported by Work Coaches that had reduced the number of potential beneficiaries identified throughout the first seven to nine months of the trial (and it is feasible they continued to exert an influence beyond this).

These various factors and challenges are summarised in Figure 3.3 and then evidence on each is presented.

Figure 3.3: Factors influencing numbers of potential beneficiaries recognised

Source: Process evaluation interviews with Jobcentre Plus staff

The pre and post implementation training

Work Coaches were generally positive about the quality, coverage and appropriateness of the pre-implementation training they had received, particularly in terms of the theory underpinning Group Work and the evidence of its impact in other countries (which many used when introducing the course to benefit claimants). One factor perceived to have limited the effectiveness of the pre-implementation training was its timing, and specifically the intervening Christmas and New Year period between when it was delivered and the start of the trial:

“For many people that was so far in advance of them actually doing it, that they’d forgotten.” (Jobcentre Manager; Mercia)

There were also some Work Coaches who said they did not recall receiving any training on Group Work before the start of the trial. These issues with timing and coverage are likely to partly explain the low number of potential beneficiaries recognised in the first two months of the trial.

The post-implementation refresher training, delivered from March 2017 onwards, was seen to have been effective in extending awareness of Group Work to Work Coaches who may not have participated in the pre-implementation training. It was also beneficial in providing a reminder of the course for those who had attended the previous training. It was also reported to be helpful in clarifying and expanding upon elements of which they were uncertain. Hearing from Group Leaders or observing Group Work being delivered was also reported to have been particularly effective in making Work Coaches more aware of what made the course distinctive from other provision:

“It was a real eye opener to have that kind of insight into it from people who delivered it…he left us in no uncertain terms about what it was about, what its strengths were, and that was very enlightening.” (Work Coach; Midland Shires)

Several Work Coaches said the post implementation training and this wider exposure to the course had been the trigger for them to start recognising potential beneficiaries, and a spur for others to recognise more as the trial progressed.

Interpretations of the criteria for recognising potential beneficiaries

Rightly or wrongly, jobcentre staff said they had understood from the pre-implementation training that Group Work was intended for benefit claimants with, or at risk of developing, mental health issues. Several Work Coaches expressed doubts over their ability to recognise benefit claimants who may fall into this category and/or discuss mental health issues with them and indicated this was an initial barrier to them recognising potential beneficiaries.

The post-implementation training sought to clarify the trial referral criteria, and specifically that the course was “…aimed at benefit claimants who are struggling with their job search or work-related activity and/or are feeling low or anxious and lacking in confidence. They may or may not have mental health issues”.^{[footnote 21]} Although DWP policy psychologists note that this was consistent with the pre-implementation training, it was interpreted by many Work Coaches as a change in the criteria:

“Initially because the training went through the customers it was targeted towards, there was a very slow uptake, but I think now they understand that it’s really for anyone concerned about their job search.” (Jobcentre Lead; Durham and Tees)

It was also reported that, following the post-implementation training, senior staff in some jobcentres promoted a wider interpretation of the criteria to their Work Coaches:

“We’re being asked to consider everyone. It’s if you’ve got the time pretty much do it with everyone that you can.” (Work Coach; Avon, Severn and Thames)

In one of the jobcentres this included new claims, while Work Coaches in another jobcentre had been advised that every claimant with a claim of 13 weeks or longer should be considered as a potential referral, and a member of staff had been assigned to review Work Coaches’ caseloads and flag benefit claimants meeting this criterion.

Drives to increase numbers

Jobcentre managers stressed they had not been given targets for the number of potential beneficiaries they should recognise. Equally, they described being informed after the first three to six months of the trial that the number of benefit claimants identified were below forecast and being given a rough estimate of how many benefit claimants their jobcentre was expected to recognise.

This had filtered down from jobcentre managers to Work Coaches, with managers reinforcing the high profile and importance of the trial and encouraging Work Coaches to recognise more potential beneficiaries. However, more exceptionally, some thought that a degree of pressure had been put on Work Coaches in their jobcentre:

“I think because of the pressure that’s on them to fill the provision they [the Work Coaches] are probably a bit sick of it to be honest. But I think that’s, that’s because it is constant pressure.” (Jobcentre Lead; Durham and Tees)

Work Coaches in some jobcentres also described being told by their managers to recognise at least one potential beneficiary per week (on the basis that if replicated the expected numbers for the jobcentre would be achieved). These various measures are likely to partly explain the further increase in the number of potential beneficiaries being recognised in the trial from July 2017 onwards.

Time constraints

These constraints related primarily to the time taken to complete the onscreen survey with benefit claimants after they had been recognised as a potential beneficiary but prior to randomisation.^{[footnote 22]} Work Coaches reported that this generally took 5 to 10 minutes to complete but could take longer (e.g. 15 to 20 minutes) for benefit claimants needing help to complete it. As Work Coaches generally have 10 to 15 minutes per appointment and little spare time in their diaries, this was considered a significant barrier:

“Sometimes people don’t feel they have time to even think if this is appropriate, never mind do it, that’s the more difficult part.” (Jobcentre Lead; Avon, Severn and Thames)

These time constraints were reportedly the most acute in jobcentres with pre-existing staffing pressures, such as sick leave or unfilled posts.

Work Coaches have some flexibility to undertake longer appointments, and in several jobcentres had been advised to do so with potential referrals. However, they said this flexibility only stretched so far and that accommodating more than one or two longer appointments a week was difficult. Two jobcentres reported developing their own approaches to mitigate this. One had enlisted their Disability Employment Advisors to help claimants complete the survey and another had delegated this responsibility to a single Work Coach with special dispensation for longer appointments.

Despite these localised efforts, time constraints were perceived to have contributed to lower numbers of potential beneficiaries being recognised in every trial district during the first seven to nine months of the trial. It is, however, worth noting that if Group Work were to be rolled out the survey would not be required.

Benefit claimant feedback

Receiving feedback from participants, the majority of which was reported to be positive, had encouraged Work Coaches to recognise more potential beneficiaries as the trial progressed:

“Because the feedback I was getting off the people that had come back from the course, I just thought, well give everyone a chance.” (Work Coach; Merseyside)

As the quotation suggests, some of the Work Coaches considered an increasingly diverse range of benefit claimant types (such as the long-term unemployed and relatively new claims) on the basis of the positive feedback received.

Underlying Work Coach attitudes

This was the primary factor cited by jobcentre managers and leads to explain differences in the number of potential beneficiaries recognised by individual Work Coaches. This partly related to the experience and confidence of the Work Coach - i.e. with the more confident generally being more willing to recognise potential beneficiaries - although as interpretations of the criteria changed this difference had fallen away.

A more enduring factor was the outlook of individual Work Coaches. While the majority were said to engage with and be enthusiastic about new interventions, a minority of Work Coaches were said to be less enthusiastic and hard to engage in any new initiatives introduced, which had been reflected in their attitude to Group Work. Seven to nine months into the trial, certain jobcentre managers and leads said they were aware of Work Coaches in this category who had still not recognised any potential beneficiaries.

3.2.2. Characteristics of the potential beneficiaries recognised

Data from the onscreen survey^{[footnote 23]} conducted with potential beneficiaries at the point of randomisation provide some broad indications of the extent to which their characteristics matched the intended criteria (i.e. struggling with their job search or work-related activity and/or were feeling low or anxious and lacking in confidence about their job search).

The onscreen survey data showed that at the point of randomisation almost two thirds (62 per cent) of potential beneficiaries thought they were certain or likely to find a job within 13 weeks. In addition, 53 per cent agreed or strongly agreed that their personal qualities would help them find work, and 43 per cent that their experience was in demand in the labour market. Perceived barriers to finding and sustaining work included a lack of confidence or motivation (22 per cent), lack of skills (23 per cent), and their qualifications (27 per cent).

The data also provided insights into potential beneficiaries’ satisfaction with life, feelings of self-worth, happiness and anxiety:

• 67 per cent reported low or medium levels of life satisfaction;
• 56 per cent reported low or medium levels of self-worth;
• 60 per cent reported low or medium levels of happiness; and
• 52 per cent reported medium or high levels of anxiety.

Table 3.1 shows results from the onscreen survey for all the potential beneficiaries recognised by Work Coaches during the trial and broken down by each quarter.

Table 3.1: Characteristics of potential beneficiaries recognised

Question 1: Which of the following best describes your confidence in getting a job within 13 weeks?

Response	Total %	Q1 %	Q2 %	Q3 %	Q4 %	Q5 %
Certain that I will find a job	14	10	13	16	14	17
Likely that I will find a job	48	42	45	48	49	52
Unlikely that I will find a job	30	39	35	29	28	26
Certain that I won’t find a job	8	9	8	8	8	6

Question 2: How much do you agree or disagree: My personal qualities make it easy to get a new job?

Response	Total %	Q1 %	Q2 %	Q3 %	Q4 %	Q5 %
Strongly agree	10	9	10	10	10	10
Agree	43	39	40	45	42	46
Slightly agree	25	26	26	24	27	24
Slightly disagree	9	11	9	8	9	8
Disagree	10	12	10	8	10	10
Strongly disagree	3	3	4	4	3	3

Question 3: How much do you agree or disagree: My experience is in demand in the labour market?

Response	Total %	Q1 %	Q2 %	Q3 %	Q4 %	Q5 %
Strongly agree	7	6	7	8	7	8
Agree	36	32	33	39	36	38
Slightly agree	24	23	25	22	24	24
Slightly disagree	11	11	12	11	11	10
Disagree	17	21	18	15	17	15
Strongly disagree	5	6	5	5	4	4

Question 4: Which of the following would you say is currently affecting your ability to find/keep work?

Response	Total %	Q1 %	Q2 %	Q3 %	Q4 %	Q5 %
Lack of motivation/confidence	22	28	23	20	23	21
Lack of skills	23	29	24	23	22	22
My qualifications	27	32	28	27	26	24

Question 5: Overall, how satisfied are you with your life nowadays? Where 0 is ‘not at all satisfied’ and 10 is ‘completely satisfied’?

Response	Total %	Q1 %	Q2 %	Q3 %	Q4 %	Q5 %
0-4 (low)	32	36	34	32	33	30
5-6 (medium)	35	35	36	35	34	35
7-8 (high)	23	22	22	23	24	25
9-10 (very high)	9	7	9	10	9	9

Question 6: Overall, to what extent do you feel the things you do in your life are worthwhile? Where 0 is ‘not at all worthwhile’ and 10 is ‘completely worthwhile’?

Response	Total %	Q1 %	Q2 %	Q3 %	Q4 %	Q5 %
0-4 (low)	25	27	26	23	25	23
5-6 (medium)	31	33	32	30	31	32
7-8 (high)	30	27	28	30	30	30
9-10 (very high)	15	13	14	16	15	15

Question 7: Overall, how happy did you feel yesterday? Where 0 is ‘not at all happy’ and 10 is ‘completely happy’?

Response	Total %	Q1 %	Q2 %	Q3 %	Q4 %	Q5 %
0-4 (low)	33	37	33	32	33	31
5-6 (medium)	27	28	27	26	26	27
7-8 (high)	25	22	24	25	26	26
9-10 (very high)	16	13	16	17	15	16

Question 8: Overall, how anxious did you feel yesterday? Where 0 is ‘not at all anxious’ and 10 is ‘completely anxious’?

Response	Total %	Q1 %	Q2 %	Q3 %	Q4 %	Q5 %
0-1 (very low)	29	26	28	29	29	31
2-3 (low)	18	17	19	18	19	19
4-5 (medium)	22	25	23	22	22	22
6-10 (high)	30	32	31	31	30	28

Base: Onscreen survey – All potential beneficiaries identified and completing survey

Response	Total %	Q1 %	Q2 %	Q3 %	Q4 %	Q5 %
Base	16,193	2,068	2,910	4,502	3,638	3,075

When viewed across the five quarters of the trial it can be seen that, on the basis of data from the onscreen survey, the profile of the potential beneficiaries being identified changed in some respects over time. For example, the proportion who thought they were certain or likely to find a job within 13 weeks increased from just over a half (52 per cent) in Quarter 1 to over two thirds (69 per cent) in Quarter 5. Similarly, those reporting motivation/confidence, a lack of skills or their qualifications as barriers to finding or keeping work reduced between Quarter 1 and Quarter 5. Overall, less variation can be observed for the well-being questions on feelings of self-worth, happiness, life satisfaction and anxiety.

This suggests that in terms of their self-perception at least, the share of potential beneficiaries who appeared to be struggling with their job search reduced over time (using confidence in finding work in 13 weeks as a proxy), as did the shares reporting specific barriers to doing so. This would suggest that Work Coaches applied an increasingly broad interpretation of the selection criteria as the trial progressed.

The interviews conducted for the process evaluation, provide some further qualitative evidence on the characteristics of benefit claimants recognised as potential beneficiaries of Group Work (at least at the point in the trial when the research was conducted). Providers offered mixed opinions on this. Some thought that the majority of referrals received were suitable:

“I think in the main, they’re appropriate.” (Provider Lead)

There were others who said they thought people who possibly were not the intended audience for Group Work were being recognised by Work Coaches as potential beneficiaries, including those who were not struggling with their job search or with low confidence:

“The trial is set up for people who are worried and anxious about their job search or are feeling low and lacking confidence. Some of the people that we see, that’s not the case for them. They know where they’re at and what they’re doing and they’ve got a plan. So they’re not really the people we’re looking for, they’d probably benefit from a much more general type employability initiative.” (Provider Lead)

A perceived shift in the characteristics and suitability of the benefit claimants being recognised as potential beneficiaries and subsequently going on the course since the start of the trial was also reported:

“Now anybody can come on the course. We’ve got three people on there who have got really low confidence, been out of work for a long time, very low self-esteem. And then the rest of the people on the course haven’t been out of work very long, the only thing that they really need help with is a CV.” (Group Leader)

Some Work Coaches also perceived a potential tension between recognising sufficient numbers of benefit claimants and their suitability for the course:

“At the moment we’re being told that we’ve got to do one referral a week and you’re not necessarily going to see a customer that’s fitting the criteria in that week, so you’re being made to feel that you’ve got to do one referral regardless of if they’re suitable really.” (Work Coach; Avon, Severn and Thames)

Taken together, the survey and interview evidence indicates that the perceived widening of the criteria, drives to increase numbers, and to some extent the effect of positive feedback reported in Section 4.1, did lead to an increasingly broad range of benefit claimants being recognised as potential beneficiaries as the trial progressed.

However, the effects of this are not clear cut. As later sections of the report describe, some benefit claimants who were offered the opportunity to participate in the course and declined (or went on it but did not feel it benefited them greatly), indicated this was because they were not struggling with their job search or with their confidence and mental health. Equally, some benefit claimants of this profile had agreed to go on the course and did report benefits. From a course delivery perspective, providers also reported that a mix (albeit not too much of a mix) in participant characteristics within each group on the course was generally a positive for its effectiveness.

3.3. Introducing and explaining Group Work

This section considers the effectiveness of this element of the project from two main perspectives:

• The extent to which the process for introducing and explaining Group Work was followed by Work Coaches; and
• How effective Work Coaches were in encouraging potential beneficiaries to take up the offer of participating in Group Work.

3.3.1. The process for introducing and explaining Group Work

The intended process was for Work Coaches to introduce and explain the course after benefit claimants had been recognised as potential beneficiaries, had been asked to complete the onscreen survey, and been randomised into the Group Work arm of the trial.

The interviews with Work Coaches and benefit claimants indicated that - six to nine months into the trial - this process was being followed in most cases. Some deviations from the process were reported, particularly in the first few months of the trial, but at the time of the process evaluation interviews these were the exception rather than the rule amongst Work Coaches in every trial district.

At the start of the trial, some Work Coaches said they had been introducing and explaining the course to benefit claimants before the onscreen survey and randomisation had been applied. This had been due to misunderstanding of the intended process which had been addressed through post-implementation training.

More broadly, fidelity was reinforced by the belief amongst most Work Coaches that the intended process was the most effective approach. Introducing the course only after randomisation was recognised as avoiding the risk of raising benefit claimants’ expectations if they were not randomised in. Work Coaches switching to this approach thought it worked better than introducing the course before the survey. However, a small number of Work Coaches said they were still introducing the course before the survey at the time of the process evaluation research, apparently unaware that this was not the intended approach.

The only other differences in approach still evident at the time of the process evaluation research can be viewed more as pragmatic modifications to the intended process rather than clear instances of it not being followed. Work Coaches in two jobcentres said they were following the intended chronology of the process but conducting it over two appointments rather than a single appointment, as had been envisaged in the trial design. The onscreen survey and randomisation were being conducted in an initial appointment, and then the course introduced and explained to the individual in a subsequent appointment. The main reason given for doing this was to reduce the need for long appointments to conduct the process all in one go.

3.3.2. Encouraging take up of Group Work

Across the trial as a whole, DWP management information shows that 45 per cent of the benefit claimants identified as potential beneficiaries and offered the opportunity to go on Group Work/JOBS II initially accepted the offer – a total of 5,355 benefit claimants (of whom 2,596, 22 per cent of all those recognised, subsequently started the course - see 3.4.3 for more detail).

Figure 3.4 shows the take up rate (the proportion of benefit claimants offered the opportunity to go on the course and agreed to do this) in each month of the trial.

Figure 3.4: Take up of Group Work in each month of the trial

Month	Percentage
Jan-17	56
Feb-17	56
Mar-17	57
Apr-17	52
May-17	51
Jun-17	46
Jul-17	45
Aug-17	37
Sep-17	39
Oct-17	39
Nov-17	42
Dec-17	39
Jan-18	44
Feb-18	43
Mar-18	54

Source: DWP Management Information

There were fluctuations in the take-up rate month-by-month during the trial (from a low of 37 per cent to a high of 57 per cent) but no clear overall upward or downward trend across the 15 months as whole. The effectiveness of Work Coaches in introducing and explaining the course is also only one potential determinant of these levels. The suitability of claimants that were initially being recognised as potential beneficiaries and the appeal of the course in its own right would also have exerted an influence.

In the process evaluation interviews, senior Jobcentre Plus staff did perceive that Work Coaches had an influence on whether or not benefit claimants took up the Group Work offer. There were also reported differences in the approach of individual Work Coaches to introducing and explaining Group Work to benefit claimants. By triangulating this with findings from the interviews with benefit claimants and providers it is possible to draw some indicative conclusions about how effective approaches were in encouraging claimants to take up the Group Work offer.

What was reported to be most effective was:

• Emphasising the difference of the course - several Work Coaches talked about doing this, particularly with benefit claimants who had been on previous Jobcentre Plus provision and potentially saw little value in going on something that sounded very similar:

“It’s sometimes difficult because they say, well I’m sure I’ve done something like that before, I don’t want to be doing that again, so it’s trying to tell them that it’s something different.” (Work Coach; Durham and Tees)

Benefit claimants also indicated they had been receptive to this:

“My Work Coach had said it was like a new course and you do see things from a different perspective.” (Completer; Durham and Tees)

Equally other benefit claimants (and some providers) indicated this was an area where Work Coaches could potentially do more:

“What do they need to say to people to convince them that it’s not just the same? Tell them it’s a different kind of course and they treat you more like adults and they don’t think they’re better than you, instead of telling them nothing, like what they done with me.” (Completer; Avon, Severn and Thames)

“I think there needs to be a better understanding from job coaches about why we’re running, why we’re doing this, what they are going to benefit from.” (Group Leader)

Work Coaches who had observed the course being delivered or received a talk on it from a Group Leader appeared to be the most able and willing to convey to benefit claimants that the course was genuinely something different.

• Using benefit claimant feedback and international evidence - most benefit claimants recalled being told about the impact of the course on others, either based on evidence from other countries or about other benefit claimants who had attended previously. They indicated this had encouraged them to go on the course:

“All I can remember is that she had had recommendations for it from people that had been on it, the feedback had been good.” (Completer; Mercia)

Benefit claimants appear to have been most receptive where the Work Coach employed a combination of personalisation and encouragement – i.e. by introducing the course as something that could benefit them personally:

“I think he said it’s been successful in other parts of the world. And you may well find it of benefit to you. So, yeah, he did sell it but it was all a bit subtle.” (Completer; Avon, Severn and Thames)

• Tailoring the explanation of the course to the benefit claimant - some but not all Work Coaches talked about tailoring their explanation to the specific issues or challenges an individual had, for example by picking up on something the benefit claimant had said and using that as a hook for introducing the course:

“I try to pick up on something they’ve said and run with that.” (Work Coach; Avon, Severn and Thames)

“My confidence was shattered, as I say, and told him how I was feeling and [the work coach] said, right well this course might help you, it helps build your confidence.” (Completer; Merseyside)

The effectiveness of Work Coaches in encouraging participation in the course appeared to be determined by the following main factors:

• The pre and post implementation training - with Work Coaches drawing on information from the training to describe the basic features of the course, including the international evidence of effectiveness. However, some reported that an incomplete understanding of the detail of the course, and what made it distinctive, had undermined their ability to encourage attendance.

• Their exposure to the course - observing a day of the course, or alternatively receiving briefings from Group Leaders, had given Work Coaches additional knowledge about the course and its distinctive nature, made them better able to describe it, and better equipped to address any questions raised. Work Coaches who had been exposed to it, indicated communicating a sense of personal enthusiasm when introducing the course. This was less evident amongst Work Coaches who had not benefitted from such exposure, and it was clear that some were yet to be fully convinced of what was so different or good about the course.

• Feedback from benefit claimants - similarly, Work Coaches reported gaining additional knowledge of the course from benefit claimants who had been on it and using these individuals as examples of the effectiveness of the course.
• Their personal skills, experience and attitudes - the influence of this factor was more subtle and indirect, with jobcentre managers saying that although each had their own approach and that describing and selling provision to benefit claimants was a core part of every Work Coach’s skillset. Nonetheless, one or two differences can be determined from what Work Coaches reported. For example, picking up on something a benefit claimant has disclosed and using this as a way in to introduce the course was a technique only some reported using.

3.4. Reasons for accepting or declining the Group Work offer

This section explores the factors influencing benefit claimants’ decision-making around whether or not to attend the course when initially offered the opportunity to do so and their subsequent participation.

3.4.1. Reasons for accepting

Desire for help with job situation

The most common reason cited by benefit claimants for deciding to attend was to receive help with getting a job:

“I just really wanted to get a job, so that was the bottom line of it really and I thought if anything positive can come out of it at all then it’s worth giving it a try.” (Completer; Mercia)

Even in cases where benefit claimants said they had received vague or unclear descriptions of the course, this overriding motivation to progress back into work had been enough for most to accept. These benefit claimants were generally but not exclusively receiving Jobseeker’s Allowance (JSA). They evinced a general willingness to try “anything that will help” and an open-mindedness about seeing what the course could do for them.

Desire for help with general state of mind

Some benefit claimants were motivated to accept the offer because they thought it might help with their motivation, their self-esteem, or just their general outlook on life. This was not confined to those with a disclosed mental health condition or issue, although a small number did fall into this category:

“I was feeling a bit low at the time and then my Work Coach suggested that the course might be helpful. So I said to him that I would definitely give it a try, I’ve got nothing to lose.” (Completer; Durham and Tees)

As discussed in Sections 3.3.1 and 3.3.2, Work Coaches were not generally describing the course to benefit claimants in terms of it helping them with their mental health. However, these examples indicate that the Work Coaches concerned were effective in communicating that the course was not just about job search skills.

Desire for mental stimulation

Some benefit claimants said they had been partly motivated to accept the course offer because it gave them a reason for “getting out of the house”. The course was seen to offer them something to do and somewhere to go. For example, benefit claimants said they accepted the course out of “boredom” with their day-to-day routine, whilst others described it as an opportunity to meet new people:

“I was sick of being in the house bored of doing nothing so I thought I may as well make the most of my time and go to do something that’s going to be productive.” (Completer; Merseyside)

To meet assumed expectations or requirements

Exceptionally, a small number of benefit claimants indicated that they had agreed to attend because they assumed this was expected or required as part of their benefit claim. Although they did not say that their Work Coach had explicitly told them attendance was mandatory, they had evidently reached the conclusion that their benefits could be affected if they did not accept:

“If I’d have said no, they would have sanctioned me money.” (Completer; Merseyside)

Notwithstanding these findings, the voluntary nature of the course was both understood and strongly welcomed by most benefit claimants.

3.4.2. Reasons for declining

Interviews with Work Coaches and claimants offered but not taking up the course identified a series of reasons for benefit claimants declining to attend.

Perception the course would not help

Some benefit claimants said they had turned down the offer of going on the course because they did not believe it would help them and would not offer them anything that they did not know already. This was cited mainly by older benefit claimants 12 months or more into their claim, and who had previously been referred to jobcentre provision:

“I’ve done all that. So I weren’t interested.” (Decliner; Avon, Severn and Thames)

Work Coaches also highlighted benefit claimants in these circumstances as being particularly difficult to convince of the value of going on the course.

Discomfort with the group setting

Some benefit claimants said they turned down the offer of going on the course because they did not feel comfortable in a group setting:

“I said at the time that I wouldn’t be interested in it at all because I don’t like being in a room with strangers and people like that and I would get nervous.” (Decliner; Midland Shires)

For others concerns about the group setting could be more severe. For example, one benefit claimant interviewed had been diagnosed with social anxiety disorder and said they had suffered a severe negative reaction when attending a previous group-based course.

Health reasons

A small number of benefit claimants, each of which were claiming Employment Support Allowance (ESA) at the time of interview, said they declined the course for health-related reasons. These were either because they did not think they were well enough to attend, or that they had a fixed medical appointment that clashed with the timing of the course:

“I’d love to try and do all these things, but I just physically can’t.” (Decliner; Durham and Tees)

Non-financial concerns about travel

Benefit claimants said they had all been aware that their travel expenses would be paid, and some cited this as a valuable enabler for them to attend the course. A small number who had declined said this had partly been because of the physical challenges of travelling to the venue where the course was being delivered. These were benefit claimants who were reliant on public transport and reported having a health condition, which made it uncomfortable for them to either change buses a number of times or stay sitting down on a bus for an extended period of time.

Other personal commitments or crises

In a small number of cases other reasons were cited for not accepting the opportunity to attend the course. One individual said she had turned down the offer because it was during the school summer holidays and she had a child who she was looking after at home. Another benefit claimant said his wife had been ill and he was looking after their children at the time so had declined the opportunity to attend for this reason. A further benefit claimant had very recently been made homeless and was trying to resolve a dispute over his benefit entitlement when he was told about the course. He had felt unable to accept the offer until he had addressed these more immediate concerns.

3.4.3. Subsequent participation in the course

Although 45 per cent of those offered agreed to go on Group Work/JOBS II when it was initially offered to them by their work coach, not all subsequently participated in the course. Thirty-four per cent attended the Initial Reception Meeting (IRM) that preceded the course, and 22 per cent attended day one of the course itself.^{[footnote 24]}

It is not possible to fully determine the reasons for this “attrition” in take up and participation. The process evaluation included interviews with benefit claimants who had declined the offer when it was made by their work coach, and those who had accepted the offer and went on to attend, but did not capture those who had initially accepted then not attended. The six-month survey included one question for respondents who were offered the opportunity to go on the course but had not attended about why this was. The survey did not differentiate between those who had initially accepted then not attended and those who had declined the offer outright. Nonetheless it is likely to have captured both and provides some broad indications of why not all who accepted the offer went on to participate in the course:

• Health or other personal commitments or crises intervened in benefit claimants’ lives after they had initially accepted the offer to go on the course and prevented them from attending. Health issues or ‘not feeling well enough’ was the most widely cited reason for not attending the course in the survey; and
• Benefit claimants had found work after they had initially accepted the offer to go on the course meaning they would not have the time to attend and/or no longer perceived the need to do so. Clashes with paid work was the second most widely cited reason for not attending the course in the survey.

It is also feasible that some benefit claimants had only given a “soft yes” at the point they were initially offered the opportunity to go on the course and had second thoughts after this point, e.g. on reflection deciding it was not going to help them.

The role of the IRM in supporting participation in the course is not straightforward. Benefit claimants who attended the IRM and then participated in the course were overwhelming positive about the value of it in aiding their understanding of the course and motivating them to attend. However, the provider staff who ran the IRMs confirmed that a minority of benefit claimants decided not to attend the course at that point - typically those who had not understood the voluntary nature of the course until this had been explained to them at the IRM. Neither participants nor Group Leaders necessarily viewed this as a negative. There was a shared view that participants “had to want to be there” for the group dynamics on the course to function effectively.

While the IRM may have contributed indirectly to lower levels of participation, it may equally have performed a necessary role in reinforcing the fidelity and effectiveness of the trial.

4. Findings on the Group Work Course

This chapter reviews provider and participant experiences of the Group Work course, including the key influences on course retention and drop-out, and the perceived outcomes reported. It draws upon interviews with course providers and participants, and with jobcentre staff when reviewing the outcomes reported.

4.1. Experiences of the course

4.1.1. The Initial Reception Meeting (IRM)

The Initial Reception Meeting (IRM) was intended to provide benefit claimants with an overview of the course, what they can expect to gain from it, the opportunity to visit the course venue, and to meet the Group Leaders in advance. The IRM could also play an important role in addressing benefit claimants’ questions or concerns about the course, as well as confirming its voluntary nature.

The sessions were delivered by one of the Group Leaders who explained what the course involved, answered questions from benefit claimants, and asked them to confirm they will attend the full course. Sessions were delivered on a group or individual basis:

• In Avon, Severn and Thames, Midland Shires, Durham and Tees, and Mercia the IRM was delivered to small groups of benefit claimants in one to one and a half hour sessions; and
• In Merseyside it was being delivered on a one-to-one basis (referred to as a ‘Reception Interview’) lasting 20 to 30 minutes.

Provider staff reported that, in general, benefit claimants were invited to attend an IRM or interview within 10 days of receiving a referral, and that a degree of choice was offered over when they attended. The benefit claimants interviewed described attending meetings between one and two weeks after referral, although a small number could not recall attending a meeting before joining the course. They typically found the IRM or interview useful, particularly where they were not clear of what the course would involve. For example:

“He went through all the details. Made sure that I knew exactly what I was getting myself involved with and tried to clear up any preconceptions before we even started.” (Completer; Merseyside)

“They made it feel a lot better. After that induction I knew what I was actually doing.” (Completer; Durham and Tees)

While for most benefit claimants the IRM confirmed or slightly reinforced their motivation to attend the course, for some it had a more definite impact, as in the example below:

“It definitely made me want to go on the course to see what it was all about …If I’d have just turned up to the course on the Monday I might have given it up by Tuesday because, not necessarily understanding what the whole thing is about.” (Completer; Avon, Severn and Thames)

4.1.2. Delivery of the Group Work course

The Group Leaders interviewed commonly described the course as challenging, occasionally emotionally draining, but rewarding and ultimately enjoyable to deliver. All expressed a strong belief in the course design and the theory underpinning it, and compared Group Work positively to other Jobcentre Plus provision delivered previously, particularly the combination of job search training with psychosocial support.

To prepare for delivery each Group Leader attended a seven-week training programme, developed and delivered by the Policy Psychology Division at Department for Work and Pensions (DWP). The vast majority of interviewees considered that this had prepared them sufficiently to deliver the course (the only exception to this being one Group Leader, who felt their trainer was not fully qualified to provide the training). Each Group Leader interviewed had delivered the course multiple times and felt that their skills had developed over time. Provider leads also expressed a high degree of confidence in the capabilities of their Group Leaders. The involvement of DWP policy psychologists in the training and early monitoring of the trial had been welcomed, although some Group Leaders would have appreciated more direct feedback on their delivery of the course.

Group Leaders and provider leads reported that the course was delivered with little divergence from the intended model, as specified in the UK JOBS II manual. The only acknowledged exceptions to this were cases where elements of the content were covered quickly, or occasionally omitted, due to time pressures. Group Leaders felt there was ‘a lot to cram in’ to the time available and that sessions could run over time to allow participants to ask questions. Some of the benefit claimants interviewed also referred to this and questioned why certain elements in the course handouts were not covered in detail in the actual course. Equally benefit claimants felt that the Group Leaders had a difficult job in covering everything and were appreciative of instances where they had gone “off-script” to address a particular issue raised.

Some difficulties were reported with the scheduling of the courses. Although IRM appointments were arranged within one to two weeks of referral, the wait to join the full course could in some cases be as long as three or four weeks. Provider leads indicated this was partly due to lower than expected referral numbers – i.e. in certain weeks these were not sufficient to run a course (the UK JOBS II manual specifies a minimum of 10 participants). There were also one or two reports of courses being cancelled at very short notice due to last minute drop-outs, as the number of participants fell below the minimum specified. Conversely, one provider reported struggling to accommodate the number of referrals after a sudden increase in April 2017, although this had been addressed by the time of interview.

Group Leaders generally agreed with the specified minimum and maximum group size – and agreed that more than 20 would make the group unmanageable. One of the few negative pieces of feedback about the facilitation of the course came from a benefit claimant who had been in a group of 20 participants, which they felt the Group Leaders had struggled to manage and gave the course a kind of ‘classroom’ feel. This contrasted with other benefit claimants’ experiences of smaller group sizes (typically 12-15 attendees). Group Leaders were open to the idea of running courses with slightly fewer than the specified minimum number of participants, particularly if the alternative was cancelling at short notice and risking benefit claimants not attending a later course.

The course was delivered by pairs of Group Leaders in each district – sometimes as a male-female or as a team of the same gender. There was no evidence from the interviews that this influenced participants’ experiences on the course. In either configuration, benefit claimants perceived the Group Leaders to have worked very well as a team. The only reported exception was a case where Group Leaders were replaced due to ill-health part way through a course, which was felt to have impacted negatively on delivery.

Participant perspectives of the course overall

Participant’s reflections of the course overall ranged from the overwhelmingly positive to the mildly appreciative and, exceptionally, to the negative. Group Leaders also perceived that some participants gained more from the course than others, although they felt that ‘everyone can take something from it’ and did not generally perceive it to have a negative effect on anybody. These differing reflections appeared to reflect:

• The needs of different types of participants – participants who were overwhelmingly positive about the course were typically those who were struggling with their job search and had been anxious, worried or low on confidence. They indicated that the course had a positive impact in both these domains:

“It was excellent. For me it kind of did help my self-esteem…for myself, it’s the way you actually saw yourself in the workplace and how, it kind of totally different way of looking for work.” (Completer; Avon, Severn and Thames)

“It was a very good course. You got by the end and you’d actually learned a lot more than you actually thought. It gives you more incentive, more confidence for looking for work.” (Completer; Mercia)

Participants amongst the ‘mildly appreciative’ included those who felt they were not struggling with job search nor felt particularly anxious, worried or lacking in confidence. In effect, they appeared not to need what the course was trying to give them:

“It was fine. I didn’t really learn that much new, but it reconfirmed other things…it was like a refresher course for me.” (Completer; Merseyside)

“It was quite useful, yeah it was OK. It was useful that it told other people what to do, not necessarily me because I’m not shy, but it was just other people, being able to bring their personality out.” (Completer, Durham and Tees)

Others in the mildly appreciative group were participants who saw themselves as facing very significant barriers to work, typically relating to a serious physical or mental health condition. These participants indicated that the course had not been able to fully help them overcome their barriers:

“I still am unemployed but I’m semi-disabled and it didn’t help in that sort of respect.” (Completer; Avon, Severn and Thames)

Other participants also described personal barriers to engaging with the course (relating to literacy and/or English language skills), which contributed to their less positive experiences.

The presence of these differing groups is likely to reflect the widening of the criteria used to recognise potential referrals and pressures in some jobcentres to recognise increased numbers.

• Individual responses to the delivery of the course – with negative reflections on the course typically relating to an aspect of course delivery rather than design, for example how a Group Leaders had spoken to a benefit claimant during a specific session.

These differing reflections and reactions are now explored in more detail in relation to specific elements of the course.

4.1.3. Facilitation of the course

In common with previous studies into the impacts of JOBS II (Vinokur et al., 2000; Vuori et al., 2002), the Group Leaders and how they deliver the course content emerge as a key determinant of its perceived success. The majority of participants interviewed responding most positively to the course cited the Group Leaders as the main reason, while the exceptional negative experiences reported typically related to something an individual Group Leader Group Leaders had said or done. Key aspects underpinning participants’ positive views of the course facilitation included:

• Being treated as an equal - participants frequently contrasted the facilitation of the course with other provision received, with the main difference being that they felt treated as an equal:

“They treat you like adults not like idiots, not like some of the courses do, they ask what you think and how you feel, yeah, it worked.” (Completer; Avon, Severn and Thames)

Some participants and Group Leaders also highlighted the democratising value of course sessions, where participants were invited to provide feedback on something they had done, with self-disclosures by Group Leaders about their own experiences helping overcome any teacher-participant divide and establish empathy:

“They shared quite a bit of their lives with us as well. It made it much more comfortable. They were more, more like human beings, shall we say, than people doing a job.” (Completer; Midland Shires)

• Establishing a participative, safe environment - Group Leaders were widely praised for their ability to encourage participation across the groups, particularly managing group discussions without letting individuals dominate, encouraging quieter individuals to contribute, and asking for and listening to comments:

“They were really helpful, they were friendly, they didn’t like, just get on with it and you know just say like you’ve got to do this, you’ve got to do that. They actually got you involved in the group.” (Completer; Mercia)

“Everyone would say their piece and it would all be written down. It wouldn’t be, yeah, right, well we wouldn’t look for that would we, or that’s a silly thing to say. There was none of that. It was everybody’s opinion was valid. And you could see people actually beginning to take part, after the first hour or so, people were beginning to take part in it, rather than just be told.” (Completer; Midland Shires)

The ‘I pass’ rule, which allowed participants to not participate in some elements of the course, also made the course feel ‘safe’. However, the Group Leaders also played an important role in enabling more nervous or struggling individuals to participate, through the positive reinforcement of their inputs and a willingness to provide additional explanation if needed:

“They were trying to make you feel confident and that so you didn’t have to just sit there silently if you were struggling with something.” (Completer; Durham and Tees)

There were however certain downsides or limits to this participative approach. For example, one participant (not struggling with their job search) found the pace of their course too “slow”, with elements of the content being re-explained to help others to “keep up”. Conversely, some participants with low levels of literacy or spoken English struggled to fully participate, despite the best efforts of Group Leaders and other participants.

• Responding to individual needs - Group Leaders faced the challenge of drawing out shared experiences across the group while also helping participants address their individual barriers or challenges. The comments of participants indicated they were generally able to do this effectively, and while the course content was scripted and intended to be delivered to the group as a whole most benefit claimants felt their Group Leaders had been proactive in providing individual support. This responsiveness helped ameliorate some participants’ concerns about the fixed nature of the course by providing more tailored support:

“They talked to you, they asked how you feel, they were always walking round asking if you need help.” (Completer; Avon, Severn and Thames)

“They were fantastic. If you needed one on one they would, they’d give you one on one.” (Non-Completer; Durham and Tees)

• Role-modelling - part of the theory underpinning the Group Work course relates to the referent power of the Group Leaders, and their ability to motivate or inspire change through personal example. While the participants interviewed did not explicitly describe this happening to them, this is not to say it did not happen – for example benefit claimants all responded positively to self-disclosures by the Group Leaders:

“They didn’t come in and say, ‘Right we’ve got a job’, [instead] they were like, ‘We’ve been all through it. We’ve been there and done that and this is what we’ve learned’.” (Completer; Mercia)

Participants also commonly described Group Leaders displaying behaviours and traits which they evidently admired – including their professionalism, confidence in addressing the group and the ability to work well as a team.

4.1.4. The course format

The format of the course emerged as a less important factor for participants or provider staff than its facilitation or content. Nonetheless three main characteristics were highlighted as having a bearing on benefit claimants’ experiences of the course.

• The balance between facilitator-led and interactive elements – which was felt to have worked well, with participants generally finding the combination of role play, small group and facilitator-led elements central to how the course worked:

“It’s not like going into a class where you’re going to learn one thing and the teacher teaches you. It was a practical session, it was trying to get you to think a certain way, and to look at it in a different way.” (Completer; Mercia)

While some participants described initial concerns over the role play elements this hadn’t been a barrier to their participation, and for some was their favourite part of the course. Group Leaders emphasised the importance of role play in giving participants practice and confidence in the techniques they were taught.

The participants interviewed particularly valued the small group elements of the course, for example when they were split into groups of three to five to complete a task, with less vocal participants reporting feeling more able to contribute in this environment. It was also reported that the small group elements helped remove barriers to participation for certain claimant types:

“Some people did have problems with reading and writing. But when they worked in a team they got their ideas across and that helped them, because they still had ideas and experiences, they just needed a little help basically with putting those ideas together.” (Completer; Mercia)

• The written elements of the course - a small number of the participants interviewed had low levels of literacy and/or English language skills and found this a challenge to their participation in the course. Exceptionally, one reported dropping out of the course as they were unable to follow its written content, while others also reported struggling but had persisted with the course and provided positive reflections on it. In these cases, the participative elements of the course provided a means of partly, if not wholly, mitigating this.
• The timing and duration of the course - the four-hour duration of each day of the course was an important enabler for some participant types to attend and was popular amongst participants in general, as it represented a manageable chunk of time that did not leave them tired or drained. Some drew an explicit parallel between attending the course and of going to work, in part due to the benefit of “getting out of the house”, but also about having a structured routine to their day lacking since becoming unemployed:

“Just getting up, going into Liverpool, doing the Group Work, that in itself puts a certain part of a regime that was, it only took a week to be fair, I know, but it was something that I wasn’t doing before.” (Completer; Merseyside)

Indeed, extending the duration of the course, by a day or an additional week, was suggested by several participants when questioned on areas for improvement. For some this was primarily due to the enjoyment and sense of purpose it provided:

“[It was] not long enough. Because I enjoyed it too much and I wanted to be there longer.” (Completer; Durham and Tees)

While others thought extending the course would provide additional benefits and/or allow parts to be revisited:

“I think we as a group thought the best thing would be to do it for a longer period of time. Not longer each day, but more days. Because there were sections that once you started to feel a bit more hopeful you wanted to go back and do that bit again.” (Completer; Midland Shires)

4.1.5. Group dynamics

The dynamics of the group participating was seen by participants as an important element of the course, with most describing benefits arising from the group environment. While meeting new people and widening social networks were commonly cited by participants, the key benefits included the realisation that there were others in similar situations to themselves, which had enabled them to self-reflect and learn how to address common barriers from others:

“When you’re at home you feel a little isolated and you feel that you’re the only one, and when I realised that there were other people with the same frustrations that I had.” (Completer; Mercia)

“Any issues that you think could be preventing you from getting a job, you brought them forward and people try to find solutions and I think sometimes it takes that to sort of get a group of people together to bounce off each other idea and things like and hear their point of view where you can go get a better resolution to any barriers that you feel are stopping you from getting work.” (Completer; Durham and Tees)

However, the group dynamic did not function positively for everyone, with the key determinants being:

• Wanting to be there - several participants and Group Leaders shared the belief that “you had to want to be there” to benefit from the group dynamic, and that having people who did not want to be there could have a detrimental effect. This was reflected in views expressed by those dropping out of the course. Participants and Group Leaders considered that if individuals did not want to be there, it was best that they did drop out to avoid undermining the functioning of the group. With this in mind, participants and Group Leaders were protective of the voluntary nature of course participation.
• Commonality versus difference within the group - views on this point varied, although there was broad agreement between both Group Leaders and participants that too much commonality or too much difference could negatively influence the group dynamic and functioning, and the ideal would be a balance between the two.

Too much commonality was seen to risk a group turning into “a moaning shop”, within which negative perceptions could be reinforced and limit the scope for participants to benefit from new information, perspectives and strategies:

“If you’ve got the same people with the same thing it tends to be just you’re feeding off the same people. We had different people with different experiences, you know, you learn a bit off that and they learn a bit off you.” (Completer; Mercia)

Too much difference was perceived as inhibiting the development of a group dynamic or risk isolating individuals. Although “not having a job” was a common denominator, some interviewees reported not feeling part of a definable group on the course. This included a small number of participants who felt they were not struggling with their job search (and did not think they faced the same barriers as others), and conversely others who felt they faced different and larger barriers (including some with a significant disability).

Less clear-cut were differences in terms of participants’ age, benefits claimed and their educational or professional background. Exceptionally, some indicated that these had been a dividing line between participants in their group, leading one to suggest that different cohorts of benefit claimants should attend separately, for example, having a Jobseeker’s Allowance (JSA) and an Employment Support Allowance (ESA) group. Another, younger, benefit claimant thought that “if there were people closer to my age it might have been easier to mix with them”, although the majority of participants and all Group Leaders reported that having a mix of these characteristics was beneficial:

“It means you get a good mix of opinions, good mix of ideas and contributions if there is that mix of people. We do find that if there is that mix, that can be quite useful for some people who aren’t so quite academic.” (Group Leader).

This particularly applied to groups with mixed educational ability, in part due to the opportunities created for more highly qualified participants to support those with low levels of literacy.

4.1.6. Course content

Finally, the participants and Group Leaders interviewed highlighted a series of points related to the content of the course.

Perceived usefulness and relevance

Views differed on the usefulness and relevance of the course depending on the characteristics and needs of the individual participant:

• The majority reported finding some or all of the content useful and sufficiently relevant to their circumstances. Although addressing familiar topics (e.g. CVs and job interviews), the course often brought new insights and ideas, for example:

“It was enlightening. There’s things that you don’t think about in terms of presentation, even the way you talk to people on the phone.” (Completer; Avon, Severn and Thames)

Participants also referred to acquiring new tactics or techniques, which provided valuable and relevant learning and could be applied to their situation:

“I thought it was very good, because I learnt more, how to prepare yourself for the interviews. I would have never gone into an interview with a notepad, and write down what sort of questions to ask.” (Completer; Avon, Severn and Thames)

In the observational research participants commonly reported that the sessions on handling the stresses and challenges of finding work had been the most helpful, improving their resilience to setback in their job search.

Others had understood the intent behind much of the content, in terms of changing the way they perceived themselves and their relationship to work:

“I think the psychology part of it worked for all of us. But you’re expecting it to be, kind of, a plug in, this is the psychological bit. And it wasn’t like that. It obviously was progressing through the different days. But it did work on us.” (Completer; Midland Shires)

• The minority of participants who reported finding the course content less useful to them tended to be either:

  • Those who felt they were not struggling with their job search, or who considered themselves very experienced; or
  • Those with a significant disability or other long-term barrier to work, who felt much of the content was not directly relevant to them, or downplayed the challenges disabled people faced, for example:

“I just thought it would be more in depth and better details, real advice not just putting a plaster on things… They were just basically saying that with the right help you can do anything and that, that’s not the case.” (Completer; Durham and Tees)

The language and currency of the content

The evaluation of the 2015 Group Work pilot (Callalan et al., 2015) highlighted that some of the course content was seen by participants as outdated or overly Americanised in its language. In spite of changes to the manual for the trial, the same criticisms were cited by some of the benefit claimants and Group Leaders interviewed:

“Some things were a little bit outdated like the statistics, so they definitely need updating.” (Completer; Durham and Tees)

“A lot of it is very out of date, a lot of it is very American, it’s not tailored to the British market.” (Group Leader)

One additional issue related less to the language used and more to the cultural differences between the US and the UK. Specifically, a small number of participants reported feeling uncomfortable with what they saw as an over-emphasis on presenting themselves to employers in the most positive light possible.

These features were presented as an annoyance or irritant – which could be easily fixed by amending the course script and handouts – rather than an influence on its effectiveness or the benefits resulting for participants.

4.2. Course retention and drop-out

A total of 2,596 benefit claimants started the course, representing 22 per cent of all those offered it and 48 per cent of those accepting the offer and being referred to it. Of the 2,596 starters, over three quarters (1,922 benefit claimants, 74 per cent) went on to complete it. Looking across the participating jobcentre districts, start to completion rates ranged between 69 per cent and 78 per cent, with four of the five being between 76 and 78 per cent. The qualitative fieldwork also identified that, in some cases, participants had been ‘re-enrolled’, where they were unable to complete the course and so were allowed to start the course again from Day 1.

The interviews with benefit claimants completing or dropping out during the course identified a series of factors determining course completion or drop-out, shown in Figure 4.1.

Figure 4.1 Factors determining course retention and drop-out

4.2.1. Reasons for course retention

Interviewees who completed the course provided three main reasons for doing so, namely:

• Enjoyment of the course - course completers commonly described their experience of the course as being “fun”, “a right laugh”, or “enjoyable”. Although easy to dismiss this as irrelevant to the effectiveness and outcomes of the course, participants cited enjoyment as one reason why they had continued with the course to completion - “I looked forward, to be honest with you, I looked forward to going each day.” (Completer; Avon, Severn and Thames). The Group Leaders and group dynamic on the course were typically cited as the main drivers of this sense of enjoyment, with the mental stimulation provided also playing a part.
• Perceived relevance and value - participants talked about continuing to attend because they found the content relevant to their personal situation, and that it would potentially help them move forward, as one described: “What kept me coming back? It was the stuff that I was learning” (Completer; Avon, Severn and Thames). More subtly, some also said that as the course progressed they could feel it having an effect on their mind-set and outlook, which as a reinforcement for them to continue to attend.
• Personal ethos – finally, some participants reported completing the course partly because of a personal ethos and commitment to “finish what I start” and “see things through to the end” - including some who had persevered despite not feeling the course was directly relevant to them, and others who had reacted negatively to some element of it. This group included a small number of cases where completing the course became something of a mission for them, for example, one participant with a disability who found travelling to and attending the course physically gruelling, completed partly to prove to herself that she could.

4.2.2. Reasons for course drop-out

Interviews with those not completing all five days of the course, also provided insights into what had made them leave the course early, including:

• Competing commitments and crises - some participants reported having competing personal commitments which meant they were unable to attend a day of the course, and subsequently dropped out. Commitments cited included attending hospital appointments, emergency repair work needed at home, and having to sit an exam. A small number also reported being taken seriously ill or having a financial crisis, that had to be attended to rather than continuing on the course. More positively, one participant reported being offered a job starting immediately, and left the course to take up this opportunity.

Under the conditions of the trial, if a participant misses a day of the course they are required to start it again if they are still interested in completing it. The participants interviewed were generally aware of this and, in at least one case, had made arrangements to restart the course in the near future.

• Perceived lack of relevance and value - having been willing to give the course a try some participants reached the conclusion – correctly or incorrectly – that it was not going to give them anything they did not already have. These were typically older with professional backgrounds and as one described:

“I only attended the one day to be honest. I’m sure they were good tips and advice but it was all I’d covered in a lifetime so it wasn’t very advantageous to me.” (Non-Completer; Mercia).

Benefit claimants leaving the course for this reason typically did so early on, i.e. during or after the first day.

• Disrupted group dynamics – while the evidence on the influence of this factor is indicative only, Group Leaders and some participants suggested that disrupted group dynamics could be a cause of early exit. While not cited as the main reason for exit by the non-completers interviewed, one older participant leaving the course after Day 2 did describe feeling uncomfortable in a group comprising mainly younger members. Some Group Leaders and participants reported that some individuals had dropped out because of a clash or divide in a group:

“Why do they generally drop out? I think sometimes if there is a potential conflict in the dynamics of the group. If there’s somebody in the group who’s particularly vocal, if there’s somebody in the group who is, has particular views about stuff.” (Group Leaders)

• Written content of course – exceptionally, one participant reported that their limited literacy skills meant they were unable to follow the written content of the course, and so did not complete. Some Group Leaders also cited examples of benefit claimants being referred to the course despite not having the necessary language or literacy skills. In other cases, participants interviewed for this research said they struggled with the written content but had persisted and gone on to complete it - helped in part by support from other benefit claimants on the course.

4.3. Outcomes during course participation

The process evaluation and observational research indicate that participants experienced a number of positive improvements in their mental health, wellbeing and self-efficacy while on the course. Participants talked about, and were observed to experience, improvements in their general mood and positivity as the course progressed. Group Leaders described that by the end of the course some participants were ‘unrecognisable’ in terms of their confidence and sense of self-esteem.

This was reflected in changes in participants’ survey responses between Day 1 and Day 5 of the course. A summary of the measures used in the survey is provided in Chapter 1, with more detail being provided in the Technical Report on the Impacts of the Trial.

4.3.1. Job search related outcomes

Table 4.1 shows the proportion of participants who scored better (increased), the same or lower on their job search activity (measured using the Finnish Institute of Occupational Health (FIOH) Job-seeking Activity Scale) and on their general and job search self-efficacy. The Day 5 scores of participants showed greatest levels of change in job search self-efficacy, with the course apparently having an immediate effect on participants’ levels of confidence in their ability to look for work:

• Four in five (80 per cent) course completers showed improvement in their job search self-efficacy score, indicating that they believed they could undertake the tasks involved in job searching;
• Half of course completers (52 per cent) showed improved levels of general self-efficacy, a measure of optimistic self-beliefs to cope with difficult demands in life, with the general self-efficacy levels of a further quarter (24 per cent) decreasing over the five-day period and a quarter (25 per cent) staying the same; and
• Just under half (48 per cent) of course completers reported doing more job search activity by the end of the course, while 37 per cent showed a decrease in activity (which may relate to time spent attending the course).

Table 4.1: Job search related outcomes for Group Work participants Day 1 to 5

Measure	Base	Improved %	Stayed the same %	Worsened %
Job search activity level (FIOH Job Seeking Activity Scale)	919	48	15	37
General self-efficacy (GSE)	1462	52	25	24
Job search self-efficacy (JSSE)	1446	80	5	15

Source: Baseline and Day 5 surveys

4.3.2. Wellbeing outcomes

Table 4.2 shows the proportion of course completers scoring as doing better, worse or the same at the end of the course on a number of wellbeing measures:

• Around half of course completers showed improved scores on three out of four of the ONS subjective well-being measures: life satisfaction (54 per cent), happiness (51 per cent) and feeling that the things they do in their life were worthwhile (50 per cent). On the fourth ONS wellbeing measure, 42 per cent of course completers reported lower levels of anxiety (i.e. an improvement) while 34 per cent felt their levels of anxiety had increased;
• Using the UCLA loneliness scale, 45 per cent of course completers recorded no change, compared to 32 per cent showing higher levels of loneliness and 23 per cent lower levels; and
• Two thirds (66 per cent) of course completers showed improved levels of psychosocial deprivation using the 12-item Latent and Manifest Benefit (LAMB) scale. Financial strain is the only measure where more course completers indicated that the perception of their situation had worsened (37 per cent) rather than improved (29 per cent).

Table 4.2: General self-efficacy and wellbeing outcomes for Group Work participants Days 1 to 5

Measure	Base	Improved %	Stayed the same %	Worsened %
ONS subjective well-being questions Life satisfaction	1661	54	25	22
Things they do in life are worthwhile	1661	50	24	26
Happy	1658	51	22	27
Anxious	1655	42	24	34
UCLA Loneliness Scale	1401	23	45	32
LAMB psychosocial deprivation scale	1279	66	5	29
LAMB financial strain scale	1593	29	34	37

Source: Baseline and Day 5 surveys

4.3.3. Mental health outcomes

The majority of course completers demonstrated an improvement across the three standardised mental health measures (Table 4.3).

• Two-thirds (66 per cent) of course completers showed higher levels of wellbeing measured using the WHO-5 scale, but the scores of a quarter (25 per cent) of them decreased;
• Nearly two-thirds (63 per cent) of course completers scored lower on the PHQ-9 scale measuring depressive symptoms; and
• A similar pattern was seen for the GAD-7, a measure for generalised anxiety. Nearly three-fifths (57 per cent) of course completers showed an improved GAD-7 score.

Table 4.3: Mental health outcomes for Group Work participants Days 1 to 5

Measure	Base	Improved %	Stayed the same %	Worsened %
WHO-5	1595	66	10	25
PHQ-9	1561	63	16	20
GAD-7	1579	57	20	23

Source: Baseline and Day 5 surveys

4.3.4. Wider health outcomes

The EQ-5D-3L measures self-rated health across five dimensions – mobility, self-care, usual activity, pain / discomfort, anxiety / depression. The vast majority of participants showed no change in their EQ-5D-3L responses (Table 4.4) over the duration of the course. The most improvement by Day 5 was seen with pain/discomfort and anxiety/depression (a 15 per cent improvement for each).

The EQ-VAS is a measure of overall health state. Around two-fifths (43 per cent) of participants reported an improvement in their state of health according to this measure, although one third (33 per cent) reported it had worsened.^{[footnote 25]}

Table 4.4: Wider health outcomes for Group Work participants Days 1 to 5

Measure	Base	Improved %	Stayed the same %	Worsened %
EQ-5D-3L: Mobility problems	1432	5	91	4
EQ-5D-3L: Self-care problems	1439	6	90	4
EQ-5D-3L: Usual activity problems	1436	7	88	5
EQ-5D-3L: Pain or discomfort	1441	15	79	7
EQ-5D-3L: Anxiety or depression	1431	15	80	5
EQ-VAS	1289	43	23	33

Source: Baseline and Day 5 surveys

4.3.5 Differences by participant type

There were also some statistically significant^{[footnote 26]} differences in the reported changes between Day 1 to Day 5 by participant type:

• Anxiety - a higher proportion of participants with high levels of anxiety (with scores of between 15 and 21 on the GAD-7 scale) on Day 1 had improved job search self-efficacy and wellbeing (on the ONS measures), and reduced depression (on the PHQ-9 and the WHO-5 scales), and anxiety (on the GAD-7 scale) on Day 5 than other participants.
• Depression - a higher proportion of participants scoring higher on depressive symptoms (scores of 15 and above on the PHQ-9 scale) on Day 1 had improved job search self-efficacy, wellbeing (on the WHO-5 scale and ONS measures), depression (on the PHQ-9 scale), and anxiety (on the GAD-7 scale) on Day 5 than other participants.

These results indicate that those with high levels of anxiety and those scoring higher on depressive symptoms were the most responsive to the course while they were on it.

4.4. Outcomes shortly after course participation

The interviews with participants, Group Leaders and jobcentre staff conducted for the process evaluation also provided some indicative evidence of outcomes shortly after course participation. It is important to note that the interviews on which this section are based are only able to provide a partial, and relatively short-term, perspective on the outcomes of the course. Participants were interviewed within two months after they had participated in or completed the course, and jobcentre staff had been exposed to benefit claimants who had attended the course to different degrees at the time of the fieldwork. While the Group Leaders had more intense exposure to the participant group during their time on the course, the absence of a formal channel or processes for continued contact after the course meant they were only able to comment on outcomes achieved over this period.

Nevertheless, the interviews provided insights into the perceived immediate impacts of the course, and indicative findings on how these may evolve over time. The quantitative elements of the evaluation – i.e. the six month and twelve-month follow-up surveys of benefit claimants - provide more robust evidence on the impacts of the trial. Please see the Technical Report on the Impacts of the Trial for further details.

4.4.1. Types of outcomes reported

Five broad types of outcomes were most commonly reported in the qualitative fieldwork:

• Confidence;
• Motivation;
• Mental health and wellbeing;
• Job search behaviours; and
• Progress towards and into work.

In practice these outcomes were rarely experienced in isolation, for example confidence and motivation were both important perceived drivers of reported changes in job search behaviours. These outcomes were also not experienced equally by all benefit claimants – most who completed the course reported it having had a positive effect, while some reported no outcomes or, exceptionally, a negative outcome. The following sections explore these interrelationships and variable experiences for each of the outcomes reported.

Confidence

Increased confidence or self-efficacy was the most widely reported outcome of participating in the course by both participants and jobcentre Work Coaches – as these quotes exemplify:

“It builds confidence within yourself and it just helps you, it helps.” (Completer; Avon, Severn and Thames)

“I was more shy, more timid before I went on the course but that’s changed and I feel a bit more confident now.” (Completer; Durham and Tees)

“Some who went with very poor self-esteem have come back with a lot more confidence in themselves and their ability.” (Work Coach; Mercia)

Increases in confidence had resulted either directly as a result of the positive reinforcement and group dynamics of the course, or more indirectly as a product of the additional knowledge, skills and techniques the course provided:

“I mean the interview skills was one thing. CV skills was another. And all those things help build your confidence and your overall mind-set.” (Completer; Avon, Severn and Thames)

Unsurprisingly, benefit claimants who cited confidence issues before attending the course placed particular emphasis on this outcome, while others indicated that the course had more of a boosting or reaffirming effect:

“It gave me a lot of confidence to be able to stand up and be able to present. That was good because my confidence had gone downhill.” (Completer; Midland Shires)

“For me it just made me feel more confident that I know what I’m doing in my job search, that I am on the right path.” (Completer; Merseyside)

Motivation

Increased motivation was also a widely reported outcome of the course by both participants and Work Coaches, for example:

“It’s just lit the fire again where it’s given me the motivation to do something.” (Completer; Midland Shires)

“It’s made me want to step up and get back into work.” (Completer; Durham and Tees)

“Other employability courses or training courses and they’ll come back and you see very little change in their motivation, whereas once they’ve been on this you do see that change in their motivation and their outlook on looking for work.” (Work Coach; Midland Shires)

While some benefit claimants of all ages and circumstances talked about the course having a positive effect on their motivation, it appeared to have greatest significance for older benefit claimants out of work for an extended period, who felt that their prospects were limited and were resigned to never working again. This could include benefit claimants who were quite demotivated and negative about their job prospects before attending the course:

“It just gave me that motivation when I went on the course. Because when you’re out of work and you’ve been out of work for so long your motivation seems to just disappear.” (Completer; Midland Shires)

“We’ve had people who’ve come back who maybe thought they couldn’t get a job before or that anyone wouldn’t be interested in them coming back thinking ‘actually, there might be something I could do’.” (Jobcentre Manager; Mercia)

Although the way in which these benefits were achieved wasn’t always clear, it appeared the participants interviewed had experienced a combination of self-reflection on the barriers they faced, and an increased belief in their ability to overcome them, supported by positive reinforcement from the Group Leaders and other participants:

“All I could focus on initially was why someone wouldn’t give me a job. How can I apply for a job when I’m saying, well I know you won’t give it me? And that’s where my head was when I started. It wasn’t where it was after.” (Completer; Midland Shires)

Participants interviewed not reporting an effect on their motivation indicated they either did not need any additional motivation to return to work, or that they perceived the barriers they faced to returning to work were too great, for example relating to a disability.

Mental health and wellbeing

A minority of the benefit claimants interviewed reported having a mental health condition or issue prior to participating in the course, and views were mixed amongst these benefit claimants on the effects of the course in this area. Some, including those disclosing anxiety and depression, saw a definite benefit:

“I would say I was depressed before I went on the course and then I was a lot happier. So, yes, it did have an overall effect on my mental health and wellbeing, surprisingly. I don’t think I’d be in the same mental state now if I hadn’t have gone on the course. It was quite uplifting.” (Completer; Avon, Severn and Thames)

“Q. Do you think that the course has had any sort of impact on your anxiety or is that something that’s not quite there yet? A. I think it has helped, yeah. Because before I was quite scared of sitting in a group and expressing myself and I was more inward rather than outward.” (Completer; Midland Shires)

One participant drew a direct link between the sense of routine the course had helped establish in their lives and a positive change in their mental state:

“What I mostly find these days, just sorting out a regular rhythm and pattern. Just getting up at the same time every morning, doing stuff at the same time every morning. It’s good for me depression, not staying alone at home.” (Completer; Avon, Severn and Thames)

However, in other cases participants indicated that the severity of their condition meant that the course had not been able to help their mental health or wellbeing:

“Q. Did you feel that the course has helped in any way in terms of your mental health? A. Well not negatively certainly. It didn’t do any harm. Whether there’s anything concretely positive, I’m doubtful…You’d need a whole team of black belt psychoanalysts to sort me out and they’d still give up.” (Completer; Durham and Tees)

Exceptionally, amongst the individuals interviewed as part of the process evaluation, one benefit claimant reported a negative impact on their mental health as a result of participating in the course:

“My mood, at the end of it, was a lot lower than what it was at the beginning of it. My depression got a lot worse. It just pushed it home about how much of a problem things are for me really.” (Completer; Avon, Severn and Thames)

No further explanation on the cause of this was offered although a combination of factors, including the individual’s accompanying dyslexia and an enforced change of Group Leaders during the course, appear to have been contributing factors. A Group Leader also alluded to the potential risks of participants reacting negatively to the course:

“I think we have to manage it very carefully to make sure that it doesn’t make people aware of what they’re not good at and what they can’t do. I think it does have a danger of that, particularly when you, you will occasionally get people who are illiterate, and they’re in a group situation and everybody’s noticing that they’re not reading and writing anything. That could be really shaming for a lot of people.” (Group Leader)

In terms of more general well-being, many benefit claimants talked about their enjoyment of the course and the happiness it had given them while they were on it. Some had also developed friendships and social networks as a direct consequence of the course, which they linked to their increased socialisation and well-being. Work Coaches also highlighted other examples of participants appearing happier and more positive since attending the course:

“I’ve seen a lot of positive changes, a lot of uplifted personalities.” (Work Coach; Merseyside)

“It’s been ‘Oh, it’s made me think differently, it’s made me more positive’, and these are people that are five years into their claim.” (Work Coach, Avon, Severn and Thames)

Job search behaviours

Both participants and Work Coaches reported significant changes on job search behaviour that they considered were as a result of attending the course. One or more of these positive changes were reported by most of the interviewees completing the course, and Work Coaches also indicated they were fairly common amongst benefit claimants on their caseloads. Key changes in job search behaviour were:

• Increased volume and intensity of job search – benefit claimants were reportedly spending more time looking for job opportunities and submitting more applications. This was believed to be due to the enhanced motivation and enthusiasm the course had given them:

“I was quite frustrated and I wasn’t very proactive a few months ago whereas the last few months I’ve been quite proactive in seeking work. I’m looking for an answer after making several applications.” (Completer; Avon, Severn and Thames)

• More reflective and creative job search – here participants described how the course had prompted them to re-evaluate their own barriers to work and think more creatively about the type of work they were looking for:

“It’s made me look down other areas and things what I am capable of.” (Completer; Midland Shires)

“The course has opened my eyes to transferable skills.” (Completer; Avon, Severn and Thames)

Work Coaches also described how participants were now proactively raising barriers with them and asking for their help in addressing them:

“They’re thinking for themselves more of what they need to do as opposed to us trying to drag it out what the next step is going to be, in what direction they’re going to go in.” (Work Coach; Merseyside)

• More informed and diverse job search - several benefit claimants also described applying the new knowledge, skills and techniques learned on the course – with the accompanying increases in motivation and confidence being an important enabler:

“I’ve learned some tactics and referencing skills that I can now put into practice.” (Completer; Avon, Severn and Thames)

“It’s contributed to the ways I look for work and things, just different techniques instead of just using the same techniques that sometimes don’t always work.” (Completer; Durham and Tees)

“The way I do my job search now is totally different to the way I did my job search before I attended the course.” (Completer; Midland Shires)

Some benefit claimants also talked about gaining a wider network through the course that they hoped would be a potential source of job opportunities:

“We’ve got a sort of a bigger network because we were interested in a similar sort of work, well not very similar but similarish type of work so it broadens your contact list.” (Completer; Avon, Severn and Thames)

• More structured job search - some benefit claimants talked specifically about adopting a more structured approach to their job search, building on the sense of routine that attending the course had given them:

“Being on the dole you don’t need to get up til late, but now you got up doing this, it puts you in a routine of OK, this is, could be getting you a job and this is a good step for you, getting up early and thinking I’m going to get a job.” (Completer; Avon, Severn and Thames)

Progress into and towards work

Although indicative only, a handful of the participants interviewed had progressed into work since attending the course, and several others reported waiting to hear the outcomes of job interviews they had recently undertaken. Examples of those finding work included:

• A JSA claimant in her 50s with a professional background who had been unemployed for over six months, having working for the same employer for 18 years before being made redundant: “It knocked me confidence and I was getting more isolated. I was struggling, in the sense of I was getting a bit depressed.” The course was credited as giving her “a lift”, “a kick up the backside”, as well as enabling her to improve her CV. The increased confidence gained from the course influenced her to apply for a job in an area “outside her comfort zone”, and although she was not offered the job she applied for she was re-contacted by the employer following the interview to offer her another post.
• A JSA claimant in her 30s who had not worked since 2015, and who reported lacking confidence in job interviews. She thought that sharing her experiences and frustrations with others on the course, and understanding how to see things from an employer’s perspective, had helped to her to address this:

“If the jobcentre carry on with this course, I think it will benefit a lot of people who haven’t been able to find work, or have been out of work for such a long time, and they’re at that point where they’re at that they can’t do anything about it. And this course is absolutely fab, because it’s helped me.” (Completer; Mercia)

• A woman in her late 40s who had been out of work and claiming JSA for over six months, during which she had applied for several jobs without success and was “getting quite disheartened”. She felt her age, a lack of confidence, and “not knowing where to look” was holding her back. She described how the course had opened her eyes to new ways of looking for work, taught her “how to write about myself” in job applications and raised her confidence. Shortly after the course she secured a job and credited the course with playing a major part in this:

“It helped me to be more confident and I sold myself, I used all the tips and skills that they gave me. As soon as I got an interview I was confident… and I got the job.”

Several other benefit claimants described taking steps following the course to help them move closer to work, including undertaking voluntary work or work experience and enrolling on additional training. The interviewees attributed this to the increased motivation and confidence, and new ideas, resulting from the course, and often the support and facilitation offered by their jobcentre Work Coach:

“I’ve been looking more at getting onto some courses with my Work Coach now to do some actual proper training, getting a little diploma just to put on your CV. Have I actually found work? No. But I’ve had a couple of interviews. I’m getting a lot more phone calls back from employers and agencies where I was getting none before.” (Completer; Durham and Tees)

While recognising that this small qualitative sample is insufficient to support any robust conclusions about the impacts of the course in this area, overall the course completers reported much more activity and progress toward work than interviewees not completing or declining to attend. Work Coaches and provider staff also gave examples of benefit claimants moving into or closer to work following the course:

“A couple of people have gone onto other courses that are a bit more tailored towards a specific job, and then that’s something else that, they’re just more enthusiastic about it and more engaged and get a lot more out of those next steps.” (Work Coach; Durham and Tees)

“The girl that I had last week is going into work experience, because she said that she felt that she wanted to do that. I think that was something I wasn’t expecting, but we’ve got her an appointment this week to get set up with some work experience.” (Work Coach; Mercia)

However, not all those completing the course reported a positive impact on their job search or proximity to work. This again was either because they thought they had “known it all already” or they perceived they still had insurmountable personal barriers to working.

4.4.2. Longevity of outcomes

The timing of the process evaluation research meant it was not possible to explore whether the outcomes reported will be sustained, or otherwise, over time.

When questioned on the longevity of the course outcomes, the benefit claimants reported increased confidence and motivation indicated that they had been able to maintain some or all of the positive momentum they felt on Day 5 of the course:

“Since the course I have had interviews and I feel confident that I will find work soon. It hasn’t faded away in the way that I thought it might because of the skills, because they’re actually giving you skills to do things on the course. You’re like, yes, I learnt this on the course, I’m actually applying this, it’s working.” (Completer; Avon, Severn and Thames)

“Some of the places that I went for interview say I didn’t have enough experience for it, but they said when something lower down came available they’re going to let me know. So, yeah, I’m going fair enough.” (Completer; Midland Shires)

This was partly attributed to being able to apply the skills acquired on the course (and receiving positive indicators of progress as a result), and specifically to the content of the course which addresses how to deal with setbacks:

“I had a bit of a setback because I looked into doing fostering work, but a lot of that is to do with experience. But you pick the notes up, have a look and think, well send myself down another road for now. I’m looking at nannying or something like that a bit different. In the past I’d have just knocked it on the head altogether.” (Completer; Durham and Tees)

“You can have a bad day one day but there’s no point thinking every day’s going to be the same, with the course I went on that got in my head, ok, thinking you can have a bad day, you can have a good day, you can get a job.” (Completer; Midland Shires)

“I was advised about the setbacks and because of that I was able to cope with them and deal with them. Again the course covered that. It was actually very clever to predict how you would feel the following week and I’m able to cope with it.” (Completer; Midland Shires)

For their part, jobcentre staff expressed contrasting views. Some reported observing benefit claimants returning initially energised by the course, which fell away when they failed to make immediate progress towards getting a job, “It seems to be, they’ll come back quite positive but then lose the enthusiasm pretty quickly and go back to the way they were before” (Jobcentre Lead; Durham and Tees). Others painted a more positive picture:

“They’ve applied for jobs and had knock backs but they seem to deal with it a lot better. It seems to be enabling them to have the skills to build on that and get the feedback from that and take it to the next step.” (Work Coach; Merseyside)

It is difficult to conclude as to what may be driving these different perceptions. It is possible they partly reflect the characteristics of benefit claimants that Work Coaches have been referring on to the course and/or the approaches being taken to follow-up with participants after they have been on the course.

4.4.3. Follow-up

Although the Work Coach training emphasised the importance of reflection with benefit claimants completing the course and practiced skills to help them reflect on their experience and the benefits resulting, there was no formalised process for benefit claimants to have further contact with Group Leaders after completion, and jobcentre staff reported variable approaches to following up with benefit claimants on their return. While some jobcentre managers described how their Work Coaches would arrange specific follow up appointments with benefit claimants, others said that any follow up would generally take place as part of the next appointment. In some cases, Work Coaches reported differing practices within their own jobcentre, with some scheduling follow-up appointments and others not.

Benefit claimants attending the course also provided mixed reports of post-course follow-up. As described above, some reported raising ideas to improve their job search and receiving assistance to take these forward from their Work Coach. Others said that while they had discussed their experiences of the course with their Work Coach, the main focus of this discussion had been on providing the Work Coach with a better understanding of what it involved.

There was a shared belief amongst most provider staff, jobcentre staff and benefit claimants that more ‘claimant-focused’ follow-up after the course would be beneficial, to build on the increased confidence and motivation and new skills resulting from it. As one Work Coach described:

“I do feel strongly that unless that follow up is maintained, then what you will have is a situation where people feel buoyed up, people feel encouraged to move forward. Then it’s like training, unless you consolidate your training, then the skills you’ve learnt will be, basically, gradually disappear into the ether.” (Work Coach; Avon, Severn and Thames)

There was, however, no consensus on the optimal format and design of such follow-up. While some jobcentre staff indicated that Work Coaches should, and in some cases already were, performing this role, others cited barriers to this. The barriers cited included constraints on Work Coaches’ time; Work Coaches lacking the knowledge, skills or mind-set to perform this role; and Work Coaches not being able to replicate the participative and group-based elements of the course.

In one jobcentre (Avon, Severn and Thames) an approach had been designed to partially address this. This featured two designated members of staff (both Disability Employment Advisors (DEAs)who would contact benefit claimants attending the course to collect their feedback and to invite them to a fortnightly meeting with other course graduates at the jobcentre. However, at the time of the fieldwork, this approach was still in its early days and attendance at the initial meetings had been modest (i.e. four or five benefit claimants). Interviewees involved in it thought it was too early to determine whether it represented an effective model that could be implemented in other areas of the trial.

Amongst benefit claimants there was a general preference for any follow-up to be undertaken by the course providers, and ideally the same Group Leaders from the course they had attended. Some benefit claimants also favoured simply extending the duration of the course rather than introducing follow-up per se, as they felt that an extended course would offer a greater opportunity to reinforce and practice what they had learned, after which they would feel equipped to “fend for themselves”.

5. Survey findings on participant experiences of the course

This chapter provides findings from the telephone survey of benefit claimants offered the opportunity to attend the Group Work course, conducted around six months from the point of offer. The survey included a small number of questions on participation in the course; its perceived usefulness; views on its design and delivery; and the effects of taking part. In total 744 course participants, 1,066 decliners and 648 control group members responded to the survey.

Overall responses to these questions and (where the data allows) statistically significant^{[footnote 27]} differences between the responses of individuals completing all or some of the course, and those declining the course offer, are reported here.

5.1. Reasons for not participating in the course

Benefit claimants who were offered the opportunity to go on the course but did not attend it (either because they declined or initially agreed but failed to attend) were asked about the reasons for this in the six-month survey.

Only 27 per cent of benefit claimants in this group recalled having been offered the opportunity to go on the course. Amongst those who did recall the offer, the most common reasons provided for not attending the course were associated with poor health (19 per cent), clashes with paid work (17 per cent), or because they ‘didn’t understand what the point was’ (11 per cent), as shown in Table 5.1.

Table 5.1: Reasons for not attending the Group Work course

Reason	% citing reason
Health issues / didn’t feel well enough	19
Clashed with paid work	17
Didn’t understand what the point of it was	11
Too far to travel	5
Just didn’t want to	5
Difficulty arranging childcare	<5
Clashed with another course / interview	<5
Nervous about what we’d be asked to do	<5
Nervous about other participants	<5
Couldn’t afford to travel there	<5
Didn’t understand what was involved	<5
Clashed with voluntary work	<5
Missed it by accident (e.g. forgot, overslept)	<5
Other	30
Don’t know	6
Base = 6 month survey: All recalling invitation to Group Work but did not attend induction or course sessions	97

Benefit claimants who had accepted the opportunity to go on the course, and subsequently attended it, were also asked about how much of the course they had attended. The findings are provided as Table 5.2.

Table 5.2: Reasons for missing sessions on the Group Work course

Reason	% citing reason
Health issues / didn’t feel well enough	39
Didn’t feel like it was doing me any good	17
Clashed with paid work	6
Weather issues	6
Clashed with job interview	<5
Didn’t understand what the point of it was	<5
Difficulty arranging childcare	<5
Didn’t enjoy what were asked to do/found session content stressful	<5
Caring responsibilities	<5
Didn’t like / feel confident in the Group Leader	<5
Clashed with voluntary work	<5
Missed it by accident (e.g. forgot, overslept)	<5
Told not to come back	<5
Other	13
Don’t know	<5
Base = 6 month survey: All who attended some but not all sessions	71

Most (86 per cent) said they had attended all five sessions. As Table 5.2 shows, amongst those who had missed one or more sessions the most common reason given related to health issues/not feeling well enough (39 per cent), and a feeling that the course was not doing them any good (17 per cent).

The number of respondents answering these two questions was too small to allow analysis by sub-group.

5.2. Perceived usefulness of the course^{[footnote 28]}

Overall, a large majority of participants (92 per cent) said they found the course useful. These positive results are consistent across different types of participant, as shown in Tables 5.3 and 5.4. Although there is some variation, there are no statistically significant differences by age, gender, benefits claiming, or time since last worked when they attended the course.

Table 5.3: Perceived usefulness of the Group Work course by participant type

Participant type	% who found course useful
Age: 16-34	89
35-49	95
50+	90
Gender: Male	90
Female	92
Benefits claiming: Jobseeker’s Allowance	93
Other	88
Time since last worked: Less than 6 months	91
6-12 months	89
Over 12 months	91
Base = 6 month survey: All Group Work participants	544

There are no statistically significant differences between participants with different levels of well-being and on some mental health measures, except in terms of depression measured by PHQ-9 and financial strain. Ninety-four per cent of those with high levels of financial strain (measured on the LAMB scale) found the course useful compared to 84 per cent of those with low levels of financial strain^{[footnote 29]}, whereas 87 per cent with moderate to severe depression (measured using PHQ-9) found it useful compared to 93 per cent with no depression.

Table 5.4: Perceived usefulness of the Group Work course by participant characteristics

Participant characteristics	% who found course useful
GAD-7 anxiety: No anxiety	91
Mild to severe anxiety	91
PHQ-9 depression: No depression	93
Moderate to severe depression	87*
WHO5 well-being: Good wellbeing	91
Poor well-being	95
Likely depression	88
UCLA loneliness: Not lonely	91
Lonely	91
LAMB financial strain: Low financial strain	84
Medium financial strain	90
High financial strain	94*
Base = 6-month survey: All Group Work participants *- statistically significant	544

5.3. Views on different elements of the course

5.3.1. Course facilitation

Table 5.5 sets out the findings for participants’ views of their Group Leader(s). Over three-quarters of participants agreed that their Group Leaders really understood the challenges of finding work (79 per cent) and had personal experience of being unemployed (76 per cent). Less than one in ten disagreed with these statements.

Table 5.5: Perceptions of the person or people leading the Group Work course

Statement	% agree	% neither or don’t know	% disagree
They really understood the challenges of finding work	79	13	8
They had personal experience of being unemployed	76	17	7
Base = 6 month survey: All Group Work participants		544

These perceptions were very similar across different participant types and characteristics, although there was one statistically significant difference by benefit receipt. Seventy-seven per cent of participants who were receiving Jobseeker’s Allowance (JSA) at the point they attended the course agreed that the Group Leaders really understood the challenges of finding work compared to 88 per cent of participants receiving other benefits.

5.3.2. The group dynamic

Around two-thirds of participants agreed that working with a group of others on the course, rather than receiving similar support independently, had meant they made more progress (62 per cent), felt more supported (65 per cent), and gave them new ideas (68 per cent). As Table 5.6 shows, less than a third agreed there were negatives to working with a group of other people, in terms of discomfort with sharing ideas and experiences (32 per cent), not getting enough individual help (23 per cent) or time being wasted by other participants (20 per cent).

Table 5.6: Perceived positives and negatives of working with a group of other people on a Group Work course

Statement	% agree	% neither or don’t know	% disagree
Meant you made more progress	62	26	12
Meant that you felt more supported	65	23	12
Gave you new ideas	68	20	12
Made it less comfortable to share ideas or experiences	32	26	42
Meant you didn’t get enough help individually	23	24	53
Meant that time was wasted by other participants (e.g. being late or disruptive)	20	19	61
Base = 6 month survey: All Group Work participants		544

The only statistically significant differences in these results by participant type are by age. A significantly higher proportion of participants aged 16 to 34 (70 per cent) agreed that working with a group of people meant they made more progress than participants aged 35 to 49 (58 per cent) and participants aged 50 or over (60 per cent).

There are statistically significant differences between participants with different levels of anxiety and depression:

• Thirty-nine per cent of participants whose GAD-7 score indicated possible mild to severe anxiety^{[footnote 30]} disagreed with the statement that working with a group ‘made it less comfortable to share ideas or experiences’, compared to 50 per cent of participants with no anxiety;
• Fifty-five per cent of participants whose WHO-5 score indicated likely depression^{[footnote 31]} agreed that working with a group of other people meant they made more progress compared to 66 per cent of participants with unimpaired well-being; and
• Fifty-eight per cent of participants whose WHO-5 score indicated likely depression agreed that working with a group of other people meant they felt more supported compared to 68 per cent of participants with unimpaired well-being.

5.3.3. Flexibility

Over half (56 per cent) of participants agreed that the course had been tailored to their needs and less than a quarter (21 per cent) disagreed.

There was a statistically significant difference in these results by age. Twenty-six per cent of participants aged 50 years or older disagreed with the statement that the course they attended had been tailored to their needs compared to 17 per cent of participants aged 35 to 49.

5.4. The effects of taking part in the course

5.4.1. Job search

Table 5.7 shows that the majority of participants said the course had a positive effect on different aspects of their job search. For example, nearly three-quarters (72 per cent) agreed they had a better understanding of what was needed to find and retain work, and two-thirds (67 per cent) felt better equipped to write applications and CVs.

Table 5.7: Effects of the Group Work course on job search

Statement	% agree	% neither or don’t know	% disagree
Gave you a better understanding of what is needed to find and retain work	72	18	10
Made you feel better equipped to write applications and CVs	67	18	15
Meant you developed new skills relevant to job seeking Z	59	23	18
Made you better at identifying job vacancies that are suitable for you	59	24	17
Helped you move closer to employment	55	25	20
Base = 6 month survey: All Group Work participants		544

The only statistically significant differences in these results by participant type or characteristics are in terms of gender and age:

• Only 15 per cent of female participants disagreed with the statement that the course meant they developed new skills relevant to job seeking compared to 23 per cent of male participants.
• Only five per cent participants aged 35 to 49 disagreed with the statement that the course gave them a better understanding of what is needed to find and retain work compared to 11 per cent of participants aged 16 to 34 and 13 per cent of participants aged 50 years or older.

5.4.2. Motivation and confidence

The majority of participants said the course had improved their belief in their ability to find work, and their motivation and confidence (both in general and specifically in relation to their job search). As Table 5.8 shows, over half (56 per cent) also felt their resilience, i.e. ability to handle setbacks, had also improved.

Table 5.8: Effects of the Group Work course on motivation and confidence

Statement	% improved
Your belief in your ability to find work	72
Your motivation to find work	71
Your confidence in social situations	62
Your communication skills	63
Your ability to handle setbacks in your job search	56
Base = 6 month survey: All Group Work participants	544

Participants aged 50 years or older were consistently the least likely to cite improvements in motivation and confidence in comparison to younger age groups, and these differences are statistically significant (see Table 5.9).

Table 5.9: Effects of the Group Work course on motivation and confidence by age group

Statement	% improved 16 to 34	% improved 35 to 49	% improved 50+
Your belief in your ability to find work	78	73	67
Your motivation to find work	77	72	68
Your confidence in social situations	67	64	58
Your communication skills	72	64	58
Your ability to handle setbacks in your job search	58	61	51
Base = 6 month survey: All Group Work participants	132	165	247

The only statistically significant difference was for individuals with mild to severe levels of anxiety, 60 per cent of whom said the course had increased their ability to handle job search setbacks compared to 48 per cent for participants with no anxiety.

5.4.3. Health

As Table 5.10 shows, around one in five participants said the course had improved their health (19 per cent) and nearly a quarter (23 per cent) said it had improved their ability to manage their health condition(s).

Table 5.10: Effects of the Group Work course on health

Statement	% improved
Your health	19
Your ability to manage your health condition(s)	23
Base = 6 month survey: All Group Work participants	544

Younger participants were the most likely to say that the course had improved both their health and their ability to manage their health condition(s), and these differences are statistically significant:

• Thirty-nine per cent of participants aged 16 to 34 said the course improved their health compared to 18 per cent of participants aged 35 to 49 and 13 per cent of participants aged 50 years; and
• Thirty-three per cent of participants aged 16 to 34 said the course improved their ability to manage their health condition(s) compared to 18 per cent of participants aged 50 years or older.

Participants who were receiving benefits other than JSA at the point they attended the course were also significantly more likely to say that it had improved their health than participants receiving JSA (28 per cent compared to 16 per cent).

5.5. Concluding comments

The chapter provided findings from the telephone survey of benefit claimants who were offered the opportunity to take up the course. This survey was carried out six months after the offer and explored their experiences of the course and its perceived benefit to them.

The reasons for not attending the course were identified as being poor health, clash with a paid work opportunity and because they did not see how the course would benefit them. While the majority of those attending did so for all five days, those not completing cited health issues and a view that the course was not helping them as reasons for leaving early.

The large majority of participants found the course useful, with key elements in their overall experience being:

• The quality of the course facilitator – and their ability to understand, based on personal experience, the challenges of finding work;
• The group dynamic – which helped people progress, feel supported and devise new ideas; and
• The tailoring of the course to individual needs.

Participants in the survey also reported a series of perceived benefits from their participation, including strengthening different aspects of their job search; improving their motivation, confidence and resilience; and positive health effects (including improvements in personal health and the management of existing health conditions).

6. Participant observation of Group Work: exploring the active elements of the intervention

6.1. Introduction

This chapter draws on findings from field observations of the design, implementation and evaluation of the Group Work intervention undertaken by Dr Adam Coutts of the University of Cambridge. Observation of the intervention across the five sites took place between April 2017 and April 2018. Follow up semi-structured interviews and observations of the post programme experiences of participants were conducted until April 2019. The chapter builds upon and provides further insights into the results of the process evaluation reported in the previous chapters. The methodology followed is detailed in Chapter 1.

The evaluation represents the first study to utilise participant observation methodologies, to examine the health impacts and effects of an active labour market policy in the United Kingdom. The embedded nature of the research provided a first-hand account that gave detailed insights into the policy and trialling process, as well as how participants and Group Leaders experienced the intervention. This was particularly useful in understanding how they were affected by the active elements of the intervention, and why this interaction may have led to changes in health, wellbeing and job search behaviours. Specifically, this chapter examines:

• How the Group Work course operates on-the-ground;
• How health and wellbeing change as a result of participation;
• What are the active intervention elements which generate perceived or self-reported changes in mental health and wellbeing of participants?

Participant observation methodologies have been increasingly utilised in Randomised Control Trials (RCTs) by social scientists and development agencies such as the World Bank to tease out what is going on within interventions (Rao et al., 2017). Various authors (Deaton and Cartwright, 2016) have outlined a number of limitations of RCTs-based entirely on quantitative empirical research, and while quantitative impact evaluations may provide estimates of intervention effects, they do not say much about how those effects and impacts were generated, why behaviour changed and what actually are the processes and active elements which lead to changes.

Live social policy interventions are highly complex in their design, implementation and impacts, and are liable to be affected by other ongoing trials and pre-existing policies.^{[footnote 32]} Individuals who participate in RCTs also come with their own set of unique characteristics and socio-economic backgrounds, which may affect how they experience the intervention and mediate the processes used and consequently whether impacts are generated. Added to this are the largely unobserved contextual factors which are outside the control of the intervention and trial such as macro-economic conditions and the local labour market such as whether there are jobs available.^{[footnote 33]} Understanding and acknowledging these influences on policies and evaluation is sometimes referred to as social equipoise – a recognition of uncertainty of what is happening within an intervention or policy.^{[footnote 34]} Participant observation methodologies offer an opportunity to explore this complexity, and its unobserved factors by providing an insider’s account of how an intervention operates and the transition which participants experience.

6.2. Exploring the outcomes and active elements of Group Work

The following sections describe the perceived and reported impacts and outcomes of the intervention on participants between Days 1 and 5 of the Group Work course. It details the types of changes that occurred due to participation within the course, and the active elements and intervention processes which may be responsible for the reported changes in mental health and wellbeing.

6.2.1. Outcomes observed from participation between Day 1 and Day 5

Self-reported outcomes

The main observation was that between Day 1 and Day 5 of Group Work the mental health and wellbeing for the majority of participants across sites appeared to improve. Self-reports from participants and observations by Group Leaders showed that overall health status, self-confidence, self-esteem, and sense of control all appeared to increase. In addition, reduced feelings of anxiety were commonly reported in all trial sites and, in a number of cases reduced symptoms of depression, with a subsequent self-reported decrease in the use of medication. A participant on Day Two of the course identified some of the sources of this positive change:

“It’s like magic this course. From Day 1 I felt like I was a normal person again. Not stuck at home with the kids and my mind going numb. You get a sort of ‘emotional shock’ from being here. It’s a good shock ‘cause now am active and I can see some direction.” (Participant; Midland Shires)

Group Leaders often spoke about a ‘wellbeing shock’ when participants first entered the intervention, with various positive outcomes being reported by the end of Day 1. Group Leaders across sites observed that the most noticeable changes occurred with those who entered the course with underlying health and anxiety issues and that the course ‘worked best’ for them. All Group Leaders that were interviewed and those who took part in the one-day workshop stated that by the end of the course participants would be ‘unrecognisable’ in terms of the newfound confidence and sense of self-esteem that had developed through their participation. As many participants themselves confirmed in enthusiastic terms:

“I’m telling you I am a completely different person from when I came on the course. You saw how I was – I was shaking and embarrassed when they made me get up n’ say who I was and write on the board [on Day1]. Now I feel a little bit more in control and less ‘frozen’ if you know what I mean? Being on the benefits you don’t know which way to turn and end up going nowhere. You’re frozen and stuck in a rut!” (Participant; Merseyside)

“It’s the best course the job centre have ever put me on! I felt it after the first day. Like a warm glow that you’ve achieved something and you’re not kicking yourself ‘cause you spent the day doing nothing. It’s the mental bit that’s really done it for me. I know we’re supposed to be doing job search and networking but just being here has made me feel better about myself.” (Participant; Durham and Tees)

In all districts the participants reported positive psychosocial^{[footnote 35]} benefits. These were described as feelings of social support from other participants and general support from Group Leaders. These improvements were particularly noticeable as participants throughout Group Work sessions often stated that the dominant feature of being out-of-work was a feeling of social isolation and loneliness, which had a negative impact, they believed on their mental health and wellbeing. Participants characterised the worst aspects of unemployment as a loss of social support:

“Being out of work is a lonely state of affairs. You don’t know which way to turn or who to talk to ‘cause there’s usually no one to talk to. When I lost my job I lost a whole network of people I could rely on. Even if I didn’t get along with them at least I knew they were there.” (Participant; Midland Shires)

“You can get by with not having money to go out but you never get used to not having people to talk to. Unemployment and loneliness are like salt and pepper.” (Participant; Merseyside)

Participants felt that the experience of feeling less socially isolated in Group Work, generated feelings of respect from others, which in turn led to increased confidence and an improved sense of self-identity amongst their families and friends. Participants across districts frequently stated that Group Work would help them ‘get on’ and not just ‘get by’ on welfare benefits. For many, attending Group Work reduced the sense of stigmatisation associated with being out-of-work:

“I didn’t think I would get off the welfare and get out of poverty coming on Group Work…I am still under a lot of stress from debts but I’ve felt less stigmatised by coming here.” (Participant; Merseyside)

The sense of being stigmatised had reinforced feelings of isolation, loss of social identity, and lack of confidence that participation in Group Work counteracted:

“On benefits you’re going down the welfare office every other week…you feel demoralised. You don’t have an identity as a person and you know what people are thinking…Lazy so n’so. I felt alone and stuck in the system. After a few days here [on Group Work] I felt more confident. I hadn’t had that feeling for ages…I could speak to people…and the instructor [Group Leader] treated me like I was someone. Now am a more hopeful about the future.” (Participant, Mercia)

Comparing Group Work to interactions with Jobcentre Plus (JCP) and previous employability provision

Participants and Group Leaders in each district often compared their interactions with Jobcentre Plus and attendance on previous employability provision to their experience of Group Work. In comparison to Group Work, existing employability courses were commonly perceived as relying on ‘less than robust’, inadequate, ‘tired’ or ‘out-of-date’ approaches which, as noted in this chapter, resembled a traditional school classroom-based approach. As one Group Leader with extensive experience in delivering employability provision explained:

“The majority of traditional Welfare-to-Work courses which typically are one week or two week courses delivered by a private company covering things like – Job Search, updating your CV, interview skills etc…are ‘perceived’ as a pressure punishment for those that seem to be unwilling to or avoiding getting a job – when it could be the case that some are struggling to focus because of the very issues that we are discussing…like mental health and feeling bad all the time…The usual courses we have delivered are poorly written and poorly delivered by staff whose focus is on a single important task; to ensure that every customer completes their course folder. Courses are often delivered in hired venues with little to no supporting equipment or staff, and all at the lowest possible cost. The customers know this – walk in to any training room and you will see the companies’ attitude towards their customers. Often some of the customers will ask openly; ‘how much are you getting for us on this course?’ That says it all!” (Group Leader)

Many participants across sites often spoke about how the school system had failed them and been unable to support them with learning difficulties and other personal issues they faced such as mental health. Participants and Group Leaders described how Group Work was embedded within a more ‘psychological, holistic and empathetic approach’ to helping job seekers and provided ‘safe space’ in which participants could disclose their worries and challenges. One Group Leader acclaimed this emphasis on psychology as the key factor in Group Work’s success:

“The key difference between what we normally deliver and Group Work is the ‘thinking part’…the psychological support which is in the course materials and how it’s delivered. The instructors [Group Leaders] are there to support people. That…psychological bit is missing in most welfare to work courses”. (Group Leader)

Participants stated that this support enabled them to be more open than in their relationship with Work Coaches. Participants observed that the ‘safe space’ of Group Work helped them address the wider difficulties they faced, above all with sensitive issues such as addiction and mental health, in addition to the routine problems and challenges of job search behaviours. One participant emphasised this level of personal care in an account of what made Group Work much more effective for them than previous employability provision:

“This course [Group Work] is a lot different from the other stuff I’ve been on…Usually you sit in a classroom for a week or two and they tell you how to do the CV or use the internet. It don’t work. You don’t learn anything and I’ve never found a job by that route…On this one [Group Work] they treat you with respect, give you support with all sorts of stuff and you meet people…I don’t feel so isolated now. I spoke to XXXX [Group Work leader] the other day about my anxiety and he put me in touch with someone in social services. No one has done that for me before.” (Participant, Midland shires)

How and why does mental health and wellbeing change? Regaining personal stability and control through participation in Group Work

The ‘summing up sessions’ held at the end of each day by the Group Leaders highlighted how each participant had been affected and changed during the day and over the course of the week. As previously discussed the dominant response was that their self-esteem and confidence had improved, and that they had become less anxious in social or group situations. In these summing up sessions, and particularly on Day 5 at the end of the course, participants and Group Leaders talked about how they felt more stable and in control as a result of the Group Work sessions:

“It’s about stabilising my life mate. End of. Before I came on this I was panicking about everything – bills, dealing with the job centre, making sure the kids don’t go off the rails. Group Work has given me a purpose and I feel in control.” (Participant; Midland Shires)

As one Group Leader noted, this feeling was widely shared:

“For a lot of them it’s about regaining a sense of control, being able to get some stability back in their lives. A lot of the participants have all sorts going on at home – dealing with debt, sick children, looking after relatives.” (Group Leader)

This newfound sense of stability and control was brought about by three active elements embedded in the course design and content, as well as through the intervention environment, in terms of how the course facilitated access to social support and positive interactions with fellow participants and Group Leaders. These are described in Section 6.3.

6.2.2. Perceived stressors and worries during participation

Although expressed only by a minority of participants in the summing up sessions and in personal conversations, concerns associated with participation in the course were apparent across all sites. Those most frequently cited by participants and Group Leaders were the accompanying experiences of financial strain and conflicting demands on participants’ time.

Conflicting demands on participants’ time and financial strain

In a small number of cases in each district the participants’ interviewed described how they were struggling to combine demands at home, such as childcare and dealing with debts, with attending the course. Childcare costs were covered by the intervention but many participants especially single parents stated understandably how they were worried about trying to manage the new routine of attending Group Work along with caring for their children.

Group Leaders often attributed these worries and the feeling of being overwhelmed to either the participants ‘falling out of a work routine’ or to their never having developed one beforehand due to repeated cycles or ‘churns’ between unemployment and temporary part-time work in low-paying jobs. As Group Leaders explained:

“The problem is a lot of the participants haven’t worked for a while and they have got out of the habit and routine of getting up and going to work. The soft skills…Turning up here is a shock to their system.” (Group Leader) “It’s tough for someone who has been out of work for three years to come on these types of courses. On benefits you know the routine and know when the money is coming in. It’s not much but it’s regular. With something like this they’re being asked to change that mind-set and think about work and create their own routine. It’s hard and it causes a lot of stress.” (Group Leader)

As the impact evaluation demonstrates, Group Work participants at baseline reported high levels of perceived financial strain as measured by the Latent and Manifest Benefits scale (LAMB).^{[footnote 36]} Eighty-six per cent of participants reported medium to high levels of perceived financial strain (See Table 6.6 of the impact evaluation). Indeed, concerns about personal finances and poverty were often talked about across sites. The main concern was about household personal debts. Attendance on Group Work also generated new worries about how to pay for travel in order to get to the course. Participants described how they lacked funds to pay ‘up front’ before reclaiming their expenses, which often generated fears of worry and anxiety and in a number of cases meant they arrived late.

In previous JOBS II studies such as that of Vinokur and Schul (2002), it was found that individuals experiencing financial strain may not effectively engage in job search activities, preventing them from successfully acquiring a job and reducing the efficacy of the intervention. Essentially, the perceived financial strain and associated worry acts as a distraction from what a participant is supposed to be focusing on during the intervention. As the wider evidence base also demonstrates, there is a strong and significant association between financial strain, mental health and wellbeing (Vuori et al., 2015; Klebe and Van Hooft, 2018).

‘What do we do after Group Work?’ Approaching the intervention ‘cliff-edge’

Another potentially negative aspect identified by participants and Group Leaders was the perception that they would be ‘dropped’ after the course finished, given that as described in section 4.3.3 there were no formal follow-up processes on offer to them. Within the group sessions, toward the end of each week it was common to hear participants speak of feeling as though they were approaching a ‘cliff edge’ once they had completed the course and would return to being socially isolated and unemployed. As the following examples indicate, there was a clear worry of being abandoned after the course:

“I worry about what happens next. I felt like we’ve been built up, we’ve met all these people and then what? I really don’t want to go back to being stuck at home.” (Participant; Merseyside)

‘I’ve done a load of these courses and you always get dropped afterwards and end up back at the Job Centre looking for the next one’ (Participant; Durham and Tees)

Group Leaders and Jobcentre Plus staff confirmed that this is a common feature of employability provision, which Group Leaders described as a ‘stepping stone’ and ‘temporary fix’ for most people, and almost as one Group Work Leader commented ‘a holiday from unemployment … an emotional see saw, where you build them up and let them down.’

Such feelings of worry may have important implications for any improvements in self-esteem and motivation created through participants’ positive experiences in the intervention. If participants’ expectations and confidence are raised surrounding their potential post-participation employment, and these are then not met by their ‘actual’ experiences in the labour market, any wellbeing gains made during the course could potentially disappear. Group Leaders noted that raising employment expectations amongst participants must be framed by a sense of realism. This was due in part to the local context as well as the availability of jobs for participants with low education and skill levels and who have been unemployed for a prolonged period of time or have experienced ‘cycles’ of unemployment and low wage work:

“I make it clear to them [participants] on Day 1 that they need to be realistic about their job search and what jobs they apply for. We’re building up their confidence and self-esteem which you clearly see change over the week. Problem is, for a lot [of participants] they’re gonna end up back at Jobcentre Plus in two weeks with the Work Coach on to them to apply for any old job. That can really hurt them.” (Group Leader)

The group setting exposing personal vulnerabilities

Across intervention sites it was apparent that the combination of the self-reflection activities and access to social support and contact from fellow participants could have a potentially negative effect on participants’ levels of wellbeing. Participants and Group Leaders often described how this change in their psychosocial environment, from being at home and unemployed to the intervention environment -made them realise how much they had declined in their living standards and sense of wellbeing. This was described by one Group Leader as once more a form of ‘shock’:

“For some [participants] it’s the first time they’ve been in a group setting since school…for a lot of them that was a negative experience. It can be a bit of shock to them to meet people who they don’t know. If you have been living alone for years, not in work and not seeing anyone it’s difficult to be put in something like this.” (Group Work Leader).

Several participants and Group Leaders noted how this was ‘like being in therapy’, where the individual is asked to confront their situation with job search issues they had experienced. Participants talked about how the group-setting and the experience of meeting new people served to reveal the extent to which they had become isolated. Difficulty in communication was one example of how this prolonged social isolation might raise insecurities and anxieties:

“After I talk to the others (participants) I wonder how long it’s going to be before am ready to do an interview. They all seem well better than me at that…I haven’t done one for years. It’s a bit overwhelming to think about it…having to get all dressed up and talking about myself to strangers.” (Participant; Avon, Severn and Thames).

Despite these worries, most participants acknowledged that the ‘realisation’ and confrontation of these issues necessary for them to ‘get out of their situation’, feel better about themselves, and regain a sense of stability and personal control.

6.3. The active elements of Group Work

The following section explores the active elements observed and identified within the course which may lead to changes in health, wellbeing and job search behaviour among participants. It provides insights into how participants experienced the intervention process and these active elements. Three active elements were identified:

• Active element 1: supporting active participation in a group context;
• Active element 2: replicating the experience of employment;
• Active element 3: the role and quality of the Group Leader.

Each active element is explored in detail below.

6.3.1. Active element 1: supporting active participation in a group context – access to social contact and social support

Day 2 introduces participants to what are called ‘identification of personal barriers to work’, the ‘fear of job search’ and learning by ‘using concrete examples’ of their previous employment, job search and general life experiences. As previously noted Group Leaders and participants reported that during Day 1 and 2 there was a rapid increase, ‘spike’ or ‘wellbeing shock’ in relation to self-reported feelings of confidence, self-esteem and self-worth.

The encouragement of direct participation in each session is a core aspect of the JOBS II methodology of ‘active learning’. Group Leaders outlined how the course content featured constructive tasks or activities, such as mock interviews, role playing and job search networking, that directly facilitate participation and interaction with peers and Group Leaders. In section 6.2.1, participants viewed these sessions as generating feelings of control and a sense of ownership insofar as they felt ‘that they have done something useful’. Without such goal orientated activities, Group Leaders and participants felt that the social contact and support element would not be as strong. As many participants and all Group Leaders commented it would ‘just be like the other courses or provision’, which resemble the aforementioned traditional school classroom approach where participants are treated as students, who passively listen to information and are not directly encouraged to interact with each other, let alone to speak about their feelings. As one Group Leader stated, the participants considered Group Work to offer a more professional and organised approach to employability provision:

“The customers’ feedback at the end of each day really did cement their perception that the course [Group Work] was a professional, serious attempt to help them, and that their opinion really did help to refine theirs’ and others’ future experience of the course. As we have said, it is active participation that is the key to the course.” (Group Leader)

From observing the sessions in action, a collective group change in attitudes and wellbeing was often heard, and visible in participants’ use of language and general demeanour. On Day 1 participants appeared reserved, and in many cases displayed visible signs of anxiety and worry, including being uncomfortable when asked to speak about themselves in front of other participants, which was a new experience for many. In addition, as section 6.2.2 described, a small share of the participants observed found the group work elements could be off-putting and expose personal vulnerabilities. Group Leaders reported that some participants would be so worried about this initial part of the course that they would want to drop out on Day 1. Indeed, a large proportion of participants on Day 1 appeared unwilling and too shy to stand up in front of the group to introduce themselves or write on the flipchart boards. However, these inhibitions were considerably alleviated through sessions which required participants to speak to the group directly and openly detail the challenges they face in job search. Participants observed that this active participation and interaction helped to foster a sense of shared experiences and difficulties, which increased feelings of social support and reduced social isolation:

“I think getting people to say how they feel about their situation really helps. I thought no one would care about hearing me being shy, demotivated and depressed but I was shocked when [a fellow participant] blurted out [to the group] that he’s been through the same thing. I didn’t see that coming.” (Participant; Midland Shires).

“I’d never been asked before about how I felt or what’s stopping me get a job. At school and the other [employability] courses I’ve been on you can hide at the back and say nothing. At least here they make you join in. I feel a lot better for that.” (Participant; Merseyside)

As participants and Group Leaders stated throughout, it was not enough to just have ‘people there’ as they had experienced on other employability courses. In Group Work participants noted how they were ‘…inspired by seeing changes in other people’. As a Group Leader observed: “People like meeting each other especially if you’ve been off work for a while. You don’t talk to many people. Group Work…It says it all!”

Group Leaders were there to add reassurance and guidance when participants began to disclose personal information to the group, and offered empathetic responses to these disclosures, often using their own experiences to provide perspective to the participant’s situation. It was clear, as the quotations from participants show, that they felt the course activities requiring them to talk about themselves and reveal their fears and personal issues had the most significant impact in countering feelings of loneliness and social isolation:

“When you speak to the rest of them [other participants on the course] you realise we’re all in the same boat. We have empathy for each other and can struggle together.” (Participant; Midland Shires)

“On other courses I’ve done no one talks to each other. I didn’t have anything in common with them [other people on the course]. Here [On Group Work] I’ve met people with the same problems I have…like being on benefits for ages, having no money and worrying all the time. I learned a lot from the others here…You get advice and support cause you’ve been through the same stuff…A couple of weeks ago I didn’t have no one to talk to.” (Participant, Merseyside)

Participants reported that meeting people like themselves on the course, i.e. from similar social and economic backgrounds, with similar experiences of the job market and dealing with family or personal health problems, gave them new perspectives on their situation and enabled them to cope better with the challenges they faced.

From observing participants over the five days, it was possible to see how they developed friendships and networks with fellow participants based on shared personal problems or simply having something in common, such as a hobby or watching the same TV programme. Often this rapport would be developed at the start of each session, when the Group Leader asked each participant about what they did the previous evening. This ‘icebreaker’ activity helped participants ease themselves into each day and recognise commonalities between themselves, such as TV soap operas, playing darts or dealing with childcare issues. This sense of a ‘shared experience’, mutual support giving, and working through problems together was frequently reported to be beneficial to developing personal confidence, self-esteem and overall feelings of wellbeing:

“The biggest thing I got from coming to Group Work was getting to know other people who I could talk to about stuff. When you lose your job you don’t just lose the pay. You lose a whole network of friends and people to speak to. Now I can call [another participant] if I am feeling unsure about things.” (Participant; Midland Shires)

“There’s stereotypes about the unemployed and I was one of them – depressed, anxious and lonely. I came here and you all treated me with respect, let me speak and helped me. I feel like it’s a family I never had!” (Participant; Midland Shires)

Across Group Work sites, participants reported that they had begun helping one another with a variety of personal issues such as emotional support, help with job search and the sharing of job advertisements. These networks remained following the programme, with WhatsApp or Facebook groups being set up with fellow participants to share information or ‘chat’ with each other about personal problems.

6.3.2. Active element 2: replicating the time structure and routine of employment

The second significant element through which participants reported changes in their mental health and wellbeing was via the development of a sense of routine and structure to the day. Participants described how by attending the course they had got into or back into (if they had previously worked) a habit of ‘getting up and getting out of the house’ to do something constructive which they perceived helped them to develop a sense of self-worth and dignity. Often participants explained how the requirement to attend the course for four hours per day for five days generated a constructive daily routine, rather than what most stated was a ‘boring’ domestic one of being out-of-work or undertaking full-time childcare responsibilities:

“For me the big benefit of the [Group Work] sessions is that it gets you back into the habit of waking up, going to an office and having to deal with people day-to-day. It’s only four hours a day…it’s a start if you’ve been stuck at home for years with the kids and got out of the habit of working… You lose that feeling when you don’t have a job” (Participant, Merseyside).

“It’s strange ‘cause coming here I feel like I have done a day’s work when I get home. You meet people and you feel like you have achieved something instead of wandering around at home screaming at the kids!” (Participant, Avon, Severn and Thames)

Group Leaders and participants frequently referred to how participation in the intervention seemed to replicate or was like the experience of having a job, or what they perceived having a job must be like if they had never had one. This was above all in terms of structuring a daily routine:

“When I read the course materials you get a sense that whoever designed it was trying to make it like ‘having a job’. You have to get here for 9, talk to people, interact and do tasks…constructive ones. I think that’s the nub of all of it. We are giving people an idea of what it’s like to have a job… It’s the soft skills which so many people don’t have let alone hard skills and qualifications.” (Group Leader)

Participants often described to the group and in personal conversation, how prior to Group Work and while being out of work they would try to adopt a routine or daily structure by extending the time needed to complete simple tasks, such as posting a letter and going to the shops, in order to ‘fill hours in the day’. Overwhelmingly, participants linked the feeling that they were constructively organising their time and developing a routine by attending the course, with positive effects on their mental health and wellbeing. As many interviewees reported, this provided a sense of stability and control over their day-to-day lives, even if it was just for the duration of the week of the course, as described in Section 6.2.1.

It was also apparent that the degree of responsiveness in terms of health and wellbeing outcomes was tempered by how individuals perceived the substantive change or difference in their psychosocial environment by attending the course - i.e. the transition between the ‘unemployment environment’ (being at home, socially isolated and lonely) to the ‘Group Work intervention environment’ (where they were meeting people, developing social support, and were actively involved in completing tasks and developing job search skills). The differences between the two environments were frequently raised by participants in the end of day feedback sessions:

“When you’re depressed, out-of-a-job and stuck at home, your brain is in a fuzz. You can’t see clearly or plan. Being on Group Work has given me something to hold onto…A little bit of hope. I feel better for being here.” (Participant; Merseyside)

“Being here [Group Work] is way better than staying at home watching TV. It’s the first thing I noticed when I finished on the first day. Time flew by. Am doing something and meeting people. When you’re at home, bored and haven’t spoken to anyone all day, time drags on and you get frustrated.” (Participant; Midland Shires)

6.3.3. Active element 3: Group Leader effectiveness and credibility

The most important intervention element identified across the trial sites, and fundamental to how the other active elements operated, was the role and quality of the Group Leader. One Group Leader reflected:

“Group Work content is quite different from the normal stuff we deliver here at XXXX. It’s designed to get people to look at themselves and for others on the course to support this. But the main reason [I think] why it works is the coordinators [Group Leaders]. They were properly selected and trained for this course not like other stuff where they get anyone off the street to run them. I think the participants can tell when you don’t know what you’re doing.” (Group Leader)

The Group Leader tended to act as a catalyst for the other active elements of Group Work and was identified by participants and Group Leaders themselves as key in generating changes in participants’ health, wellbeing and how they experienced the intervention overall. From observations across the trial sites the most effective Group Leaders were those who displayed empathy, relatedness and experience of being unemployed, particularly those with previous episodes of being out-of-work and or dealing with their own mental health issues or those of relatives.

In districts where both Group Leaders displayed these personality characteristics, participants expressed high levels of satisfaction and appeared to be far more actively involved and engaged in the sessions. As participants explained these Group Leaders were more credible and likeable, than those they had experienced on previous employability provision:

“I can tell if I am gonna get something out of a course by whoever is teaching on it. The best instructors have usually had some experience of what you’ve been through like being out of work for year or having the bailiffs breaking down your door!” (Participant; Midland Shires)

The most effective Group Leaders were directly involved and interacted with participants from Day 1, and it was apparent they had put considerable preparation into each session and the course as a whole. This, however, might be contrasted with observations of two courses, where the Group Leaders appeared to be reading the Group Work scripts verbatim and sat in front of the participants as in a traditional classroom-based format. In another case, a Group Leader constantly referred to themselves and their experiences and would not let participants talk about their issues or challenges they had faced. In these districts, participants did not appear to be interested or engaged in what was being said by the Group Leaders or the course as whole, with a number stating on Day 2 that they probably would not come back on Day 3, as they perceived the course to be ‘just like school’ and a ‘waste of time’. However, these were the exceptions rather than the norm across sites.

Effective Group Leaders adopted the role of facilitator and prompted the participants to find answers themselves to the tasks in each session. Specifically, this involved Group Leaders moving around the room to listen into group exercises, and ‘acting out’ the case-study parts of each session rather than simply reading them from the text. An important function of the Group Leader which is encouraged by the JOBS II philosophy is to help people to develop independence by ‘analysing their own priorities and creating practical solutions’. This is intended to provide a sense of control and autonomy so the participants will feel valued and supported. Participants and Group Leaders highlighted the absence of such an approach or mentality in previous employability courses they had either attended or delivered. In each trial site participants would describe how they had rarely been asked or encouraged to engage with other participants in previous employability courses:

“It’s the first time someone’s asked me to stand up at the front of one of these [employability] classes and speak to everyone. The usual situation is you sit there like a lemon for three weeks and sleep through it…You don’t learn anything and there’s no benefit. On this one you have to work with each other and get stuck in.” (Participant; Avon, Severn and Thames)

In addition, these Group Leaders did not criticise or question people about why they did not have a job or had struggled to find one. Participants noted their experiences from attending other employability provision had been very different, with many reporting negative experiences, for example:

“I was on a CV writing course before coming here where the coordinator kept telling us that we were never gonna get a decent job ‘cause we had no proper education qualifications. I thought that was a bit much…You come away feeling worse than when you went in.” (Participant; Durham and Tees)

By adopting a more involved role, providing positive reinforcement and asking participants about issues they faced outside of job search, Group Leaders aimed to improve confidence, motivation and above all provide participants with a sense of support. In many cases Group Leaders were reported to have gone out of their way after sessions had finished to provide additional support, such as helping participants in disputes with Work Coaches or with access to mental health services. This assistance went considerably beyond their official responsibilities as a Group Leader. The value of this support often led to participants returning to visit the Group Leaders once they had completed the course.

All Group Leaders emphasised that in order for individuals to move from unemployment to employment, interventions needed to help participants develop their own self confidence. This was contrasted with being passive recipients of information or advice, which was described as another common feature and challenge of existing employability provision. As one Group Leader illustrated:

“As I said before a lot of the employability courses that we deliver tend to be like traditional classroom type. You stand at the front and read out ‘how to write a CV’, ‘Where to find jobs’. It doesn’t work. It’s passive and people give up listening after a couple of hours. A lot of the people who come on Group Work left school at 14. School didn’t work for them so why make employability courses like school? Group Work is completely different. You’re the catalyst for getting them motivated and activated.” (Group Leader)

Group Work was seen as entirely different as it encouraged participants to ‘get actively involved’ and ‘get stuck in’, as Group Leaders frequently stated, through the learning techniques embodied in the JOBS II philosophy. Participants interacted with each other and completed tasks collaboratively rather than just ‘turning up’ and listening to the Group Leader stood at the front of a room.

6.3.4. Group composition

The combination or setting of active elements appeared to be mitigated by the composition of groups. In common with the process evaluation (section 4.1.5), the most effective groups, in terms of those which appeared to work best and demonstrated improvements in mental health, wellbeing and job search behaviours, tended to consist of a mix of participants. These groups featured those unemployed for longer periods (6 months plus) and those recently unemployed; those with low and high levels of wellbeing and self-reported mental health issues; high and low self-reported social support and an equal gender ratio. It was particularly evident from the observations and reports of the Group Leaders that the more confident, less anxious participants would help those experiencing anxiety and reluctant to participate on Day 1. As one Group leader recalled:

“It was clear to us from delivering the course [Group Work] over the first couple of months that people would change… a lot! They’d come in all shy, anxious and not speaking to anyone… by Day 4 and 5 they would be happy as anything…you couldn’t stop them talking. The confidence of the ‘more able’ participants if you want to call them, rubbed off on them. They gave each other support. Like peer role models…I’ve never seen such a big boost as that” (Group Leader)

In a number of districts, it appeared from observations that particular demographic groups and individuals did not benefit from participation as much as others in terms of changes in wellbeing and job search behaviours. From the observations and feedback from Group Leaders these tended to be mostly men usually aged over 50 who had worked for many years in heavy industrial occupations. They were typically long-term unemployed (for 12 months plus) and had not held continuous employment for more than a few months since being made redundant. Most Group Leaders commented that this demographic make-up was ‘the hardest to crack’ and ‘hardest to reach’ in terms of ‘getting them on courses’, let alone encouraging ‘active participation’ and self-reflection techniques in the Group Work setting. Some participants who had worked in heavy industry felt that their skillset and work experiences did not fit with the course content which they perceived as ‘not for them’. As one participant explained:

“This stuff [Group Work sessions] doesn’t apply to my background. I can’t do social networking and like…I was trained to drill and build not run Google!” (Participant, Merseyside)

Group Leaders also attributed this apparent reluctance to fully engage to people’s negative experiences attending multiple local employability courses with little change in their employment situation and life circumstances. They stated that in certain districts ex-industrial workers had long been the focus of DWP and Jobcentre Plus policy, with some participating in 20 or more employability courses. Both Group Leaders and participants commented that some participants had become ‘immune’ and fatigued by the ‘revolving door’ of unemployment to employability provision to unemployment or what was perceived as ‘bad’ work. Existing courses and interventions were viewed as ‘just another part of being unemployed’, rather than a process or way in which they could develop skills and re-enter the workforce.

6.4. Post Group Work experiences

This final section provides insights from a sample of follow-up interviews with previous Group Work participants. These took place at five time points, at one week and then one, three, six and twelve-months points post-participation. The aim was to capture the common experiences among participants in how they dealt with the transition from Group Work, in terms of their job search, mental health and wellbeing. Overall 125 semi-structured interviews were completed, with a cohort of 25 participants across sites contributing at each point. The composition of the cohort is described in Table 1.5 in Chapter 1.

6.4.1. Post course experiences: week 1 to month 3

Follow up research explored to what extent the individual health and wellbeing benefits from participation in Group Work were sustained over time. Nearly all those interviewed after completing the course (n = 24 out of 25) felt that that these gains were still present at month 3 post-Group Work, despite many describing how they had expected a ‘mental drop’ after leaving the course and that they would return to their situation prior to the course. Participants felt their overall health and wellbeing had improved or had at least not declined to a pre-Group Work state. This was apparent for those who had entered employment (n=12) or voluntary work (n=5), and for those who had not found employment (n=8) by month 3.

In general participants indicated that the skills gained in the sessions on ‘inoculation against setbacks in job search’ had been the most useful post-Group Work, particularly in dealing with challenges in finding employment and in their personal circumstances. Many stated that they had never experienced this type of course content in previous employability programmes. These sessions were praised for providing a set of skills and thought processes for dealing with issues they would previously have found overwhelming and unable to cope with on their own. Above all, they reported no longer feeling a sense of panic when receiving a job rejection letter or having to address personal debts, which had caused them considerable stress prior to Group Work. Respondents specifically referred to being better able to cope and having an enhanced sense of control and stability over their general life situation. Despite nearly all stating that they remained under severe financial strain and ‘in poverty’, former participants openly felt more able to make decisions ‘with a clear head’ and experienced less worry about ‘how X bill would be paid’, or whether they would get into work in the near future. As two former participants observed:

“I did worry that I would feel like I did weeks ago [before coming on the course] but after we finished I felt like I could cope with anything. I didn’t get that sick feeling at night that I used to have…with worrying about how to pay the council tax or kids dinner money. I’m more in control.” (Participant; Midland Shires, voluntary work; 1-month post-participation)

“It’s funny cause I thought I’d be back to square one ‘cause I haven’t found a job yet and I’d be depressed…But I feel mentally healthy and less stressed than I used to. Group Work seemed to have change me a bit.” (Participant; Merseyside, unemployed; 3 months post-participation)

In addition to this sense of stability and control, respondents also drew attention to how they had maintained the social support and networks developed as a result of participating in Group Work. The majority stated they had remained in contact with fellow participants either in person or via social media (i.e. WhatsApp or Facebook groups):

“My kids told me I should get on WhatsApp to stay in touch with people on Group Work…I thought I’d never get the hang of it but it’s really helped to have someone to ask things about even just stupid stuff…I feel embarrassed asking them at the job centre about how to do X and that on the computer when I need to do job search.” (Participant; Merseyside; 3 months post-participation)

It was also clear that for those who reported feeling lonely or socially isolated prior to the course that maintaining contact with fellow participants was vital for maintaining their overall wellbeing:

“I don’t have a job yet…but I still have got the friends I made when we did the Group Work course. They help a lot and I can talk to them whenever I need to. That helps me.” (Participant; Midland Shires; 3 months post-participation)

In two districts Group Leaders, Work Coaches and participants reported that formal efforts by Jobcentre Plus had been taken to establish follow-up group meetings with former Group Work participants. These were set up under the initiative of individual Work Coaches and Disability Employment Advisors (DEAs) within Jobcentre Plus and were not part of the formal Group Work trial design. However, former participants explained how these meetings were a useful support mechanism for accessing information about vacancies, other job search provision and for maintaining contact with people. In fact, those interviewed who entered employment immediately after Group Work had done so via connections made with fellow participants and Group Leaders, who had recommended them to local employers known to have vacancies. Those interviewed felt that a network and ‘stock of social support’ had been developed by Group Work which it ‘would be a shame to lose’, and participants, Work Coaches and Group Leaders believed that these networks could be maintained and developed via support from Jobcentre Plus and local providers.

Interviewees reported finding these informal follow-up sessions less threatening in comparison with the formal work focused meetings they experienced at jobcentres. More broadly, it was felt that routine work focused interviews were unable to help benefit claimants with personal issues, such as mental health or to provide adequate support and information about job search, in contrast to their experiences in Group Work. Several respondents discussed in-depth their interactions with Jobcentre Plus pre and post-Group Work, and how they found particular Work Coaches to be less empathetic concerning the challenges they faced:

“You get hope and the course gave me a lot of it, but after 30 knock backs I gave up. It’s depressing. The Work Coach told me I should leave the area and move to London. How can I do that with two kids in primary school and no money?!” (Participant; Durham and Tees; 4 months post-participation)

“First thing I told [Work Coach] at the [job] centre was how nice it was on the course. She looked at me like I was talking mad…I don’t even think she knew what Group Work was. They [Work Coaches] are there to sign your forms and check you’re doing this and that. It’s not gonna help me get a job is it?” (Participant; Midland Shires; 5 months post-participation)

Participants completing the course and all Group Leaders strongly felt that follow up ‘refresher’ or ‘booster’ sessions for those completing Group Work, were essential for maintaining individuals’ levels of motivation and confidence, as well as for keeping in touch with support networks.

6.4.2. Experiences between months 3 to 12

At month 3 onward it was apparent in conversations with former participants who had not found employment or any form of voluntary work that they were struggling with their situation, particularly in terms of self-reported mental health, wellbeing and especially levels of confidence and motivation. Of the eight who were unemployed between week 1 and month three, six remained unemployed and two had moved into temporary voluntary positions. Four of the 12 who had been in work immediately post Group Work had become unemployed and described how they had lost the sense of optimism gained through attending Group Work.

Respondents who remained unemployed or had recently left work stated that their mental health and wellbeing had deteriorated as a result and reported a gradual decline the longer they remained out-of-work. One Group Leader underlined the pressures and returning sense of malaise that contributed to this:

“It was striking how quickly the effects of the intervention wore off in the face of struggling to look for a job, get that job and keep that job. With It is the return of debt, doubt, depression and for few, despair…They went back to the unemployment doldrums!” (Group Leader).

Between months 3 to 12 former participants who had been unable to find work recounted how the daily routines they had developed as a result of participation within Group Work had ceased, and how applying for jobs had become a ‘hopeless task’. All had submitted multiple applications, with little success or feedback from employers as to why they were unsuccessful:

“I could sorta see the cloud coming…I’d applied for a load of jobs in the past three to four months…must be over 30 and didn’t get one of them. I didn’t even get replies. It’s too hard to take all those…Some of the jobs were for cleaning and I didn’t even get them.” (Participant; Midland Shires; 4 months post-participation)

Respondents also described how they had stopped speaking to or meeting people in general, and how feelings of loneliness and isolation had returned:

“Being alone has come back. I don’t see where am going and there’s no one to talk to about those things…You try to stay in touch with people and go out. But when you’ve no money it’s hard to meet people and they don’t wanna know.” (Participant; Avon, Severn and Thames; 3 months post-participation)

Overall, they expressed disappointment at feeling once more like they had before attending Group Work:

“I got a boost from Group Work. Like my mental health was a lot better after but I ended up sinking back into depression when the voluntary work stopped.” (Participant; Mercia; 4 months post-participation)

In addition to the loss of what participants had previously described as a positive daily routine and structure, respondents also explained how worries about debt and finances had returned, further adding to their feelings of instability. While they had previously stated they were able to cope with these issues, for the majority these worries had become overwhelming, particularly when they experienced ‘financial shocks’ where payments needed to be made immediately.

For respondents remaining employed (the majority in education, health and the hotel and leisure industry) or participating in voluntary work, mental health and wellbeing levels appeared to have been maintained. Those employed from week one to one year post-Group Work expressed how they had completely changed in terms of their mental health and wellbeing, and that their wellbeing had increased to the point that they now felt as they did when they had previously been in work:

“I feel like that when I was out of a job it was like a black hole…A dream even. I don’t remember what I was doing half-the-time. You think day-to-day and don’t plan much. Now I am in full-time work, paying off debts and feel in control of things.” (Participant; Midland Shires; 12 months post-participation)

“I remember you (interviewer) asked me ages ago about what I thought about unemployment and my mental health n’ that…I couldn’t see what I was up to. It’s like a fog has lifted…I’ve got a decent position now at X and I can see why I was so down all the time. I had no structure, no mates and no money. No wonder I got depressed.” (Participant; Mercia; 12 months post-participation)

Respondents experiencing moves between employment and unemployment over the course of the year reported that their mental health and wellbeing had suffered. They described how moving into temporary part-time work, often on zero hours contracts, for large local employers always involved minimum wages, little job security and lasted only between two and three months. Respondents described how these jobs which were predominantly service sector occupations such as stock picking, cleaning and catering were ‘not worth the misery’ and how they perceived they were worse for their mental health and wellbeing than being unemployed and relying on benefits. One respondent described the negative impact on health and wellbeing of zero hours contracts:

“They (employer X) would ring me at 5am saying I have to come in today to work or they’d never call. You can’t live like that and you don’t sleep properly. It drove me mad waiting around for them to call.” (Participant; Mercia; 9 months post-participation)

Respondents specified how they experienced little stability and control in these types of jobs, as they were unable to structure or plan their lives. This was due to work schedules being based on the rapidly altering needs of employers – which often would change during a week and from week to week.

6.5. Summary

The observational findings presented in this chapter complement and add further insights into the qualitative process and quantitative impact assessment. They help to illuminate the nature of the self-reported impacts on mental health and wellbeing and why these changed as a result of participation within Group Work. Across trial sites, participants and Group Work leaders overwhelmingly reported and validated the participant observations that the intervention appeared to have positive effects on mental health and wellbeing, especially in helping to improve a sense of confidence and motivation among participants.

It was apparent from the observational research that the effects of Group Work were tempered by the composition of groups and participants in terms of their background social and economic characteristics, as well as mental and wellbeing challenges people were experiencing. This was supported by Group Leaders comments that those who arrived on the course with high levels of anxiety and low levels of general wellbeing appeared to benefit more than other participants. Group Work leaders reported that a particular demographic group – usually men aged over 50 who had worked in heavy industry - did not gain as much from participation in terms of mental health and wellbeing.

The three active elements which were drawn out of the participant observations, provide in-depth insights into the ‘nuts and bolts’ of how these positive effects were achieved. It was demonstrated how these active elements interact with course sessions and learning materials, in particular those that encouraged sharing personal issues and challenges with fellow participants, thinking about how to use social networks to find jobs, engaging in role playing exercises, undertaking mock interviews with fellow participants and receiving positive reinforcement and support from the Group leaders. The active elements and course materials functioned to improve participants (self-reported) access to social support, thereby reducing what participants reported as feelings of social isolation and loneliness. Added to this was the perception that attending the course gave participants a sense of control, stability and routine compared to what many described as ‘chaotic’ domestic and personal economic situations. All described how this lack of control and ‘constant panic’ led to feelings of anxiety and accentuated existing mental health issues.

A key observation is that the combination of active elements identified could potentially provide the basis and foundation for establishing good quality and more effective employability interventions. Participants and Group Leaders perceived these to be largely absent from existing employability provision, which were described as being designed and delivered in an ad-hoc and simplistic manner, often resembling the traditional and passive classroom-based format of teacher – learner / student. Participants and Group Leaders felt these existing interventions amounted to little more than providing ‘bums on seats’ and as one Group Leader commented “‘tick box courses’, where you get paid for getting someone to turn up each day rather than helping them improve themselves”.

In summary, the active elements identified should be supported by good quality learning materials and sessions, which encourage the direct and active participation of job seekers in an intervention. This suggests that the design and implementation of any employability intervention based on the identified active elements, should not be ‘content blind’ but offer the sorts of constructive and interactive sessions offered through Group Work course materials.

In terms of the evaluation of an RCT and any social policy intervention, the use of participant observation in this research demonstrates how it can add analytical value and provide unique insights. These methods can complement quantitative impact analysis and enable evaluators to understand the complexity of policy interventions, particularly in terms of how those involved themselves understand and explain the processes responsible for impacts and changes in wellbeing and behaviours.

7. Conclusions and Lessons Learned

This chapter presents the conclusions and lessons learned from the process evaluation of the Group Work trial, including the course observation and follow-up research conducted by Dr Adam Coutts.

7.1. Conclusions

7.1.1. The Jobcentre Plus elements of the trial

Jobcentre Plus Work Coaches performed a central role in the trial. They were responsible for recognising benefit claimants who would potentially benefit from Group Work and for introducing and offering potential beneficiaries the opportunity to participate in the course. Over the 15 months of the trial, they identified a large volume of potential beneficiaries (over 16,000), and 45 per cent of those offered the opportunity to go on the course by a Work Coach agreed to participate.^{[footnote 37]} The evidence from the process evaluation and observational fieldwork highlights the extent to which practices evolved during the trial and could vary quite widely between Work Coaches. It also highlights some of the challenges and potential trade-offs associated with trialling a new intervention in what is a complex environment.

The key findings on the Jobcentre Plus elements of the trial are set out below, including findings on the training received by Work Coaches.

Training to support project delivery

Training and development activities for Work Coaches took place in two main waves: prior to the trial start in October to December 2016, and after the trial had commenced from March 2017 onwards. Work Coaches were generally positive about the quality, coverage and appropriateness of the training, and reported particular benefits to additional activities such as briefings by and direct contact with Group Work Group Leaders, and opportunities to observe delivery. However, it was felt that the timing of the pre-implementation training, and the level of detail provided about the course and what differentiates it from other provision, could have been improved. Although the post-implementation training helped to address these issues, some Work Coaches remained unclear on the distinctive elements of the course.

Recognising potential beneficiaries

It was widely acknowledged that the trial got off to a slow start, with low numbers of potential beneficiaries being recognised in its first two months. This was partly linked to the timing of the pre-implementation training but numbers did increase significantly in March 2017, when further post-implementation training was delivered. From then on numbers continued to be higher but fluctuated month by month.

By the end of the trial, a total of 16,193 benefit claimants had been recognised by Work Coaches as potential beneficiaries, against the original target of 26,000. The main factors (in addition to the training) influencing this were:

• Interpretations of the criteria for recognising potential beneficiaries - Work Coaches had often understood from the initial training that the course was aimed at those with, or at risk of developing, mental health issues. They thought a broader interpretation of the criteria had been communicated subsequently, which led to them recognising more potential beneficiaries.
• Drives to increase numbers - Work Coaches across the trial districts also reported that they had increasingly been encouraged to consider more and/or more wide-ranging types of benefit claimants as potential beneficiaries.
• Time constraints – Work Coaches reported that the time required to complete the onscreen survey, in the context of a 15-minute work focussed interview with benefit claimants, was the main barrier to them recognising more potential beneficiaries (although the survey was specific to the trial and would not feature in any wider roll-out of Group Work).
• Benefit claimant feedback – positive feedback from benefit claimants who had been on the course encouraged Work Coaches to recognise more potential beneficiaries.
• Underlying Work Coach attitudes – a minority were said by the jobcentre managers and Group Work leads interviewed to be unenthusiastic about any new initiatives introduced in their jobcentre, not specifically Group Work /JOBS II, and had identified few or no potential beneficiaries.

During the trial the profile of benefit claimants being identified as potential beneficiaries also changed, with those who were more confident about their job prospects increasingly being identified.

Group Work providers thought the majority of benefit claimants identified as potential beneficiaries were suitable for the course. However, at the time the process evaluation was conducted, several perceived that they were seeing increasingly diverse benefit claimants coming onto the course, including some who were not struggling with their job search, confidence or well-being. Some felt this potentially diluted the impact of the trial but equally other Group Leaders thought that a mix of participant characteristics was a positive for the group dynamic on the course. Views on the ideal group composition varied, with a broad agreement among Group Leaders that too much commonality or too much difference was not helpful and that the ideal would be a balance between the two. For instance, it was felt that the most manageable group composition comprised a mix of individuals. Group Leaders and participants themselves described how ‘the more able members of the group’ would support and help those who began the course in a state of anxiety, low confidence and motivation.

Introducing and explaining the course

Evidence from the process evaluation suggests that not all Work Coaches had initially followed the intended process when they introduced the course to benefit claimants. However, six to nine months into the trial, and thanks to further post-implementation training, this process was consistently being followed in most cases.

Overall, 45 per cent of benefit claimants who were offered the opportunity to go on the course initially accepted (although this did not necessarily mean they subsequently attended). A number of factors are likely to have determined take-up but Work Coaches were perceived to have an important influence on this. What appeared to be most effective in terms of encouraging take up was:

• Emphasising the difference of the course - particularly with benefit claimants who had been on previous Jobcentre Plus provision and potentially saw little value in going on something that superficially sounded very similar.
• Using benefit claimant feedback and international evidence – benefit claimants were receptive to being told about the impact of the course on others, either based on evidence from other countries or about others who had attended.
• Tailoring the explanation to the benefit claimant - for example, by picking up on something they had said and using that as a way in for introducing the course.

Benefit claimant decision making

Four main factors influenced benefit claimants’ decisions to attend the course:

• The course would help them find work was the most common reason given;
• Improving their general state of mind (i.e. increased confidence in both finding work and in their day-to-day lives) was important for others;
• Others attended due to the desire for mental stimulation; and
• A few attended as they assumed or expected they were required to do so.

Benefit claimants declining to attend did so for a combination of reasons, including the belief that the course would be of little use to them (most commonly older, long-term jobseekers), discomfort with the idea of the group setting, or for health reasons (such as being too unwell or having regular medical appointments). Concerns over the ability to travel to provider venues were also mentioned by some benefit claimants with a physical or mental health condition. These reasons were also reflected in the responses to the six-month quantitative survey.

7.1.2. Experiences of the Group Work course

The process evaluation and the observational fieldwork found that overall the course was being delivered as intended in the trial districts, although some variations in practice and issues with scheduling were identified. Participants also consistently reported that their experience of Group Work compared positively to their experience of previous Jobcentre Plus and other employability provision.

Experiences of the Initial Reception Meeting (IRM)

The IRM was felt by provider staff to be important in informing benefit claimants of the detail of the course, confirming its voluntary nature, and addressing any concerns in advance. In most districts the IRM took place in small group sessions of one to one-and-a-half-hour duration. The providers considered the IRM was delivered effectively and reported commonly arranging sessions within 10 days of a referral and offering a degree of choice over their timing. The benefit claimants also found the IRM useful, particularly if they were unclear of the course content and coverage, and the majority described a positive effect on their motivation to attend. For others, who felt the course was not for them, the IRM provided the opportunity to exit the course without disrupting the group dynamic.

Experiences of the Group Work course

The full course was delivered by a pair of facilitators of mixed or the same gender, and who received extensive training and support to deliver their roles. Course delivery was reported as following the intended model, although in some cases elements were passed over due to time pressures. Overall benefit claimants praised the ability of the facilitators, felt they worked well as a team and had made efforts to cover questions raised within the time available to them. In the six-month quantitative survey, over three quarters of participants agreed that the Group Leaders understood the challenges of finding work (79 per cent) and had personal experiences of being unemployed (76 per cent).

Group sizes were within the parameters stated, although providers found that group sizes towards the upper limit of 20 could be challenging to manage, with the ideal group size being between 12 and 15 participants according to the Group Leaders. In some cases, the scheduling was problematic, with waits to attend being between three and four weeks due to low referral numbers and the group size requirement. In a few cases, courses were cancelled at short notice when individuals dropped out at the last minute. Overall the providers reported being able to meet demand for courses.

Participants’ experiences varied between the overwhelmingly positive, the mildly appreciative and, less frequently, the negative, based on two groups of factors:

• Their individual needs, the extent to which they were struggling with their job search, and whether they had significant health-related barriers; and
• Their experience of the course facilitation - which the vast majority found positive, citing being treated as an equal, the participative environment fostered, responsiveness to individuals’ needs, and the use of personal example to establish empathy and shared experience. Most benefit claimants considered that the course struck the right balance between the role play, small group work and facilitator-led elements. The four-hour daily duration was helpful in enabling some participants to attend and provided a structure to their day that emulated the experience of the workplace.

The majority of participants found the course content useful and relevant to their circumstances and provided new insights and techniques relevant to their job search efforts. These included facilitating self-reflection amongst participants, preparing them for setbacks in their future job search, and course leaders sharing their experiences with the course groups. However, those claiming not to be struggling with their job search and/or with severe limiting conditions or disabilities were most likely to report the content being of limited relevance. One area for minor improvement raised by benefit claimants and facilitators, regarded the instances of American language usage and currency of information provided in the course materials.

The Group Leaders interviewed commonly described the course as challenging, but enjoyable and rewarding to deliver. They all described a strong belief in the course design and the theory that underpinned it.

Course retention and drop-out

Factors influencing whether benefit claimants completed the course included the extent to which they enjoyed it (a combination of the course facilitators, the group dynamic and the mental stimulation offered), its perceived value and relevance and their personal ethos or determination to complete.

For those dropping out before completion, the main factors appeared to be competing commitments (commonly medical appointments, but also medical and financial crises), a perceived lack of relevance or value, and issues with the dynamics of the group attended. In the latter case, this included being in groups of different ages, clashes or disruption within the group, and a perceived lack of commonality. It is worth noting, however, that for other course completers the differences between the participants, and the ability to help or be helped by others, was an important contributor to them completing.

7.1.3. Perceived outcomes

The vast majority (92 per cent) of respondents to the six-month quantitative survey reported finding the course useful to them, with over two thirds reporting positive effects in terms of their motivation and confidence, and over half on different aspects of their job search. The process evaluation explored the immediate outcomes of the course with participants completing it in the previous two months, and with jobcentre and provider staff. Those who completed the course who were interviewed in the qualitative process evaluation fieldwork reported outcomes across five main areas:

• Improved confidence – the most commonly reported outcome, resulting either from the positive reinforcement and group dynamics fostered by the course or from new skills and techniques acquired.
• Improved motivation – widely reported, particularly amongst older participants out of work for some time, and resulting from a combination of self-reflection on the barriers they faced and an increased belief that they could be overcome.
• Improved mental health and wellbeing – while only a minority of benefit claimants disclosed a clinically diagnosed mental health condition, some reported a definite benefit to their mental health. Most others described benefits in terms of their overall happiness and wellbeing. It was apparent that many benefit claimants had enjoyed the course, and developed new friendships and social networks during it, which had led to increased socialisation for some.
• Enhanced job search behaviour – benefit claimants and Work Coaches reported: an increased volume and intensity of job search activity; participants being more reflective, creative and informed in their job search; and applying more structured approaches. Course completers reported higher levels of job search activity than those who failed to complete, a view supported by the Work Coaches consulted.
• Progress towards or into work – although interviewed shortly after completing the course, a handful had found work, and several others were awaiting the outcome of recent interviews.
• Other employability benefits - including participating in voluntary work and additional training or work placements, which benefit claimants had not considered or were reluctant to undertake previously.

The observational research identified similar benefits for participants during the period of the course itself, with a majority reporting improved mental health and wellbeing alongside improved confidence, motivation and self-esteem. This resulted from the feelings of enhanced stability and control, and the mental and social stimulation, provided by the course.

The minority of participants who reported not benefitting from the course were commonly amongst those closer to the labour market, not perceiving themselves as struggling with their job search, or who considered they faced particular challenges associated with their physical or mental health.

While the process evaluation fieldwork focussed on participants who had recently completed the course, the observational research included follow-up consultations with a small number of participants (a cohort of 25 individuals, interviewed at five time points, totalling 125 interviews over a 12-month period). The findings should be treated as illustrative but offer some potentially useful insights. It was observed that:

• Between week 1 to month 3 post-participation, that most (half of whom were now in work or voluntary positions) felt that mental health and wellbeing gains from the course had been maintained. Overall respondents said that Group Work had given them the coping skills and ability to deal with the stressors of job search such as application rejections. Within the JOBS II approach this is a key feature and is termed ‘inoculation against setbacks’.
• At month 3 to 12, it was apparent that those not finding work, or had lost jobs secured after the course, were struggling with their mental health, wellbeing, confidence and motivation, and had lost their sense of daily routine. Those remaining in work reported that the mental health and wellbeing gains had been maintained.

This suggests that while participants’ expectations and confidence in finding work may be raised during their time on the course, if these are not matched by their experience of the labour market in reality, any wellbeing gains made during the course may disappear.

7.1.4. Post-course follow-up

Although the Work Coach training emphasised the importance of discussing benefit claimants’ experience of the course on their return, the absence of formal follow-up procedures in Jobcentre Plus meant that opportunities for reflection and maintaining the positive momentum generated, were uncommon. Some Work Coaches described having specific follow-up appointments with benefit claimants attending the course, while others did so at their next scheduled appointment. Benefit claimants provided mixed experiences of Work Coach follow-up, with some discussing next steps and receiving support to take these forward, while others reported little consultation on their return. There was broad agreement amongst course participants, and jobcentre and provider staff, that a more formalised approach to follow up was required to help maintain the momentum developed.

7.1.5. The active elements of Group Work

The process evaluation and observational research provided insights into the active elements which appear to lead to changes in participant wellbeing and job search behaviour. Three active elements were identified, as described below.

Active element 1: supporting active participation in a group context

The combination of active participation and the group dynamic established were important, with the balance between Group Leader led and the more interactive elements being considered to have worked well. Where positive, the group dynamic could lead to benefits including realising that others were in the same position as themselves, the fostering of self-reflection, learning from others on how to address shared barriers, and establishing new friendships and social networks.

The course observations showed how the ‘group effect’ can lead to rapid increases in reported feelings of confidence, self-esteem and self-worth, and reduced loneliness due to the social contact and support afforded by the group. The observational fieldwork and feedback from Group Leaders suggests that the most manageable group composition, comprised a mix of individuals: Group Leaders and participants themselves described how ‘the more able members of the group’ would support and help those who began the course in a state of anxiety and demotivated. In addition, Group Leaders reported that attendance being on a voluntary basis and participants ‘wanting to be there’ was important to individual engagement and group cohesion.

Active element 2: replicating the time structure and routine of employment

The development of a routine and structure to the day was also a key element, alongside providing participants with constructive activities and a change to what were often described as monotonous daily routines they experienced while being out of work. Group Leaders and participants in the observational research referred to the intervention replicating the experience of being in a work environment, which also helped participants get back into or develop a more positively focused routine.

Active element 3: Group Work Group Leaders quality and credibility

The role and quality of the Group Leaders was a key element, and they acted as a catalyst for the other active elements of the course. Group Leaders ‘ influence as role models, positive reinforcement and their ability to display behaviours and traits admired by participants, was evident through their ability to motivate and inspire change by example. Those able to recount recent experiences of unemployment and dealing with mental health issues themselves or within their families, were most able to empathise and appear credible to the course participants. The vast majority of participants interviewed in the process evaluation fieldwork, and as part of the observational research, expressed high levels of satisfaction with their Group Leaders.

7.1.6. The overall fidelity of the trial

The research also explored the fidelity of the trial from an evaluation perspective. While there was evidence that Work Coaches had followed increasingly broad interpretations of who would benefit from the course, and the referral of benefit claimants not struggling with their job search raised questions around value for money and effectiveness, overall the implementation of the trial was robust – the random allocation process worked effectively and the broad referral criteria were followed.

Findings from interviews and a focus group with Group Leaders, as part of the observation work, suggested that the course may be an appropriate intervention for individuals who have recently lost their jobs. Within the focus group discussions there was a consensus among Group Leaders, that Group Work could act as a means of supporting and enhancing the potential of an individual’s speedier re-entry to work. They stated that this could help people avoid slipping into negative routines and declines in mental health and wellbeing, which may be experienced the longer they remain out-of-work. However, where benefit claimants are referred early this should be on the basis that they are, or are likely to, face challenges finding work.

The research also provided learning for future trials in similar policy areas, including the importance of:

• Ensuring any introductory activity and training provided for Work Coaches and others involved in the trial is comprehensive and delivered well in advance of the trial start;
• Considering the timing of the trial start – to avoid holiday periods (just after Christmas) or times of the year when pressures on Work Coaches may be greatest;
• Responding appropriately to lower than expected numbers of recruits – to ensure the numbers recognised to participate increase while maintaining fidelity with the trial participation criteria;
• Monitoring recruitment on a weekly basis and responding if necessary - for example, monitoring the characteristics of those recruited as well as the numbers, to ensure the correct benefit claimants are being targeted, and responding if this is not the case; and
• Examining whether the Zelen model should be applied in future – while this approach helped limit Work Coach concerns over the randomisation process (i.e. removing the need to inform benefit claimants of the opportunity to attend the course only for them to be subsequently randomised out), this must be set against the implications for evaluation (i.e. ensuring that sufficient numbers of benefit claimants agree to participate in the intervention to allow analysis).
• Considering the inclusion of mental health and well-being measures as indicators of policy and intervention effectiveness in addition to the routine indicators such as rates of job entry and time off benefits. Their inclusion may help demonstrate the broader and more holistic impacts of an intervention and policy effectiveness.

7.2. Lessons Learned from the Group Work Trial

7.2.1. Work Coach and Group Leader training

The study confirmed the importance of the training and development activities for both Work Coaches and Group Leaders, key lessons from the trial include:

• That the timing of any training should be aligned with the start of delivery; training materials should describe the course’s unique features; and that Group Leaders should make presentations to jobcentre staff (who ideally attend at least one day of the course).
• Work Coach training materials particularly should include examples of how trial participants have benefited. For example, case examples or vignettes, to supplement existing evidence.
• That ‘update’ and ‘refresher’ training is provided on an ongoing basis if Group Work was delivered in the future, to ensure Group Work becomes embedded into mainstream provision.

7.2.2. Participant recognition, response and hand-over process

The findings from the study demonstrated that the participant recognition, response and hand-over process played an important role in ensuring effective course delivery and identified the following considerations for any future delivery of Group Work:

• Participation in the course should continue to be voluntary – as wanting to be on the course and actively choosing to do so appeared to be important.
• Ensure that benefit claimants receive clear messages about the content, coverage and benefits for them of attending the course, with key areas of emphasis including:
• Providing international and local evidence of how attending the course can help benefit claimants find work – including case examples drawn from the trial;
• Where appropriate, informing benefit claimants who face specific challenges how the course will help them – for example if the individual is lacking in confidence.
• Explaining that the course is different from existing skills or job search provision – and explaining what the differences are; and
• Informing that those attending the course will be facing similar issues – carefully introducing the group element for those uncomfortable in a group setting.

Importantly, these messages should be tailored to the individual benefit claimant’s characteristics and needs and agreed with the Work Coach. When communicating with staff involved in the referral process, emphasise the importance of recognising participants who are struggling with job search and able to benefit from participation. New benefit claimants could still be considered where job search difficulties are recognised or described by the individual (this could be where they are new to job search and think job search will impact on their wellbeing).

• Explore whether ongoing medical appointments can be negotiated to allow benefit claimants with long-term health conditions to participate.
• Where provider venues are some distance from or difficult to reach by public transport, explore the feasibility of using venues closer to or within the specific jobcentre catchment areas.
• If there is a future trial of Group Work, allow sufficient time for any baseline survey to be completed, for example extending the 15 minutes allocated to work focussed interviews, offering extended appointments or allocating the completion of the survey to other staff members.

7.2.3. Group Work course delivery

The study found that the Group Work course was delivered broadly in accordance with the UK JOBS II manual. Key learnings from the trial that could be applied in the future include:

• Consider allowing course sizes to drop below the minimum – to avoid cancelling courses at short notice which may deter benefit claimants attending in future – although as this would have implications in terms of unit costs it should be considered in exceptional case only.
• Revising the course materials and presentations to bring the figures cited up to date and reduce the use of Americanisms. In future materials, contemplate including case examples or vignettes of the trial outcomes and impacts for individuals.
• The research provided examples where differences between group members both limited and supported the group dynamic. Therefore, consider whether the composition of the participants would aid the dynamics of the group.
• Consider how the follow-up process can be more reliably and efficiently implemented to ensure that participants are fully debriefed on their return to the jobcentre and the positive momentum is supported, including action planning for job acquisition. In addition, assess the feasibility of a monthly one-day refresher courses for former participants to help sustain momentum towards work and maintain the positive social connections developed.
• Consider if, and how, the Group Leaders could play a further role in further embedding the positive developments resulting from the course.

References

Andersen, S.H. (2008) The short- and long-term effects of government training on subjective well-being. European Sociological Review 24(4): 451–462.

Ayala, L., and Rodríguez, M. (2013) Health-related effects of welfare-to-work policies. Social Science & Medicine 93: 103–112.

Bandura, A. (1986) Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs: Prentice-Hall, Inc.

Berkman L.F., Kawachi, I., and Glymour, M.M. (eds.) (2011) Social Epidemiology. Second edition.

Berkman, L.F., and Syme, S.L. (1979) Social networks, host resistance, and mortality: a nine-year follow-up study of Alameda County residents. American Journal of Epidemiology 109(2): 186–204.

Bonell, C., Fletcher, A., Morton, M., Lorenc, T., and Moore, L. (2012) Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Social Science and Medicine 75(12):2299-306.

Bonin, H., and Rinne, U. (2014) ‘Beautiful Serbia’ - objective and subjective outcomes of active labour market policy in a transition economy. Economics of Transition 22(1): 43–67.

Cacioppo, J.T., Hawkley, L.C., Norman, G.J., and Berntson, G.G. (2011) Social isolation. Annals of the New York Academy of Sciences 1231(1): 17–22.

Callanan, M., Moke, T.M., and Edovald, T. (2015) Evaluation of the Group Work Psychological Wellbeing and Work Feasibility Pilot. DWP Research Report No 905. London.

Card, D., Kluve, J., and Weber, A. (2018) What works? A meta analysis of recent Active Labor Market Program evaluations. Journal of the European Economic Association 16(3): 894–931.

Choi, J.N., Price, R.H. and Vinokur, A.D. (2003) Self-Efficacy Changes in Groups: Effects of Diversity, Leadership, and Group Climate. Journal of Organizational Behavior 24(4).

Coutts, A. (2009) Active Labour Market Programmes (ALMPs) and health: An evidence-base. London. Accessed 20 June 2019).

Coutts, A., Stuckler, D., and Cann. D. (2014) The health and wellbeing effects of active labour market programs. In: Huppert FA and Cooper C (eds), Interventions and Policies to Enhance Wellbeing, London: Wiley, pp. 465–482.

Crost, B. (2016) Can workfare programs offset the negative effect of unemployment on subjective well-being? Economics Letters 140: 42–47.

Deaton, A., and Cartwright, N. (2016). Understanding and misunderstanding randomized control trials, NBER Working Paper No. 22595.

Department for Work and Pensions (2013). The disability and health employment strategy: The discussion so far. London. Cm 8763.

Department for Work and Pensions and the Department of Health (2016) Improving Lives: The Work, Health and Disability Green Paper. London. Cm 9342.

Department for Work and Pensions and the Department of Health (2017) Improving lives: the future of work, health and disability. Policy paper. London. Cm 9526

Department for Work and Pensions (2018) Protecting defined benefit pension schemes. London. Cm 9591.

Dinh, H., Strazdins, L., and Welsh, J. (2017) Hour-glass ceilings: Work-hour thresholds, gendered health inequities. Social Science and Medicine 176: 42–51.

Fryer, D. (1986) Employment deprivation and personal agency during unemployment: A critical discussion of Jahoda’s explanation of the psychological effects of unemployment. Social behaviour 1(1): 3–23.

Helliwell, J.F. (2011) Institutions as enablers of wellbeing: The Singapore prison case study. International Journal of Wellbeing, 1 (2), 255-265.

Hult M., Lappalainen, K., Saaranen, T.K., Räsänen ,K., Vanroelen, C., and Burdorf, A. (2020) Health-improving interventions for obtaining employment in unemployed job seekers. Cochrane Database of Systematic Reviews 2020, Issue 1. Art. No.: CD013152. DOI: 10.1002/14651858.CD013152.pub2. Page 18.

Jahoda, M. (1982) Employment and Unemployment: A Social-Psychological Analysis. Cambridge: Cambridge University Press.

Kamerāde, D., Wang, S., Burchell, B., Balderson, S.U. and Coutts, A. (2019) A shorter working week for everyone: How much paid work is needed for mental health and well-being? Social Science and Medicine: 112–353.

Kim, T.J., and Von dem Knesebeck, O. (2015) Is an insecure job better for health than having no job at all? A systematic review of studies investigating the health-related risks of both job insecurity and unemployment. BMC public health, 15(1), 985.

Klebe, U-C., van Hooft, E. (2018) The Oxford Handbook of Job Loss and Job Search, Oxford University Press.

Kluve, J., Puerto, S., Robalino, D., Romero, J.M., Rother, F., Stoterau, J., Weidenkaff, F., and Witte, M. (2017) Interventions to improve the labour market outcomes of youth: a systematic review of training, entrepreneurship promotion, employment services, and subsidized employment interventions. Campbell Systematic Reviews 12: 1–288.

Knight, T., Lloyd, R., Rayment, M., Purdon, S., Bryson, C., Downing, C., Svanaes, S., Coutts, A., McKay, S., and Mukuria, C. (2021b) Group Work/JOBS II: Evaluation Synthesis Report. London. DWP Research Report No 991. London.

Kovacs, C., Batinic, B., Stiglbauer, B., and Gnambs, T. (2019). Development of a shortened version of the Latent and Manifest Benefits of Work (LAMB) scale. European Journal of Psychological Assessment, 35(5), 685–697.

Llena-Nozal, A., Martin, N., and and Murtin, F. (2019) “The economy of well-being: Creating opportunities for people’s well-being and economic growth”, OECD Statistics Working Papers, No. 2019/02, OECD Publishing, Paris.

Marmot, M., Allen, J., Boyce, T., Goldblatt, P., and Morrison, J. (2020) Health Equality in England: The Marmot Review 10 Years On

McKee-Ryan, M.F., Song, Z, Wanberg, C.R., and Kinicki, A.J (2005) Psychological and Physical Wellbeing During Unemployment: A Meta-Analytic Study, Journal of Applied Psychology. 90, (1), pp. 53–76.

McManus, S., Bebbington, P., Jenkins, R., and Brugha, T. (eds.) (2016) Mental health and wellbeing in England: Adult Psychiatric Morbidity Survey 2014. Leeds: NHS Digital

McManus, S., Meltze, M., Brugha, T., Bebbington, P., and Jenkins, R. (2009) Adult psychiatric morbidity in England (2007): Results of a household survey. Leeds: The NHS Information Centre for Health and Social Care.

McManus, S., Mowlam, A., Dorsett, R., Stansfeld, S., Clark, C., Brown, V., Wollny, I., Rahim, N., Morrell, G., Graham, J., Whalley, R., Lee, L., and Meltzer, H. (2012). Mental health in context: the national study of work-search and wellbeing. London. DWP Research Report 810.

Meehan, M., Birkin, R., Ruby, K., and Moore-Purvis, H. (Eds.) (2015) UK JOBS II: A Manual for Teaching People Successful Job Search Strategies. London: DWP. (The UK edition is a revision of Curran J, Wishart P, Gingrich J and The JOBS Project Staff (1999) JOBS: A Manual for Teaching People Successful Job Search Strategies. Ann Arbor, MI: University of Michigan).

Murphy, G.C., and Athanasou, J.A. (1999) The effect of unemployment on mental health. Journal of Occupational and organizational Psychology, 72(1), 83-99.

Niedzwiedz, C.L., Mitchell, R.J., Shortt, N.K., and Pearce, J.R. (2016) Social protection spending and inequalities in depressive symptoms across Europe. Social Psychiatry and Psychiatric Epidemiology 51(7): 1005–1014.

Organisation for Economic Co-operation and Development (2013) Active Labour Market Programmes. Glossary of Statistical terms. Available at: https://stats.oecd.org/glossary/detail.asp?ID=28

Paul, K., and Moser, K. (2009) Unemployment impairs mental health: Meta-analyses, Journal of Vocational Behaviour 74, 3, pp. 264-282.

Pearlin, L., Menaghan, E., Lieberman, M., and Mullan, J. (1981). The stress process. Journal of Health and Social Behavior, 22, 337-356.

Petticrew, M., McKee, M., Lock, K., Green, J., and Phillips, G. (2013) “In search of social equipoise”. British Medical Journal (Clinical research ed), 347.

Price, R.H., and Vinokur, A.D. (2014) The JOBS Program: Impact on job seeker motivation, reemployment, and mental health. In: Klehe, U-C. and Van Hooft, E. (eds), The Oxford Handbook of Job Loss and Job Search, Oxford: Oxford University Press.

Purdon, S., and Bryson, C., (2021) Group Work/JOBS II: Technical Report on the Impacts of the Trial. DWP Research Report No 988. London.

Rao, V., Ananthpur, K., and Malik, K. (2017). The Anatomy of Failure: An Ethnography of a Randomized Trial to Deepen Democracy in Rural India, World Development, Elsevier, vol. 99(C), pages 481-497

Rayment, M., Knight, T., Lloyd, R., Purdon, S., Bryson, C. and McKay, S. (2021) Group Work/JOBS II: Cost Benefit Analysis Technical Report. DWP Research Report No 990. London.

Rose, D. (2019) The impact of active labour market policies on the well-being of the unemployed. Journal of European Social Policy 29(3): 396–410.

Sage, D. (2014) Do Active Labour Market Policies Promote the Subjective Well-Being of the Unemployed? Evidence from the UK. National Well-Being Programme. Journal of Happiness Studies. August 5th.

Sage, D. (2015a) Do active labour market policies promote the subjective well-being of the unemployed? Evidence from the UK National Well-Being Programme. Journal of Happiness Studies 16(5): 1281–1298.

Sage, D. (2015b) Do active labour market policies promote the well-being, health and social capital of the unemployed? Evidence from the UK. Social Indicators Research 124(2): 319–337.

Stuckler, D., Basu, S., Suhrcke, M., Coutts, A., and McKee, M. (2009) The public health effect of economic crises and alternative policy responses in Europe: An empirical analysis. The Lancet 374(9686): 315–323.

Van Hooft, E.E., Wanberg, C.R., and Van Hoye, G. (2012) Moving beyond job search quantity: Towards a conceptualization and self-regulatory framework of job search quality. Organizational Psychology Review. 3(1). p.3-40.

Van Stolk, C., Hofman, J., Hafner, M., and Janta, B. (2014) Psychological wellbeing and work: Improving service provision and outcomes. Department for Work and Pensions and Department of Health: London, UK.

Vinokur, A. D., Price, R. H., and Schul, Y. (1995). Impact of the JOBS intervention on unemployed workers varying in risk for depression. American Journal of Community Psychology, 23(1), 39–74.

Vinokur, A.D., and Schul, Y. (2002) The web of coping resources and pathways to reemployment following a job loss. Journal of Occupational Health Psychology, 7(1), 68–83.

Vinokur, A.D., Schul, Y., Vuori J., and Price, R. (2000) Two years after a job loss: Long-term impact of the JOBS program on reemployment and mental health. Journal of Occupational Health Psychology 5(1): 32–47.

Vuori, J., Blonk, R., and Price, R.H. (2015) (EDS) Sustainable Working Lives: Managing Work Transitions and Health throughout the Life Course, Springer Netherlands.

Vuori, J., Silvonen, J., Vinokur, A. D. and Price, R.H. (2002) The Tyohon Job Search Program in Finland: Benefits for the Unemployed with Risk of Depression or Discouragement. Journal of Occupational Health Psychology, Vol 7, No. 1, 5-19.

Vuori, J., and Vinokur, A.D. (2005) Job-search preparedness as a mediator of the effects of the Työhön Job Search Intervention on re-employment and mental health. Journal of Organizational Behavior 26(3): 275–291.

Wang, S., Coutts, A., Burchell, B., Kamerade, D., and Balderson, U. (Forthcoming) Can Active Labour Market Programmes emulate the mental health benefits of regular paid employment? Longitudinal evidence from the United Kingdom. Work, Employment and Society.

Warr, P. (1999). Well-being and the workplace. In D. Kahneman, E. Diener, & N. Schwarz (Eds.), Well-being: The foundations of hedonic psychology (p. 392–412). Russell Sage Foundation.

Waters, L. and Moore, K. (2002) Reducing latent deprivation during unemployment: The role of meaningful leisure activity. Journal of Occupational and Organizational Psychology, 75, 15–32.

Wood, A.J., and Burchell, B. (2018) Unemployment and well-being. In: Lewis A (ed.), The Cambridge Handbook of Psychology and Economic Behaviour, Cambridge: Cambridge University Press, pp. 234–259.

Wulfgramm, M. (2014) Life satisfaction effects of unemployment in Europe: The moderating influence of labour market policy. Journal of European Social Policy 24(3): 258–272.

Young, C. (2012) Losing a job: The nonpecuniary cost of unemployment in the United States. Social Forces 91(2): 609–634.

Zabkiewicz, D., and Schmidt, L. (2007) Behavioural health problems as barriers to work: Results from a 6-year panel study of welfare recipients. The Journal of Behavioural Health Services and Research, 34(2), 168–185.

Appendices

Appendix A: Topic guides used in interviews for the process evaluation

A1: Provider (lead) topic guide

Guidance

This topic guide provides the key themes and sub-themes to be explored in interviews with Group Work leads. It is not a set script. The phrasing, pacing and ordering of questions should be tailored to reflect the individual respondent and the flow of the discussion in each interview.

The aim of the interview is to collect detailed, tangible insights into the respondent’s understanding and experiences of the management and delivery of the Group Work course, and their perceptions of its impact on claimants who go on it. This will feed directly into our assessment of how effectively the intervention has been designed and delivered.

Topic guide sections:

1. Introduction

Explaining the purpose of the interview, how the findings will be used, its confidentiality, and gaining their informed consent.

2. Respondent and provider background

Their role, involvement in the trial, and details of staff delivering the Group Work course.

3. Awareness and understanding

Their understanding of the aims of GW JOBS II, its key features, and the guidance and training they have received to support the delivery of the trial.

4. Referral Process

Their experiences of each stage in the process and views on its effectiveness.

5. Take up and drop out

Their perceptions of why claimants initially agree to participate and then either go on to drop out or complete.

6. Experiences of GW JOBS II

Their experiences and perceptions of the delivery and effectiveness of the GW JOBS II course.

7. Outcomes

Their perceptions of the effect of GW JOBS II on the attitudes, behaviours and circumstances of claimants in relation to health and employment.

In preparation for each interview ensure you are familiar with: the information provided by DWP about the respondent; and the intended intervention process and GW JOBS II course content. Respondents should not be initially prompted with this information but any significant apparent differences between what they describe and the intended delivery model should be explored.

1. Introduction

• Thank them for giving their initial agreement to be interviewed.

• Explain that:

  – The interview is part of research that ICF is conducting for the Department for Work and Pensions (DWP) and the Department of Health (DH) about GW JOBS II.

  – They have asked us to evaluate the GW JOBS II trial, and as part of that we are interviewing the provider leads in the areas where it is being trialled. The interview is not a test. We just want to hear their honest opinions, and there are no right or wrong answers.

  – ICF is an independent research consultancy and not part of DWP, DH or Jobcentre Plus.

  – Everything they say in the interview will be treated as confidential. This means that when we report the findings we will not say who we interviewed or who said what.

  – We would like to record the interview, just so we have an accurate record of what they say. Only members of the ICF research team will access to the recording. It will not be shared with DWP, DH, Jobcentre Plus staff or anyone else.

  – The interview will last up to 45 minutes.

• Ask them if what you have said is clear, if they have any questions, and then whether they agree to take part in the interview.
• If they agree, start the recorder and say “this is X with X”, state the date, “and can you just confirm that you’re happy to be recorded?’”

2. Respondent and provider background [5 minutes]

• What is your job role?
• How long have you been in this role?
• How long have you been involved in the GW JOBS II trial?
• What are your specific responsibilities in relation to the trial?

• Can you give me an overview of:

  – The number and type of your staff involved in the delivery of the trial. E.g. in terms of how staff were selected for facilitator role, characteristics looked for, any recruitment specifically for the trial.

  – The number and type of claimants your staff are generally delivering the Group Work course to.

3. Awareness and understanding [5 minutes]

• What do you understand the aims of GW JOBS II to be?

• What is your understanding of its underlying logic / rationale? E.g. in terms of:

  – Who it is targeted at?

  – What issues / barriers it is trying to address?

  – How it is trying to do this?

  – How it compares / contrasts with other JC+ provision?

• Overall how well do you think you understand GW JOBS II?

• Thinking more broadly:

  – Do you think that having a job is good for individuals’ health and self-esteem? Why?

  – Conversely, do you think that being out of work is bad for individuals’ health? Why?

• Has your understanding changed at all over time / since the start of the trial? If so, How? Why? In what ways?

• What guidance or training have you received on GW JOBS II?

  – When it was provided/delivered?

  – Who by?

  – What it covered?

  – How useful was it?

• And how well do you think your staff understand GW JOBS II?
• Has their understanding changed at all over time / since the start of the trial? If so, How? Why? In what ways?

• What guidance or training have they received on GW JOBS II?

  – When it was provided/delivered?

  – Who by?

  – What it covered?

  – How useful was it?

• Overall, how effective was the guidance and training received by you and your staff?
• Are there any ways in which you think either the content, timing or delivery of the guidance/training could be improved? With hindsight, was there anything that wasn’t covered?

4. Referral process [5 minutes]

• Can you tell me what you know about how Jobcentre Plus identify claimants for the course?
• What’s your impression of how well this is working?

• How suitable do you think the referrals your organisation is receiving are? E.g. in terms of:

  – Are they intended audience for GW JOBS II?

  – Have they taken an informed / voluntary decision to take part? Did any say they were told to attend?

• Roughly how many claimants have been referred to your organisation to take part in GW JOBS II since the start of the trial?

• Can you talk me through what the process is for Jobcentre Plus referring claimants to your organisation? Particularly in terms of:

  – The transfer of claimant details

  – The arrangement of the IRM meeting

  – Any subsequent co-ordination to reimburse claimants’ travel and childcare costs

• How effectively does this process work?
• Have there been any problems?
• Has the process changed at all over time? If so, Why? How?
• Are there any ways in which it can be improved?

5. Take-up and drop out [5 minutes]

• Roughly what proportion of claimants that are referred to you also subsequently attend the IRM?

  – Does this differ at all between different types of claimant? If so, How? Why?

• If a claimant doesn’t attend the IRM they were scheduled come to, what happens? E.g. can they attend a later one instead?

• Why do you think claimants initially agree or disagree to go on the course? – Does this differ at all between different types of claimant? If so, How? Why?

• And as far as you are aware, roughly what proportion go on to:

  – Start the Group Work course but drop out without completing it?

  – Start the Group Work course and complete it?

• Why do you think claimants either continue or drop out at these different points?

  – Does this differ at all between different types of claimant? If so, How? Why?

  – How would you rate levels of awareness, uptake, and effectiveness of help with travel and childcare costs?

• Are there any ways in which the process could be improved to increase retention / decrease attrition at these different points?

6. The Group Work course [15 minutes]

• How long after a referral does the claimant typically attend an IRM?
• What does the IRM involve (e.g. in terms of how it is structure and facilitated, and how many other people are there)?
• In terms of the Group Work course itself, roughly how long after the IRM does the claimant go on it?

• And can you give me a brief overview of how your staff are delivering the course itself? Specifically in terms of:

  – Its content

  – The format and structure

  – The facilitator role

  – The number / composition of people on the course

• Does this vary at all across different facilitators and/or at different times the course has been delivered? If so, How? Why?
• Has this been changed or modified at all during the trial? If so, How? Why?
• Have facilitators adapted or skipped elements of the course at all? If so, How? Why? With what consequences?

• Overall what do you and your staff think of the course? Allow for spontaneous responses then probe:

  – How effective it is

  – How claimants respond to it

  – What features of it work well

  – What features of it work less

  – How easy or difficult they have found it to deliver

  – How effectively you think they have been delivering it

If not already explored above:

• What did you think of the content of the course? E.g. in terms of:

  – What it covers (and balance between “work”/”job search” and “well-being”/”resilience”)

  – Its relevance and usefulness to the claimants

  – Its quality

• What did you think of the format and structure of the course? E.g. in terms of:

  – The length, frequency, pace of the sessions

  – The split between taught and group elements

• What did you think of the facilitator role? E.g. in terms of:– How they present themselves and communicate with group

  – Ability to manage group

  – Relevant knowledge and skills

• What did you think about the composition of each group that attend the course? E.g. in terms of:

  – How many there are, and is this too many or too few

  – How diverse or similar their characteristics are, including the mix of JSA v ESA claimants in each group

  – Extent to which people in group interact / support / learn from each other

• And just going back to our earlier discussion about retention and drop out, are they are features of the course (as opposed to the wider GW JOBS II process) that inhibit or reinforce engagement and course completion? If so, What? How?

• Are there any ways in which you think the course could be improved? Both in terms of:

  – Reinforcing engagement and course completion

  – Maximising the effectiveness of the course

7. Outcomes [10 minutes]

• Do you and/or your staff generally observe any changes in claimants over the duration of the course? Probe for any changes in:

  – Attitudes and behaviours towards job search / finding a job

  – Mental health, well-being, confidence and resilience.

  – Any other changes not captured above.

• In your experience are different types of claimants (e.g. different benefits claimants, hard and soft skills, those with poor mental health at the start, those closer to/furthest from work, etc.):

  – More or less responsive to Group Work provision? How? Why?

  – More or less likely to benefit from it? How? Why?

If changes noted:

• To what extent do you think the course contributes to / enables / helps these changes?
• Why? What are the active elements of the course that make a difference? Probe for whether they think one of these active elements is the ability of the course to mimic work, in terms of giving participants structure, activity, something meaningful to do, turning up each day.
• How? E.g. have they applied skills they learnt on the course? Or has it been more a change in mindset?

If no changes noted:

• Why do you think that is? Probe for the extent to which they believe this is because of:

  – External barriers or issues.

  – The course itself.

• Have your staff had any kind of follow-up contact with claimants after they have been on the course?
• Do your staff have any contact with Jobcentre Plus/individual Work Coaches once a claimant completes the course?
• Looking ahead, do you and/or your staff think taking part in the course may have any longer term benefits for claimants in the future? Why? Why not? How?

Close

• Thank them for sharing their experiences.
• Ask them if there is anything else they’d like to add.

A2: Provider (facilitator) topic guide

Guidance

This topic guide provides the key themes and sub-themes to be explored in interviews with Group Work facilitators. It is not a set script. The phrasing, pacing and ordering of questions should be tailored to reflect the individual respondent and the flow of the discussion in each interview.

The aim of the interview is to collect detailed, tangible insights into the respondent’s understanding and experiences of delivering the Group Work course, and their perceptions of its impact on claimants who go on it. This will feed directly into our assessment of how effectively the intervention has been designed and delivered.

Topic guide sections:

1. Introduction

Explaining the purpose of the interview, how the findings will be used, its confidentiality, and gaining their informed consent.

2. Respondent role

Their role, involvement in the trial, and personal/professional background.

3. Awareness and understanding

Their understanding of the aims of GW JOBS II, its key features, and the guidance and training they have received to support the delivery of the trial.

4. Referral Process

Their experiences of each stage in the process and views on its effectiveness.

5. Take up and drop out

Their perceptions of why claimants initially agree to participate and then either go on to drop out or complete.

6. Experiences of GW JOBS II

Their experiences and perceptions of the delivery and effectiveness of the GW JOBS II course.

7. Outcomes

Their perceptions of the effect of GW JOBS II on the attitudes, behaviours and circumstances of claimants in relation to health and employment. In preparation for each interview ensure you are familiar with: the information provided by DWP about the respondent; and the intended intervention process and GW JOBS II course content. Respondents should not be initially prompted with this information but any significant apparent differences between what they describe and the intended delivery model should be explored.

1. Introduction

• Thank them for giving their initial agreement to be interviewed.

• Explain that:

  – The interview is part of research that ICF is conducting for the Department for Work and Pensions (DWP) and the Department of Health (DH) about GW JOBS II.

  – They have asked us to evaluate the GW JOBS II trial, and as part of that we are interviewing facilitators who are delivering the Group Work course. The interview is not a test. We just want to hear their honest opinions, and there are no right or wrong answers.

  – ICF is an independent research consultancy and not part of DWP, DH or Jobcentre Plus.

  – Everything they say in the interview will be treated as confidential. This means that when we report the findings we will not say who we interviewed or who said what.

  – We would like to record the interview, just so we have an accurate record of what they say. Only members of the ICF research team will access to the recording. It will not be shared with DWP, DH, Jobcentre Plus staff or anyone else.

  – The interview will last up to 45 minutes.

• Ask them if what you have said is clear, if they have any questions, and then whether they agree to take part in the interview.
• If they agree, start the recorder and say “this is X with X”, state the date, “and can you just confirm that you’re happy to be recorded?’”

2. Respondent background [5 minutes]

• How long have you been a Group Work facilitator?

• What was your personal/professional background before you became a facilitator? E.g. any previous experience they had of:

  – Providing support / training to enable people to return to work – Being out of work

• Can you give me an overview of the number and type of claimants you are generally delivering the Group Work course to?

3. Awareness and understanding [5 minutes]

• What do you understand the aims of GW JOBS II to be?

• What is your understanding of its underlying logic / rationale? E.g. in terms of:

  – Who it is targeted at?

  – What issues / barriers it is trying to address?

  – How it is trying to do this?

  – How it compares / contrasts with other JC+ provision?

• Overall how well do you think you understand GW JOBS II?

• Thinking more broadly:

  – Do you think that having a job is good for individuals’ health and self-esteem? Why?

  – Conversely, do you think that being out of work is bad for individuals’ health? Why?

• Has your understanding changed at all over time / since the start of the trial? If so, How? Why? In what ways?

• What guidance or training have you received on GW JOBS II?

  – When it was provided/delivered?

  – Who by?

  – What it covered?

  – How useful was it?

• And having gone on to deliver the course yourself now, how effective do you think the guidance and training you received was in preparing you to do this?
• Are there any ways in which you think either the content, timing or delivery of the guidance/training could be improved?

4. Referral process [5 minutes]

• Can you tell me what you know about how Jobcentre Plus identify claimants for the course?
• What’s your impression of how well this is working?

• How suitable do you think the referrals your organisation is receiving are? E.g. in terms of:

  – Are they intended audience for GW JOBS II?

  – Have they taken an informed / voluntary decision to take part? Did any say they were told to attend?

• Roughly how many claimants have been referred to your organisation to take part in GW JOBS II since the start of the trial? How many have you worked with directly?

• Can you talk me through what the process is for Jobcentre Plus referring claimants to your organisation? Particularly in terms of:

  – The transfer of claimant details

  – The arrangement of the IRM meeting

  – Any subsequent co-ordination to reimburse claimants’ travel and childcare costs

• How effectively does this process work?
• Have there been any problems?
• Has the process changed at all over time? If so, Why? How?
• Are there any ways in which it can be improved?

5. Take-up and drop out [5 minutes]

• Roughly what proportion of claimants that are referred to you also subsequently attend the IRM?

  – Does this differ at all between different types of claimant? If so, How? Why?

• If a claimant doesn’t attend the IRM they were scheduled come to, what happens? E.g. can they attend a later one instead?

• Why do you think claimants initially agree or disagree to go on the course? – Does this differ at all between different types of claimant? If so, How? Why?

• And as far as you are aware, roughly what proportion go on to:

  – Start the Group Work course but drop out without completing it?

  – Start the Group Work course and complete it?

• Why do you think claimants either continue or drop out at these different points?

  – Does this differ at all between different types of claimant? If so, How? Why?

  – How would you rate levels of awareness, uptake, and effectiveness of help with travel and childcare costs?

• Are there any ways in which the process could be improved to increase retention / decrease attrition at these different points?

6. The Group Work course [15 minutes]

• How long after a referral does the claimant typically attend an IRM?
• What does the IRM involve (e.g. in terms of how it is structure and facilitated, and how many other people are there)?
• In terms of the Group Work course itself, roughly how long after the IRM does the claimant go on it?

• And can you give me a brief overview of how you are delivering the course itself? Specifically in terms of:

  – Its content

  – The format and structure

  – The facilitator role

  – The number / composition of people on the course

• Has this been changed or modified at all during the trial? If so, How? Why?
• Have you adapted or skipped elements of the course at all? If so, How? Why? With what consequences?

• Overall what do you think of the course? Allow for spontaneous responses then probe:

  – How effective it is

  – How claimants respond to it

  – What features of it work well

  – What features of it work less

  – How easy or difficult they have found it to deliver

If not already explored above:

• What did you think of the content of the course? E.g. in terms of:

  – What it covers (and balance between “work”/”job search” and “well-being”/”resilience”)

  – Its relevance and usefulness to the claimants

  – Its quality

• What did you think of the format and structure of the course? E.g. in terms of:

  – The length, frequency, pace of the sessions

  – The split between taught and group elements

• What did you think of the facilitator role? E.g. in terms of:

  – How they present themselves and communicate with group

  – Ability to manage group

  – Relevant knowledge and skills

• What did you think about the composition of each group that attend the course? E.g. in terms of:

  – How many there are, and is this too many or too few

  – How diverse or similar their characteristics are, including the mix of JSA v ESA claimants in each group

  – Extent to which people in group interact / support / learn from each other

• And just going back to our earlier discussion about retention and drop out, are they are features of the course (as opposed to the wider GW JOBS II process) that inhibit or reinforce engagement and course completion? If so, What? How?

• Are there any ways in which you think the course could be improved? Both in terms of:

  – Reinforcing engagement and course completion

  – Maximising the effectiveness of the course

7. Outcomes [10 minutes]

• Do you generally observe any changes in claimants over the duration of the course? Probe for any changes in:

  – Attitudes and behaviours towards job search / finding a job.

  – Mental health, well-being, confidence and resilience.

  – Any other changes not captured above.

• Does this differ at all between different types of claimant? If so, How? Why?

• In your experience are different types of claimants (e.g. different benefits claimants, hard and soft skills, those with poor mental health at the start, those closer to/furthest from work, etc.):

  – More or less responsive to Group Work provision? How? Why?

  – More or less likely to benefit from it? How? Why?

If changes noted:

• To what extent do you think the course contributes to / enables / helps these changes?
• Why? What are the active elements of the course that make a difference? Probe for whether they think one of these active elements is the ability of the course to mimic work, in terms of giving participants structure, activity, something meaningful to do, turning up each day.
• How? E.g. have they applied skills they learnt on the course? Or has it been more a change in mindset?

If no changes noted:

• Why do you think that is? Probe for the extent to which they believe this is because of:

  – External barriers or issues.

  – The course itself.

• Have you had any kind of follow-up contact with claimants after they have been on the course?
• Do you have any contact with Jobcentre Plus/individual Work Coaches once a claimant completes the course?
• Looking ahead, do you think taking part in the course may have any longer term benefits for claimants in the future? Why? Why not? How?

Close

• Thank them for sharing their experiences.
• Ask them if there is anything else they’d like to add.

A3: Jobcentre Plus (manager) topic guide

Guidance

This topic guide provides the key themes and sub-themes to be explored in interviews with Jobcentre Plus managers. It is not a set script. The phrasing, pacing and ordering of questions should be tailored to reflect the individual respondent and the flow of the discussion in each interview.

The aim of the interview is to collect detailed, tangible insights into the respondent’s understanding and experiences of the management and delivery of the intervention, and their perceptions of its impact on claimants. This will feed directly into our assessment of how effectively the intervention has been designed and delivered.

Topic guide sections:

1. Introduction

Explaining the purpose of the interview, how the findings will be used, its confidentiality, and gaining their informed consent.

2. Respondent and jobcentre background

Their role, involvement in the trial, and details of other staff in their jobcentre involved in the delivery of the trial.

3. Awareness and understanding Their understanding of the aims of GW JOBS II, its key features, and the guidance and training they have received to support the delivery of the trial.

4. Identification

How and why they identify claimants as potential GW JOBS II referrals.

5. Referral process Their experiences of each stage in the process and views on its effectiveness.

6. Take up and drop out Their perceptions of why claimants initially agree or disagree to participate, and either go on to drop out or complete.

7. Outcomes

Their perceptions of the effect of GW JOBS II on the subsequent attitudes, behaviours and circumstances of claimants in relation to health and employment.

In preparation for each interview ensure you are familiar with: the information provided by DWP about the respondent (including which referral process is being used in their office); and the intended intervention process and GW JOBS II course content. Respondents should not be initially prompted with this information but any significant apparent differences between what they describe and the intended delivery model should be explored.

1. Introduction

• Thank them for giving their initial agreement to be interviewed.

• Explain that:

  – The interview is part of research that ICF is conducting for the Department for Work and Pensions (DWP) and the Department of Health (DH) about GW JOBS II.

  – They have asked us to evaluate the GW JOBS II trial, and as part of that we are interviewing managers in the jobcentres where it is being trialled. The interview is not a test. We just want to hear their honest opinions, and there are no right or wrong answers.

  – ICF is an independent research consultancy and not part of DWP,DH or JobCentre Plus.

  – Everything they say in the interview will be treated as confidential. This means that when we report the findings we will not say who we interviewed or who said what.

  – We would like to record the interview, just so we have an accurate record of what they say. Only members of the ICF research team will access to the recording. It will not be shared with DWP, DH, other Jobcentre Plus staff or anyone else.

  – The interview will last up to 45 minutes.

• Ask them if what you have said is clear, if they have any questions, and then whether they agree to take part in the interview.
• If they agree, start the recorder and say “this is X with X”, state the date, “and can you just confirm that you’re happy to be recorded?’”

2. Respondent background [5 minutes]

• What is your job role?
• How long have you been in this role?
• How long have you been involved in the GW JOBS II trial?
• What are your specific responsibilities in relation to the trial?

• Can you give me an overview of:

  – The number and type of your staff involved in the delivery of the trial.

  – The number and type of claimants your staff generally work with (including benefits claimed).

3. Awareness and understanding [5 minutes]

• What do you understand the aims of GW JOBS II to be?

• And what is your understanding of its underlying logic / rationale? E.g. in terms of:

  – Who it is targeted at?

  – What issues / barriers it is trying to address?

  – How it is trying to do this?

  – How it compares / contrasts with other JC+ provision?

• What do you understand the GW JOBS II course itself involves? E.g. in terms of:

  – What its content is?

  – What its format / structure is?

  – How it is facilitated?

• Overall how well do you think you understand GW JOBS II?

• Thinking more broadly:

  – Do you think that having a job is good for individuals’ health and self-esteem? Why?

  – Conversely, do you think that being out of work is bad for individuals’ health? Why?

• Has your understanding changed at all over time / since the start of the trial? If so, How? Why? In what ways?
• What guidance or training have you received on GW JOBS II? – When it was provided/delivered? – Who by? – What it covered? – How useful was it?
• And how well do you think your staff understand GW JOBS II?
• Has their understanding changed at all over time / since the start of the trial? If so, How? Why? In what ways?

• What guidance or training have they received on GW JOBS II?

  – When it was provided/delivered?

  – Who by?

  – What it covered?

  – How useful was it?

• Overall, how effective was the guidance and training received by you and your staff?
• Are there any ways in which you think either the content, timing or delivery of the guidance/training could be improved? With hindsight, was there anything that wasn’t covered?

• What do you personally think about GW JOBS II? Probe to explore their underlying views and any limiting assumptions they might have, e.g. in terms of:

  – Their views on the value of third-party provided support in general

  – Extent to which they believe in the value of GW JOBS II specifically

  – The perceived ability of claimants with mental health issues to move into work

  – The perceived willingness of claimants to travel to and participate in support

• And what is your sense of how GW JOBS II is viewed by your colleagues in the jobcentre? Probe to explore views of management, team leaders, Work Coaches in terms of:

  – Attitudes to third-party provided support in general

  – The relative profile / importance / priority being given to GW JOBS II specifically

• In your experience, have these views influenced the number of referrals to the course to date?
• Have your personal views and/or those of your colleagues changed at all since the start of the trial? If so, How? Why?

4. Identification [10 minutes]

• Roughly how many claimants have your staff identified as being potential referrals to GW JOBS II? Probe to establish:

  – What this represents as a proportion of their caseload?

  – How it is distributed between JSA and ESA claimants?

  – Any variations between different staff (and reasons for this)?

  – Any changes over time / since the start of the trial (and reasons for this)?

• How do your staff go about identifying potential referrals to GW JOBS II?

  – What is the process for doing this?

  – What criteria do they use to judge whether someone is a potential referral?

  – How easy or difficult have they found it to judge whether someone is a potential referral? How confident do they feel about doing it? Why?

  – To what extent is it: fixed and determined by the guidance; or open to interpretation and with any element of personal judgement?

  – Are they comfortable/confident in discussing wellbeing and mental health issues with potential participants?

  – Any variations between different staff (and reasons for this)? – Any changes over time / since the start of the trial (and reasons for this)?

• How have you/your team supported the Work Coaches to recognise suitable claimants and deliver the referrals? Have you changed anything regarding appointments or case-loading approaches?
• Apart from claimants who do not meet the referral criteria, are there any types of claimants/characteristics that your staff wouldn’t refer to the course? If so, what/why?
• What specific guidance or training have your staff received on this part of the delivery process?
• Is the number and/or characteristics of potential referrals identified by staff monitored or recorded in any way within your office?
• Have they received any feedback from DWP about the number and/or characteristics of potential referrals being identified?
• How effectively do you think this element of the delivery model works?
• Are there any ways in which it could be improved?

5. Referral process [10 minutes]

• So once your staff have identified a potential referral, what happens next? For each of the following steps, ask them to describe how their staff go about it, claimant reactions, and any issues with delivery:

  – Consent to attend Group Work course

  – Appointment for IRM

  – Communication of claimant details to provider

  – Any subsequent co-ordination with provider to reimburse claimant travel and childcare costs

• How effectively do you think the referral process works?

  – Overall?

  – In terms of each of the elements we’ve talked about within it?

• How consistently is it being delivered by different staff? Probe for: – Any differences in practice

  – And any resultant implications of these differences for the effectiveness of delivery

• Are there any ways in which the process could be improved?

If not fully explored above:

Introducing GW JOBS II

• At what point in the process do staff first mention the Group Work course to the claimant?

• How do staff describe it to them?

  – What do they say? What language do they use?

  – Does their approach differ at all between different types of claimant? If so, How? Why?

  – How well do you think staff are able to explain the course to them?

  – Has their approach changed at all over time? If so, Why? How?

• What specific guidance or training have they received on this part of the delivery process?
• If their staff have received more than one round of training, how did the second training compare to the first?
• How useful was the training in enabling the staff to deliver the trial?

• How do claimants generally respond/react to GW JOBS II when staff explain it to them?

  – E.g. Do they seem to understand it? Do they ask any questions? If so, What?

  – Does this differ at all between different types of claimant? If so, How? Why?

• Are there any ways in which how GW JOBS II is presented / described to claimants could be improved? E.g. in terms of:

  – Encouraging more to agree to go on the course.

  – Preparing them better for going on the course.

Making the IRM appointment

All:

• Tell me a bit about the process for doing this, how involved is the claimant in setting the time of the appointment?
• How do claimants generally respond to this?
• Do you think the process used in this office had any influence on whether the claimant attended the reception meeting or not? If so, How? Why? How does it compare to other approaches?
• How well has the process been working with regards to calling the provider to book a claimant onto the Reception meeting? Are there any improvements that could be made to the process?

6. Take-up and drop out [5 minutes]

• Roughly what proportion of claimants agree to go on the course - either straightaway or at a later date?

  – Does this differ at all between different types of claimant? If so, How? Why?

  – Does this differ at all between different staff? If so, How? Why?

• Why do you think claimants initially agree or disagree to go on the course? – Does this differ at all between different types of claimant? If so, How? Why?

• And as far as you are aware, roughly what proportion go on to: – Attend the Initial Reception Meeting with the provider but not the Group Work course?

  – Start the Group Work course but drop out without completing it?

  – Start the Group Work course and complete it?

• Why do you think claimants either continue or drop out at these different points?

  – Does this differ at all between different types of claimant? If so, How? Why?

• Are there any ways in which the process could be improved to increase take-up and decrease attrition at these different points? E.g. do you think feedback from individuals that have completed the course and found it beneficial could help improve take-up?

7. Outcomes [10 minutes]

• Have you and/or your staff observed any changes in claimants after they have been on the course? Probe for any changes in their:

  – Employment circumstances.

  – Attitudes and behaviours towards job search / finding a job (e.g. are feeling more positive about it, doing more of it, approaching it differently, looking for different types of jobs, etc.)

  – Mental health, well-being, confidence and resilience.

  – Any other changes not captured above.

• In your experience are different types of claimants (e.g. different benefits claimants, hard and soft skills, those with poor mental health at the start, those closer to/furthest from work, etc.):

  – More or less responsive to Group Work provision? How? Why?

  – More or less likely to benefit from it? How? Why?

If changes noted:

• To what extent do you and/or your staff think the course contributed to / enabled / helped these changes?
• Why? What are the active elements of the course that make a difference? Probe for whether they think one of these active elements is the ability of the course to mimic work, in terms of giving participants structure, activity, something meaningful to do, turning up each day.
• How? E.g. have they applied skills they learnt on the course? Or has it been more a change in mindset?

If changes in more than one area noted:

• Were these all a direct result of going on the course or did one lead on to another? e.g. did any mental health benefits arise directly as a consequence of the course or following an improvement in their employment circumstances after the course?

If no changes noted:

• Why do you think that is? Probe for the extent to which they believe this is because of:

  – External barriers or issues.

  – The course itself.

All:

• Are there any elements or aspects of the Group Work course that you think might negatively affect some claimants? If so, Who? How? Why?
• Do your staff have any kind of follow-up discussion with claimants after they have been on the course?
• Do they do anything to build on the momentum established with claimants who complete the course on their return? If so, What?
• Are you aware claimants having any ongoing contact with other participants or facilitators after the course?
• Have claimants experienced any setbacks since going on the course? If so, What happened? How did they respond? Was there any evidence of them being more resilient because of the course?
• Looking ahead, do you think taking part in the course may have any longer term benefits for claimants in the future? Why? Why not? How?

Close

• Thank them for sharing their experiences.
• Ask them if there is anything else they’d like to add.

A4: Jobcentre Plus (work coach) topic guide

Guidance

This topic guide provides the key themes and sub-themes to be explored in interviews with Jobcentre Plus work coaches. It is not a set script. The phrasing, pacing and ordering of questions should be tailored to reflect the individual respondent and the flow of the discussion in each interview.

The aim of the interview is to collect detailed, tangible insights into the respondent’s understanding and experiences of delivering the intervention, and their perceptions of its impact on claimants. This will feed directly into our assessment of how effectively the intervention has been designed and delivered.

Topic guide sections:

1. Introduction Explaining the purpose of the interview, how the findings will be used, its confidentiality, and gaining their informed consent.

2. Respondent role Their role, involvement in the trial, and their caseload.

3. Awareness and understanding

Their understanding of the aims of GW JOBS II, its key features, and the guidance and training they have received to support the delivery of the trial.

4. Identification How and why they identify claimants as potential GW JOBS II referrals.

5. Referral process Their experiences of each stage in the process and views on its effectiveness.

6. Take up and drop out Their perceptions of why claimants initially agree or disagree to participate, and either go on to drop out or complete.

7. Outcomes

Their perceptions of the effect of GW JOBS II on the subsequent attitudes, behaviours and circumstances of claimants in relation to health and employment.

In preparation for each interview ensure you are familiar with: the information provided by DWP about the respondent (including which referral process is being used in their office); and the intended intervention process and GW JOBS II course content. Respondents should not be initially prompted with this information but any significant apparent differences between what they describe and the intended delivery model should be explored.

1. Introduction

• Thank them for giving their initial agreement to be interviewed.

• Explain that:

  – The interview is part of research that ICF is conducting for the Department for Work and Pensions (DWP) and the Department of Health (DH) about GW JOBS II.

  – They have asked us to evaluate the GW JOBS II trial, and as part of that we are interviewing work coaches in the jobcentres where it is being trialled. The interview is not a test. We just want to hear their honest opinions, and there are no right or wrong answers.

  – ICF is an independent research consultancy and not part of DWP, DH or JobCentre Plus.

  – Everything they say in the interview will be treated as confidential. This means that when we report the findings we will not say who we interviewed or who said what.

  – We would like to record the interview, just so we have an accurate record of what they say. Only members of the ICF research team will access to the recording. It will not be shared with DWP, DH, other Jobcentre Plus staff or anyone else. – The interview will last up to 45 minutes.

• Ask them if what you have said is clear, if they have any questions, and then whether they agree to take part in the interview.
• If they agree, start the recorder and say “this is X with X”, state the date, “and can you just confirm that you’re happy to be recorded?’”

2. Respondent background [5 minutes]

• What is your job role (i.e. work coach or assistant work coach)?
• How long have you been in this role?
• How long have you been involved in the GW JOBS II trial?
• Can you give me an overview of the number and type of claimants you generally work with (including benefits claimed)?

3. Awareness and understanding [5 minutes]

• How was GW JOBS II introduced to you?
• What do you understand the aims of GW JOBS II to be?

• And what is your understanding of its underlying logic / rationale? E.g. in terms of:

  – Who it is targeted at?

  – What issues / barriers it is trying to address?

  – How it is trying to do this?

  – How it compares / contrasts with other JC+ provision?

• What do you understand the GW JOBS II course itself involves? E.g. in terms of:

  – What its content is?

  – What its format / structure is?

  – How it is facilitated?

• Has your understanding changed at all over time / since the start of the trial? If so, How? Why? In what ways?
• What guidance or training have you received on GW JOBS II? – When it was provided/delivered? – Who by? – What it covered? – How useful was it?
• How effective was this in supporting your understanding of what the GW JOBS II course involves?

• Thinking more broadly:

  – Do you think that having a job is good for individuals’ health and self-esteem? Why?

  – Conversely, do you think that being out of work is bad for individuals’ health? Why?

• Are there any ways in which you think either the content, timing or delivery of the guidance/training could be improved? With hindsight, was there anything that wasn’t covered?

• What do you personally think about GW JOBS II? Probe to explore their underlying views and any limiting assumptions they might have, e.g. in terms of:

  – Their views on the value of third-party provided support in general

  – Extent to which they believe in the value of GW JOBS II specifically

  – The perceived ability of claimants with mental health issues to move into work

  – The perceived willingness of claimants to travel to and participate in support

  – The feedback from claimants who have attended the support

  – Any personal visit/observation of the course delivery

• And what is your sense of how GW JOBS II is viewed by your colleagues in the jobcentre? Probe to explore views of management, team leaders, other Work Coaches in terms of:

  – Attitudes to third-party provided support in general

  – The relative profile / importance / priority being given to GW JOBS II specifically

• In your experience, have these views influenced the number of referrals to the course to date?
• Have your personal views and/or those of your colleagues changed at all since the start of the trial? If so, How? Why?

4. Identification [10 minutes]

• Roughly how many claimants have you identified as being potential referrals to GW JOBS II? Probe to establish:

  – What this represents as a proportion of their caseload?

  – How it is distributed between JSA and ESA claimants?

  – Any changes over time / since the start of the trial (and reasons for this)?

• How do you go about identifying potential referrals to GW JOBS II?

  – What is the process for doing this?

  – What criteria do you use to judge whether someone is a potential referral?

  – How easy or difficult have you found it to judge whether someone is a potential referral? How confident do you feel about doing it? Why?

  – To what extent is it: fixed and determined by the guidance; or open to interpretation and with any element of personal judgement?

  – Are you comfortable/confident in discussing wellbeing and mental health issues with potential participants? – Any changes over time / since the start of the trial (and reasons for this)?

  – How has the guidance and training you have received supported this?

• Apart from claimants who do not meet the referral criteria, are there any types of claimants/characteristics that you wouldn’t refer to the course? If so, what/why?
• What specific guidance or training have you received on this part of the delivery process?
• Is the number and/or characteristics of potential referrals identified by work coaches monitored or recorded in any way within your office?
• Have you received any feedback from senior managers in DWP or Jobcentre Plus about the number and/or characteristics of potential referrals being identified?
• How effectively do you think this element of the delivery model works?
• Are there any ways in which it could be improved?

5. Referral process [10 minutes]

• So once you have identified a potential referral, what happens next? For each of the following steps, ask them to describe how they introduce it to claimants, what the claimant and/or they have to do, and how claimants respond to it:

  – Consent to attend Group Work course

  – Appointment for IRM

  – Communication of claimant details to provider

  – Any subsequent co-ordination with provider to reimburse claimant travel and childcare costs

• How effectively do you think the referral process works?

  – Overall?

  – In terms of each of the elements we’ve talked about within it?

• Are there any ways in which it could be improved?

If not fully explored above:

Introducing GW JOBS II

• At what point in the process do you first mention the Group Work course to the claimant?

• How do you describe it to them?

  – What do you say? What language do you use?

  – Does your approach differ at all between different types of claimant? If so, How? Why?

  – How well do you think you are able to explain the course to them?

  – Has your approach changed at all over time? If so, Why? How?

• What specific guidance or training have you received on this part of the delivery process?
• If they have received more than one round of training, how did the second training compare to the first?

• How do claimants generally respond/react to GW JOBS II when you explain it to them?

  – E.g. Do they seem to understand it? Do they ask any questions? If so, What?

  – Does this differ at all between different types of claimant? If so, How? Why?

• Are there any ways in which how GW JOBS II is presented / described to claimants could be improved? E.g. in terms of:

  – Encouraging more to agree to go on the course.

  – Preparing them better for going on the course.

Making the IRM appointment

• Tell me a bit about the process for doing this, how involved is the claimant in setting the time of the appointment?
• How do claimants generally respond to this?
• Do you think the process used in this office had any influence on whether the claimant attended the reception meeting or not? If so, How? Why? How does it compare to other approaches?
• How well has the process been working with regards to calling the provider to book a claimant onto the Reception meeting? Are there any improvements that could be made to the process?

6. Take-up and drop out [5 minutes]

• Roughly what proportion of claimants agree to go on the course - either straightaway or at a later date?

  – Does this differ at all between different types of claimant? If so, How? Why?

• Why do you think claimants initially agree or disagree to go on the course? – Does this differ at all between different types of claimant? If so, How? Why?

• And as far as you are aware, roughly what proportion go on to: – Attend the Initial Reception Meeting with the provider but not the Group Work course?

  – Start the Group Work course but drop out without completing it?

  – Start the Group Work course and complete it?

• Why do you think claimants either continue or drop out at these different points?

  – Does this differ at all between different types of claimant? If so, How? Why?

• Are there any ways in which the process could be improved to increase retention / decrease attrition at these different points? E.g. do you think feedback from individuals that have completed the course and found it beneficial could help improve take-up?

7. Outcomes [10 minutes]

• Have you observed any changes in claimants after they have been on the course? Probe for any changes in their:

  – Employment circumstances.

  – Attitudes and behaviours towards job search / finding a job (e.g. are they feeling more positive about it, doing more of it, approaching it differently, looking for different types of jobs, etc.)

  – Mental health, well-being, confidence and resilience.

  – Any other changes not captured above.

• In your experience are different types of claimants (e.g. different benefits claimants, hard and soft skills, those with poor mental health at the start, those closer to/furthest from work, etc.):

  – More or less responsive to Group Work provision? How? Why?

  – More or less likely to benefit from it? How? Why?

• If changes noted:
• To what extent do you think the course contributed to / enabled / helped these changes?
• Why? What are the active elements of the course that make a difference? Probe for whether they think one of these active elements is the ability of the course to mimic work, in terms of giving participants structure, activity, something meaningful to do, turning up each day.
• How? E.g. have they applied skills they learnt on the course? Or has it been more a change in mindset?

If changes in more than one area noted:

• Were these all a direct result of going on the course or did one lead on to another? e.g. did any mental health benefits arise directly as a consequence of the course or following an improvement in their employment circumstances after the course?

If no changes noted:

• Why do you think that is? Probe for the extent to which they believe this is because of:

  – External barriers or issues.

  – The course itself.

All:

• Are there any elements or aspects of the Group Work course that you think might negatively affect some claimants? If so, Who? How? Why?
• Have you had any kind of follow-up discussion with claimants after they have been on the course?
• Do you do anything to build on the momentum established with claimants who complete the course on their return? If so, what?
• Are you aware claimants having any ongoing contact with other participants or facilitators after the course?
• Have claimants experienced any setbacks since going on the course? If so, What happened? How did they respond? Was there any evidence of them being more resilient because of the course?
• Looking ahead, do you think taking part in the course may have any longer term benefits for claimants in the future? Why? Why not? How?

Close

• Thank them for sharing their experiences.
• Ask them if there is anything else they’d like to add.

A5: Claimant (completer) topic guide

Guidance

This topic guide provides the key themes and sub-themes to be explored in interviews with claimants who completed the GW JOBS II course. It is not a set script. The phrasing, pacing and ordering of questions should be tailored to reflect the individual respondent and the flow of the discussion in each interview.

The aim of the interview is to collect detailed, tangible insights into the respondent’s experience of the intervention; its impacts; and what factors underpin / influence / explain these experiences and impacts. This will feed directly into our assessment of how effectively the intervention has been designed and delivered.

Topic guide sections:

1. Introduction

Explaining the purpose of the interview, how the findings will be used, its confidentiality, and gaining their informed consent.

2. Respondent background Confirming their basic details, personal circumstances, health, employment and benefits history pre-GW JOBS II.

3. Initial understanding and decision making How they became aware of GW JOBS II, what information was provided, what their understanding of it was, and why they decided to take part.

4. Referral process

Their experiences of each stage in the process and views on its effectiveness.

5. Experiences of GW JOBS II Their experiences and perceptions of the format, content and facilitation of the GW JOBS II course, including any barriers or facilitators to their engagement.

6. Outcomes

Their attitudes, behaviours and circumstances in relation to health and employment since participation, and the perceived impact of GW JOBS II in these areas.

In preparation for each interview ensure you are familiar with: the information provided by DWP about the respondent (including which referral process is being used in their jobcentre); and the intended intervention process and GW JOBS II course content. Respondents should not be initially prompted with this information but any significant apparent differences between what they describe and the intended delivery model should be explored.

1. Introduction

• Thank them for giving their initial agreement to be interviewed.

• Explain that:

  – The interview is part of research that ICF is conducting for the Department for Work and Pensions (DWP) and the Department of Health (DH) about the Group Work course.

  – They have asked us to evaluate the course, and as part of that we are interviewing people who have recently been on it. The interview is not a test. We just want to hear their honest opinions, and there are no right or wrong answers.

  – ICF is an independent research consultancy and not part of DWP, DH or Jobcentre Plus.

  – Everything they say in the interview will be treated as confidential. This means that when we report the findings we will not say who we interviewed or who said what.

  – Taking part in the interview is completely voluntary. They can decide not to take part now or at any point during the interview. This will not affect their benefits in any way.

  – They also do not have to answer any questions they do not feel comfortable answering.

  – We would like to record the interview, just so we have an accurate record of what they say. Only members of the ICF research team will access to the recording. It will not be shared with DWP, DH, Jobcentre Plus staff or anyone else.

  – The interview will last up to 45 minutes and we will send them a £15 voucher they can use in most high street shops by post afterwards, as a thank you for taking part.

• Ask them if what you have said is clear, if they have any questions, and then whether they agree to take part in the interview.
• If they agree, start the recorder and say “this is X with X”, state the date, “and can you just confirm that you’re happy to be recorded?’”

2. Respondent background [5 minutes]

• To start us off, can you tell me a bit about what your situation was before you went on the Group Work course? Probe to briefly establish:

  – What benefits they were claiming and how long for.

  – Current/previous experiences of employment.

  – Perceived constraints/barriers to employment.

  – Attitudes and behaviours towards job search / finding a job.

  – Their general health and well-being.

  – Usage of any kinds of treatment/support, e.g. from their GP or hospital etc.

  – Their financial situation (e.g. did you feel financially strained? did this prevent you from doing certain things – going out to meet people, worries about not being able to pay bills etc.?

  – Day-to-day activities (e.g. could you briefly describe your average or a normal day before attending GW?)

  – Access to social support/networks (e.g. did you have family or friends to talk to / help you get by? did you feel at all isolated?)

  – Feelings about their situation, e.g. How did your situation make you feel / feel about your life? [Do not explicitly ask about mental health here unless raised spontaneously]

• Thinking more broadly:

  – Do you think having a job is good for your health and how are feeling? Why?

  – Do you think being out of work is ‘bad’ for health and how are feeling? Why?

3. Initial understanding and decision-making [10 minutes]

• Can you remember how you heard about the Group Work course? Probe to briefly establish:

  – Who they heard about it from and how it was introduced to them.

  – How it was described / what information was provided.

• And before you went on the course, what did you understand it was about? Probe to establish their understanding of:

  – Who it was target at / intended for.

  – What it would involve.

  – Whether going on it was voluntary or mandatory.

  – Why they thought they were being told to / given the opportunity to go on it.

  – What they thought they would get out of it.

If they understood it to be voluntary:

• Why did you decide to go on the course? If they understood it to be mandatory:
• What did you think the consequences would be if you did not go on the course?
• How would you have felt about going on the course if you had understood it was voluntary?

All:

• Having been on the course yourself now, what do you think about the information that was provided about it beforehand? E.g. in terms of the extent to which it:

  – Enabled them to make an informed decision about taking part.

  – Prepared them for taking part.

• Are there any ways in which the information provided at this point could be improved? E.g. in terms of:

  – The timing and amount of information provided.

  – What the information says about the course.

  – How the information is provided / delivered / communicated (inc. the language used).

  – Hearing feedback from others who have already been on the course (i.e. would this be useful?)

4. Referral process [5 minutes]

• Thinking back to when the Work Coach told you about the course, what was the process for you getting referred onto it? Allow for spontaneous responses then, if necessary, prompt them for their recollection of:

  – Giving their consent to attend the course.

  – Making the appointment for the Initial Reception Meeting.

• Overall how did you find the process?
• Are there any ways in which you think the process could be improved?

5. Experiences of GW JOBS II [15 minutes]

• Roughly how long after the meeting with your Work Coach did you have an initial meeting with the people providing the Group Work course?
• What did it involve (e.g. in terms of how it was structure and facilitated, and how many other people were there)?

• Did it have any effect on:

  – Your understanding of the course?

  – Your subsequent attitude to attending?

• In terms of the Group Work course itself, roughly how long after the initial meeting did you go on it?

• And overall what did you think of the course? Allow for spontaneous responses then probe:

  – Whether it was overall a positive or negative experience

  – What features of it they liked / worked well

  – Any features of it they disliked / worked less well

  – How it compared with what they expected

  – How it compared with any other courses they have attended before

If not already explored above:

• What did you think of the content of the course? E.g. in terms of:

  – What it covered (and balance between “work”/”job search” and “well-being”/”resilience”)

  – Its relevance and usefulness to them

  – Its quality

• What did you think of the format and structure of the course? E.g. in terms of:

  – The length, frequency, pace of the sessions

  – The split between taught and group elements

• What did you think of the facilitator(s) of the course? E.g. in terms of:

  – How they presented themselves and communicated with group

  – Ability to manage group

  – Relevant knowledge and skills

• What did you think about the other people on the course? E.g. in terms of:

  – How many were there, and was this it too many or too few

  – What were the characteristics of the other people

  – Extent to which people in group interacted / supported / learned from each other (e.g. did you meet new people? did you make friends who are in the same situation as you?)

• And just to check, did you attend all of the sessions on the course?
• Why? What motivated / demotivated you to attend?
• Were there any practical issues that helped or made it difficult for you to attend? E.g. in terms of travel, childcare, or other factors?

If relevant, probe for:
– Awareness, uptake, and effectiveness of help with travel and childcare costs.

• Thinking about your experiences of the course, are there any ways in which you think it could be improved? E.g. in terms of:

  – Its content

  – The format and structure

  – The facilitator

  – The number / composition of people on the course

6. Outcomes [10 minutes]

• Has anything changed since you’ve been on the course? Probe for any changes in their:

  – Employment circumstances.

  – Financial situation (i.e. has this improved at all?).

  – Day-to-day activities (e.g. are these more structured? are they getting out of the house more? etc.)

  – Attitudes and behaviours towards job search / finding a job (e.g. are they feeling more positive about it, doing more of it, approaching it differently, looking for different types of jobs, etc.)

  – General health and well-being (or whatever language the respondent has used themselves in the interview – try to explore any outcomes in the areas of: mental health, confidence and resilience).

  – Any other changes not captured above (e.g. has anything else changed in your life because of the course?)

If changes noted:

• To what extent did the course contribute to / enable / help these changes?

• Why? What features of how the course was structured, its content, the facilitators, and other participants made a difference? Probe for the extent to which the course:

  – Mimicked work/gave them a better routine, e.g. by providing structure, activity, something meaningful to do, turning up each day etc. – Got them to meet people like themselves and developing social networks / friends and feel less socially isolated. – Enabled them to share information or help each other with access to job adverts etc.

If no changes noted:

• Why do you think that is? Probe for the extent to which they believe this is because of:

  – External barriers or issues.

  – The course itself.

All:

• Are there any elements or aspects of the Group Work course that had a negative effect on you? If so, How? Why?

• Can you tell me a bit about what happened after you finished the course? E.g. in terms of:

  – Any follow-up discussions with Work Coach.

  – Further contact with people on the course or facilitator.

• And have you subsequently sought any other kinds of guidance or support since going on the course? Probe to establish if going on the course has prompted them to access any of the following and their reasons and experiences of this:

  – Other help with job search/finding a job

  – Other help related to health/mental health (e.g. from their GP, hospital etc.)

• Have you experienced any setbacks since going on the course? If so, What happened? How did you respond? Probe for any evidence of them being more resilient because of the course
• Looking ahead, do you think taking part in the course may have any longer term benefits for you in the future? Why? Why not? How?

Close

• Thank them for sharing their experiences.
• Ask them if there is anything else they’d like to add.
• Confirm arrangements for sending them the incentive.

A6: Claimant (non-completer) topic guide

Guidance

This topic guide provides the key themes and sub-themes to be explored in interviews with claimants who started the GW JOBS II course but did not complete it. It is not a set script. The phrasing, pacing and ordering of questions should be tailored to reflect the individual respondent and the flow of the discussion in each interview.

The aim of the interview is to collect detailed, tangible insights into the respondent’s experience of the intervention; its impacts; and what factors underpin / influence / explain these experiences and impacts. This will feed directly into our assessment of how effectively the intervention has been designed and delivered.

Topic guide sections:

1. Introduction

Explaining the purpose of the interview, how the findings will be used, its confidentiality, and gaining their informed consent.

2. Respondent background Confirming their basic details, personal circumstances, health, employment and benefits history pre-GW JOBS II.

3. Initial understanding and decision making

How they became aware of GW JOBS II, what information was provided, what their understanding of it was, and why they decided to take part.

4. Referral process

Their experiences of each stage in the process and views on its effectiveness.

5. Experiences of GW JOBS II Their experiences and perceptions of the format, content and facilitation of the GW JOBS II course, including their reasons for dropping out.

6. Outcomes

Their attitudes, behaviours and circumstances in relation to health and employment since participation, and the perceived impact of GW JOBS II in these areas.

In preparation for each interview ensure you are familiar with: the information provided by DWP about the respondent (including which referral process is being used in their jobcentre); and the intended intervention process and GW JOBS II course content. Respondents should not be initially prompted with this information but any significant apparent differences between what they describe and the intended delivery model should be explored.

1. Introduction

• Thank them for giving their initial agreement to be interviewed.

• Explain that:

  – The interview is part of research that ICF is conducting for the Department for Work and Pensions (DWP) and the Department of Health (DH) about the Group Work course.

  – They have asked us to evaluate the course, and as part of that we are interviewing people who have recently been on it. The interview is not a test. We just want to hear their honest opinions, and there are no right or wrong answers.

  – ICF is an independent research consultancy and not part of DWP, DH or Jobcentre Plus.

  – Everything they say in the interview will be treated as confidential. This means that when we report the findings we will not say who we interviewed or who said what.

  – Taking part in the interview is completely voluntary. They can decide not to take part now or at any point during the interview. This will not affect their benefits in any way. – They also do not have to answer any questions they do not feel comfortable answering.

  – We would like to record the interview, just so we have an accurate record of what they say. Only members of the ICF research team will access to the recording. It will not be shared with DWP, DH, Jobcentre Plus staff or anyone else.

  – The interview will last up to 45 minutes and we will send them a £15 voucher they can use in most high street shops by post afterwards, as a thank you for taking part.

• Ask them if what you have said is clear, if they have any questions, and then whether they agree to take part in the interview.
• If they agree, start the recorder and say “this is X with X”, state the date, “and can you just confirm that you’re happy to be recorded?’”

2. Respondent background [5 minutes]

• To start us off, can you tell me a bit about what your situation was before you went on the Group Work course? Probe to briefly establish:

  – What benefits they were claiming and how long for.

  – Current/previous experiences of employment.

  – Perceived constraints/barriers to employment.

  – Attitudes and behaviours towards job search / finding a job.

  – Their general health and well-being.

  – Usage of any kinds of treatment/support, e.g. from their GP or hospital etc.

  – Their financial situation (e.g. did you feel financially strained? did this prevent you from doing certain things – going out to meet people, worries about not being able to pay bills etc.?

  – Day-to-day activities (e.g. could you briefly describe your average or a normal day before attending GW?)

  – Access to social support/networks (e.g. did you have family or friends to talk to / help you get by? did you feel at all isolated?)

  – Feelings about their situation, e.g. How did your situation make you feel / feel about your life? [Do not explicitly ask about mental health here unless raised spontaneously]

• Thinking more broadly:

  – Do you think having a job is good for your health and how are feeling? Why?

  – Do you think being out of work is ‘bad’ for health and how are feeling? Why?

3. Initial understanding and decision-making [10 minutes]

• Can you remember how you heard about the Group Work course? Probe to briefly establish:

  – Who they heard about it from and how it was introduced to them.

  – How it was described / what information was provided.

• And before you went on the course, what did you understand it was about? Probe to establish their understanding of:

  – Who it was target at / intended for.

  – What it would involve.

  – Whether going on it was voluntary or mandatory.

  – Why they thought they were being told to / given the opportunity to go on it.

  – What they thought they would get out of it.

If they understood it to be voluntary:

• Why did you decide to go on the course? If they understood it to be mandatory:
• What did you think the consequences would be if you did not go on the course?
• How would you have felt about going on the course if you had understood it was voluntary?

All:

• Having been on the course yourself now, what do you think about the information that was provided about it beforehand? E.g. in terms of the extent to which it:

  – Enabled them to make an informed decision about taking part.

  – Prepared them for taking part.

• Are there any ways in which the information provided at this point could be improved? E.g. in terms of:

  – The timing and amount of information provided.

  – What the information says about the course.

  – How the information is provided / delivered / communicated (inc. the language used).

  – Hearing feedback from others who have already been on the course (i.e. would this be useful?)

4. Referral process [5 minutes]

• Thinking back to when the Work Coach told you about the course, what was the process for you getting referred onto it? Allow for spontaneous responses then, if necessary, prompt them for their recollection of:

  – Giving their consent to attend the course.

  – Making the appointment for the Initial Reception Meeting.

• Overall how did you find the process?
• Are there any ways in which you think the process could be improved?

5. Experiences of GW JOBS II [15 minutes]

• And how much of the Group Work course did you then attend? Establish at what point they dropped out and then explore their experiences of the IRM and the elements of the course they did attend.
• Roughly how long after the meeting with your Work Coach did you have an initial meeting with the people providing the Group Work course?
• What did it involve (e.g. in terms of how it was structure and facilitated, and how many other people were there)?

• Did it have any effect on:

  – Your understanding of the course?

  – Your subsequent attitude to attending?

• Did you go on to attend the course? Why? Why not?
• In terms of the Group Work course itself, roughly how long after the initial meeting did you go on it?

• And overall what did you think of the course? Allow for spontaneous responses then probe:

  – Whether it was overall a positive or negative experience

  – What features of it they liked / worked well

  – Any features of it they disliked / worked less well

  – How it compared with what they expected

  – How it compared with any other courses they have attended before

If not already explored above:

• What did you think of the content of the course? E.g. in terms of:

  – What it covered (and balance between “work”/”job search” and “well-being”/”resilience”)

  – Its relevance and usefulness to them

  – Its quality

• What did you think of the format and structure of the course? E.g. in terms of:

  – The length, frequency, pace of the sessions

  – The split between taught and group elements

• What did you think of the facilitator(s) of the course? E.g. in terms of:

  – How they presented themselves and communicated with group

  – Ability to manage group

  – Relevant knowledge and skills

• What did you think about the other people on the course? E.g. in terms of:

  – How many were there, and was this it too many or too few

  – What were the characteristics of the other people

  – Extent to which people in group interacted / supported / learned from each other (e.g. did you meet new people? did you make friends who are in the same situation as you?)

• Why did you not complete the Group Work course? What, if anything, put you off or demotivated you from attending?

• Were there any practical issues that helped or made it difficult for you to attend? E.g. in terms of location, travel, childcare or other factors? If relevant, probe for:

  – Awareness, uptake, and effectiveness of help with travel and childcare costs.

• Is there anything that could have prevented you from completing the course? If so, What? How?
• As far as you are aware, could you go back on the course if you wanted to? If so, would you consider going back on it in the future? Why? Why not?

• Thinking about your experiences of the course, are there any ways in which you think it could be improved? E.g. in terms of:

  – Its content

  – The format and structure

  – The facilitator

  – The number/composition of people on the course

6. Outcomes [10 minutes]

• Has anything changed since you’ve been on the course? Probe for any changes in their:

  – Employment circumstances.

  – Financial situation (i.e. has this improved at all?).

  – Day-to-day activities (e.g. are these more structured? are they getting out of the house more? etc.)

  – Attitudes and behaviours towards job search / finding a job (e.g. are they feeling more positive about it, doing more of it, approaching it differently, looking for different types of jobs, etc.)

  – General health and well-being (or whatever language the respondent has used themselves in the interview – try to explore any outcomes in the areas of: mental health, confidence and resilience).

  – Any other changes not captured above, e.g. Has anything else changed in your life because of the course?

If changes noted:

• To what extent did the course contribute to / enable / help these changes?

• Why? What features of how the course was structured, its content, the facilitators, and other participants made a difference? Probe for the extent to which the course:

  – Mimicked work/gave them a better routine, e.g. by providing structure, activity, something meaningful to do, turning up each day etc.

  – Got them to meet people like themselves and developing social networks / friends and feel less socially isolated. – Enabled them to share information or help each other with access to job adverts etc.

If no changes noted:

• Why do you think that is? Probe for the extent to which they believe this is because of:

  – External barriers or issues.

  – The course itself.

All:

• Are there any elements or aspects of the Group Work course that had a negative effect on you? If so, How? Why?

• Can you tell me a bit about what happened after you left the course? E.g. in terms of:

  – Any follow-up discussions with Work Coach.

  – Further contact with people on the course or facilitator.

• And have you subsequently sought any other kinds of guidance or support since going on the course? Probe to establish if going on the course has prompted them to access any of the following and their reasons and experiences of this:

  – Other help with job search/finding a job

  – Other help related to health/mental health (e.g. from their GP, hospital etc.)

• Have you experienced any setbacks since going on the course? If so, What happened? How did you respond? Probe for any evidence of them being more resilient because of the course
• Looking ahead, do you think taking part in the course may have any longer term benefits for you in the future? Why? Why not? How?

Close

• Thank them for sharing their experiences.
• Ask them if there is anything else they’d like to add.
• Confirm arrangements for sending them the incentive.

A7: Claimant (decliner) topic guide

Guidance

This topic guide provides the key themes and sub-themes to be explored in interviews with claimants who were offered the opportunity to go on the GW JOBS II course but declined. It is not a set script. The phrasing, pacing and ordering of questions should be tailored to reflect the individual respondent and the flow of the discussion in each interview.

The aim of the interview is to collect detailed, tangible insights into the respondent’s experience of being offered the opportunity to access GW JOBS II and their reasons for declining. This will feed directly into our assessment of how effectively the intervention has been designed and delivered.

Topic guide sections:

1. Introduction Explaining the purpose of the interview, how the findings will be used, its confidentiality, and gaining their informed consent.

2. Respondent background Confirming their basic details, personal circumstances, health, employment and benefits history.

3. Understanding and decision making

How they became aware of GW JOBS II, what information was provided, what their understanding of it was, and why they decided not to take part.

In preparation for each interview ensure you are familiar with: the information provided by DWP about the respondent; and the intended intervention process and GW JOBS II course content. Respondents should not be initially prompted with this information but any significant apparent differences between what they describe and the intended delivery model should be explored.

1. Introduction

• Thank them for giving their initial agreement to be interviewed.

• Explain that:

  – The interview is part of research that ICF is conducting for the Department for Work and Pensions (DWP) and the Department of Health (DH) about the Group Work course.

  – They have asked us to evaluate the course, and as part of that we are interviewing people who have recently been invited to go on it. The interview is not a test. We just want to hear their honest opinions, and there are no right or wrong answers.

  – ICF is an independent research consultancy and not part of DWP, DH or Jobcentre Plus.

  – Everything they say in the interview will be treated as confidential. This means that when we report the findings we will not say who we interviewed or who said what.

  – Taking part in the interview is completely voluntary. They can decide not to take part now or at any point during the interview. This will not affect their benefits in any way.

  – They also do not have to answer any questions they do not feel comfortable answering.

  – We would like to record the interview, just so we have an accurate record of what they say. Only members of the ICF research team will access to the recording. It will not be shared with DWP, DH, Jobcentre Plus staff or anyone else.

  – The interview will last up to 30 minutes and we will send them a £15 voucher they can use in most high street shops by post afterwards, as a thank you for taking part.

• Ask them if what you have said is clear, if they have any questions, and then whether they agree to take part in the interview.
• If they agree, start the recorder and say “this is X with X”, state the date, “and can you just confirm that you’re happy to be recorded?’”

2. Respondent background [5 minutes]

• To start us off, can you tell me a bit about your current situation? Probe to briefly establish:

  – What benefits they are claiming and how long for.

  – Current/previous experiences of employment.

  – Perceived constraints/barriers to employment.

  – Attitudes and behaviours towards job search / finding a job.

  – Their general health and well-being.

  – Usage of any kinds of treatment/support, e.g. from their GP or hospital etc.

  – Their financial situation (e.g. did you feel financially strained? did this prevent you from doing certain things – going out to meet people, worries about not being able to pay bills etc.?

  – Day-to-day activities (e.g. could you briefly describe your average or a normal day before attending GW?)

  – Access to social support/networks (e.g. did you have family or friends to talk to / help you get by? did you feel at all isolated?)

  – Feelings about their situation, e.g. How does your situation make you feel / feel about your life? [Do not explicitly ask about mental health here unless raised spontaneously]

• Thinking more broadly:

  – Do you think having a job is good for your health and how are feeling? Why?

  – Do you think being out of work is ‘bad’ for health and how are feeling? Why?

3. Understanding and decision-making [15 minutes]

• Can you remember being told about something called the Group Work course? If necessary rephrase question and prompt them with when the records suggest they should have been informed about course. Probe to briefly establish:

  – Who they heard about it from and how it was introduced to them.

  – How it was described / what information was provided.

• What did you understand it was about? Probe to establish their understanding of:

  – Who it was target at / intended for.

  – What it would involve.

  – Why they thought they were being told to / given the opportunity to go on it.

  – What, if anything, they thought they could potentially get out of it.

• Why did you decide not to go on the course? Allow for spontaneous responses then if necessary probe for role of:

  – Not feeling it was intended for people like them.

  – Belief that they did not need help.

  – Belief that it wouldn’t help them with their specific needs /barriers to work.

  – Thinking they would not get anything out of it.

  – Concerns about being in a group environment.

  – Perceived practical or financial barriers (e.g. travel or childcare). If cited probe for their awareness of location of course and reimbursement of travel and childcare costs.

• Is there anything that would have changed your mind about going on the course? If so, What? How?

• What do you think about the information that was provided? E.g. in terms of the extent to which it:

  – Enabled them to make an informed decision about taking part.

• Are there any ways in which the information you were given about the course could be improved? E.g. in terms of:

  – The timing and amount of information provided.

  – What the information says about the course.

  – How the information is provided / delivered / communicated (inc. the language used).

  – Hearing feedback from others who have already been on the course (i.e. would this be useful?)

• Were you offered any other forms of support, help or training after you declined to take part in the Group Work course?
• And as far as you are aware, could you still go on the Group Work course?
• If so, would you consider going on it in the future? Why? Why not?

Close

• Thank them for sharing their experiences.
• Ask them if there is anything else they’d like to add.
• Confirm arrangements for sending them the incentive.

1. The RCT employed a Zelen design, detail of which is provided in the Technical Report on the Impacts of the Trial. ↩

2. Detail on the RCT design is provided in the Technical Report on the Impacts of the Trial ↩

3. A thematic analysis approach was used to analyse the compliance and Jobcentre lead interviews, rather than using NVIVO software, given the smaller sample size. ↩

4. In some cases, courses were attended for two or three days, when logistical reasons meant attending a full week was not possible. ↩

5. Throughout the follow-up period every attempt was made to remain in contact with the initial sample of 72 participants, who were spread evenly across each district, but many participants were unreachable after two to three months or would only be contactable at particular time intervals throughout the following year. ↩

6. Active Labour Market Policies (ALMPs) aim to increase the employment opportunities for job seekers and improve matching between jobs (vacancies) and workers (i.e. the unemployed). In so doing ALMPs may contribute to reducing unemployment and benefit receipt via increased rates of employment and economic growth. ↩

7. See The employment of disabled people 2019: Tables 3.2. and 3.3 ↩

8. See Employment and Support Allowance Caseload: DWP Stats Xplore, November 2019 [https://stat-xplore.dwp.gov.uk/webapi/jsf/dataCatalogueExplorer.xhtml] ↩

9. Good quality work is characterised by features including job security; adequate pay for a healthy life; strong working relationships and social support; promotion of health, safety and psychosocial wellbeing; support for employee voice and representation; inclusion of varied and interesting work; a fair workplace; promotion of learning development and skills use; a good effort–reward balance; support for autonomy, control and task discretion; and good work–life balance. Poor quality or ‘bad’ work is essentially work with the opposite of these features. See Marmot Commission (2020) Health Equity in England: The Marmot Review Ten Years On (page 61) ↩

10. Active elements are the components and features embedded within an intervention or course, such as the nature of the learning materials, the quality of the facilitators and whether a course offers or facilitates social support and interaction with other participants. These combine with the background social, economic and psychosocial characteristics of participants which may result in changes in their health, wellbeing and job search behaviour. ↩

11. OECD (2013), Active Labour Market Programmes. Glossary of Statistical terms ↩

12. To note that even when statistically significant, employment impacts are often quite small, for example in region of five to seven percentage points. ↩

13. See evidence from Germany (Crost, 2016; Wulfgramm, 2014), Spain (Ayala and Rodríguez, 2013), Serbia (Bonin and Rinne, 2014), the USA (Vinokur et al., 2000) and Finland (Vuori and Vinokur, 2005) ↩

14. This refers to the theories and empirical evidence from Jahoda (1982), Warr (1999) and Fryer (1986), along with the research of Bandura (1986), Berkman (1979) and Cacioppo et al. (2011) ↩

15. Psychosocial indicators concern psychological and social factors that can influence health and wellbeing outcomes. Typical examples of such indicators include social support, employment status, job quality, poverty and marital status. ↩

16. See Table 2.1 for details of each study and comparison to Group Work trial in the UK. ↩

17. The mastery outcome was a composite measure taking into account scores on job search self-efficacy, self-esteem and locus of control indexes. It was designed to be a measure of someone’s emotional and practical ability to cope and take on particular situations. See Pearlin et al., 1981. ↩

18. Job stability was based on the respondents’ report of whether their new job was a stable job or just a temporary job. In the Finnish labour markets, a stable job is generally desirable and secure and has a clearly higher status compared with temporary jobs. Termination of a stable job is difficult because the law heavily restricts the reasons for layoffs. Also, layoff time in stable jobs varies from one month up to six months depending on time of employment. ↩

19. The survey and the randomisation process were specific to the trial and would not be included in any wider roll out. Consequently, they are not explored in detail, although instances where they were felt to have influenced the process are noted. ↩

20. Refresher Training slidepack, DWP 2017 ↩

21. Benefit claimants did not have to consent to complete the survey for randomisation to be applied but 95 per cent gave this consent, meaning the completion of the survey formed part of the recognition and handover process in most cases in the trial. ↩

22. The survey collected data on range of measures including: the claimant’s confidence in getting a job in the next 13 weeks; whether they agree their personal qualities will help them get a job and that their experience is in demand in the labour market; their current barriers to securing and keeping a job; and the ONS-4 wellbeing questions (including their satisfaction with their life nowadays and level of anxiety the previous day). See Chapter 3 of the Technical Report on the Impacts of the Trial for a description of the measures used. ↩

23. All benefit claimants who attended day one of the course were classed as “participants” in the trial. ↩

24. Footnote removed. ↩

25. A statistic derived from a study, such as the difference between two groups, is said to be statistically significant if the size of that statistic has only a low probability of arising by chance alone. The probability of a statistic of that size occurring by chance alone is termed the ‘p-value’. By convention, if the p-value is less than 0.05 then it is stated that the statistic is ‘significant’. ↩

26. A statistic derived from a study, such as the difference between two groups, is said to be statistically significant if the size of that statistic has only a low probability of arising by chance alone. The probability of a statistic of that size occurring by chance alone is termed the ‘p-value’. By convention, if the p-value is less than 0.05 then it is stated that the statistic is ‘significant’. ↩

27. To note that satisfaction with support from Jobcentre Plus was lower among people with common mental disorders than among those without, although a symptom of common mental health disorders is to tend towards more negative views – see McManus (2012). ↩

28. An overview of the measures used can be found in Chapter 1 of this report and additional detail available in chapter three of the Technical Report on the Impacts of the Trial. ↩

29. Namely those with GAD-7 scores of 5 and above. ↩

30. Those scoring 0 to 8 on the WHO-5 scale. ↩

31. See Petticrew et al. (2013) ↩

32. See Bonell et al. (2012). ↩

33. See Petticrew et al. (2013) ↩

34. Psychosocial indicators concern psychological and social factors that can influence health and wellbeing outcomes. Typical examples of such indicators include social support, employment status, job quality, poverty and marital status. ↩

35. Latent and Manifest Benefits (LAMB) are material and psychosocial benefits associated with being in work such as social interaction, social support, activity, identity, collective purpose, self-worth (Latent benefits) and income (Manifest). ↩

36. Twenty-two per cent of those initially offered the course went on to attend it. ↩