Article Text
PDF
Abstract
Objective The evidence base for social prescribing is inconclusive, and evaluations have been criticised for lacking rigour. This realist review sought to understand how and why social prescribing evaluations work or do not work. Findings from this review will contribute to the development of an evidence-based evaluation framework and reporting standards for social prescribing.
Design A realist review.
Data sources ASSIA, CINAHL, Embase, Medline, PsycINFO, PubMed, Scopus Online, Social Care Online, Web of Science and grey literature.
Eligibility criteria Documents reporting on social prescribing evaluations using any methods, published between 1998 and 2020 were included. Documents not reporting findings or lacking detail on methods for data collection and outcomes were excluded.
Analysis Included documents were segregated into subcases based on methodology. Data relating to context, mechanisms and outcomes and the programme theory were extracted and context-mechanism-outcome configurations were developed. Meta-inferences were drawn from all subcases to refine the programme theory.
Results 83 documents contributed to analysis. Generally, studies lacked in-depth descriptions of the methods and evaluation processes employed. A cyclical process of social prescribing evaluation was identified, involving preparation, conducting the study and interpretation. The analysis found that coproduction, alignment, research agency, sequential mixed-methods design and integration of findings all contributed to the development of an acceptable, high-quality social prescribing evaluation design. Context-mechanism-outcome configurations relating to these themes are reported.
Conclusions To develop the social prescribing evidence base and address gaps in our knowledge about the impact of social prescribing and how it works, evaluations must be high quality and acceptable to stakeholders. Development of an evaluation framework and reporting standards drawing on the findings of this realist review will support this aim.
PROSPERO registration number CRD42020183065.
- MENTAL HEALTH
- PRIMARY CARE
- PUBLIC HEALTH
- QUALITATIVE RESEARCH
- SOCIAL MEDICINE
- STATISTICS & RESEARCH METHODS
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- MENTAL HEALTH
- PRIMARY CARE
- PUBLIC HEALTH
- QUALITATIVE RESEARCH
- SOCIAL MEDICINE
- STATISTICS & RESEARCH METHODS
Strengths and limitations of this study
This is the first realist review of evaluation methodology, specifically in relation to social prescribing evaluation.
Applying a realist logic of enquiry allowed generation of a theory underpinning how and why social prescribing evaluations work.
Inclusion of published and grey literature granted the reviewers insight into different contexts within which social prescribing evaluations take place.
Descriptions of social prescribing evaluation methods and processes lacked detail of mechanisms, causality or decision-making processes, which would be useful to further refine the programme theory.
This realist review sits within the broader A soCial presCribing evaluatiOn fRamework and reporting standarDs study study to develop an evaluation framework and reporting standards, findings will be directly applied in practice.
Introduction
Attention on social prescribing is rapidly increasing. As a concept, its applications are broad, and it has been proposed as a solution to improve sustainability of general practice1; reduce health inequalities2; address the social determinants of health3; tackle loneliness and social isolation4; improve the health and well-being of citizens5 and support recovery from COVID-19.6 Given the breadth of its applications it is unsurprising that social prescribing services are highly heterogeneous, and the term is used to refer to a variety of models and activities.7 Aims of social prescribing reported in the literature are wide-ranging, including improved mental, physical and social well-being, optimised health service use and reduced health service costs.8 There is no agreed definition of social prescribing,9 but it is generally understood to involve referral to non-medical resources in the community, with the goal of improved health and well-being.10–12 This typically involves a link worker, also known as a community connector or navigator, who works with the individual to identify their needs, coproduce goals and connect them to resources in their community.13–15
In parts of the UK, the growing interest in social prescribing has been accompanied by substantial funding. The NHS Long Term Plan for England committed to placing 1000 social prescribing link workers in primary care networks by 2020/2021, benefitting 2.5 million people by 2023/2024, through 900 000 referrals.16 A further £5 million of funding for social prescribing has since been granted to support COVID-19 recovery.6 Social prescribing in the other devolved nations has not received the same NHS funding, although the Welsh and Scottish Governments have committed to developing a social prescribing offer.17 18 As such, their social prescribing models have been developed using a bottom-up approach within the community, where services and activities are predominantly designed and implemented by individual third sector organisations, without an overarching, national strategic model.11 19 20
Diverse social prescribing models and services have been evaluated using heterogeneous designs and methods. The application of these varying designs and methods has resulted in an inconsistent, inconclusive evidence base for social prescribing.1 15 Evaluations using qualitative and uncontrolled quantitative designs have reported improvements in health, well-being, social isolation and chronic health conditions.5 21–24 However, these findings have not been corroborated with studies employing controlled designs.25–28 Discrepancies in the evidence base have also been identified in mixed-methods studies9 and systematic reviews.29 Gaps in our understanding of the individual, community and system impact of social prescribing and the mechanisms through which social prescribing works, for whom and in what circumstances remain.7 30 Randomised controlled trials are considered the gold standard for generating evidence,31 however, their application in the context of social prescribing evaluation is contentious given the moral and ethical implications of denying access to services which may improve health and well-being.32 Instead, a coordinated, consistent framework for evaluation is required to produce comparable results which contribute to the social prescribing evidence base.1
To develop such a framework, we argue that it is important to understand the social prescribing evaluation literature to date. The present realist review seeks to provide insight into how and why social prescribing evaluations work, and identify good practice, and areas for improvement. By providing an understanding of the current state-of-play in social prescribing evaluation, it will inform the development of an evidence-based evaluation framework.
Realist review
A realist logic of enquiry, based on Realist philosophy of science, is a theory-driven approach which seeks to explore the interaction between context, mechanism and outcome.33 It asks the question, what works, for whom and in what context,34 going beyond attempts to understand whether something works, to identify mechanisms through which certain outcomes are generated, when triggered by a given context.35 A realist review, also known as a realist synthesis, applies the realist logic of enquiry to the secondary analysis and synthesis of primary research studies.36 37 A table with definitions of terminology used in this realist review can be found in table 1.
Realist glossary of terms
Framed as a new model for systematic review,37 the realist approach to synthesis has several benefits which make it an appropriate choice to explore the topic of social prescribing evaluation. The realist approach accepts complexity and provides a technique to understand complex interventions.38 Social prescribing is complex,13 as is its evaluation, given the use of many different approaches in different contexts. Previous systematic reviews of social prescribing evaluations have provided descriptions and critiques of the evidence base and evaluation approaches used,1 15 29 but have not gone into depth about how and why they work, or do not work. Of particular significance and benefit to the present review, is the breadth of document types and resources that can be drawn on in a realist review.36 39 Realist reviews reject the hierarchical approach for assessing research quality36 and accept a breadth of methodologies and approaches. Due to the community-based nature of social prescribing, and the aim of the review to understand the various contexts within which social prescribing evaluation occur, it was important to not limit included documents to the published literature.
Methods
The present realist review was conducted between April 2020 and June 2021 (online supplemental file 1). The protocol set out the planned steps for the synthesis, acknowledging that the process would be iteratively undertaken. As the review progressed and evolved, a number of changes were made to the protocol which we describe here. First, it became apparent that the scope and breadth of the five research questions initially set out in the protocol were too broad. Through progressive focusing,40 41 the review team agreed to narrow the scope to focus only on how and why social prescribing evaluations work. The intended duration of the Realist Review was 6 months, but given the complexity and depth of the topic, this was extended to 14 months. A final search of the literature was planned at the end of the synthesis process. Through discussions it was agreed to not complete this final search due to pragmatic limitations, and the extent of data saturation for each of the context-mechanism-outcome configurations (CMOCs) presented in the review. An additional review team member (MD) joined the review after publication of the protocol and contributed to data extraction and synthesis. Finally, as discussed in step 5, no documents were excluded on the basis of relevance or rigour, but appraisal was noted as a descriptive characteristic.
Supplemental material
An advisory group was convened with membership of social prescribing, evaluation and Realist experts and stakeholders, including members of the public. A wider social prescribing infrastructure group42 was also drawn on to support the development of the Realist Review design and comment on findings. These groups contributed to the development of the search strategy and commented on preliminary findings and CMOC development.
Six iterative steps were followed in the process of conducting this Realist Review. The design was informed by the steps set out by Pawson37 and supplemented by additional approaches taken in other realist reviews which provided further depth and information regarding searches, data extraction, analysis and synthesis.13 36 39 43–45 The RAMESES publication standards33 were used for reporting (see online supplemental file 2).
Supplemental material
Step 1: identifying the review questions
This realist review is embedded within the A soCial presCribing evaluatiOn fRamework and reporting standarDs study (ACCORD) study, which aims to develop an evaluation framework and reporting standards for social prescribing evaluation using realist and consensus methods. The review scope and purpose were guided by the aim of ACCORD, and therefore, aimed to address the following two questions: ‘How do social prescribing evaluations work?’ and ‘Why do social prescribing evaluations work?’.
Step 2: searching for studies
A formal search strategy was developed based on an initial, unstructured background search of the literature and discussions with social prescribing stakeholders. Exploration of possible substantive theory, including different evaluation methodologies and designs, was also undertaken. This informed development of the initial programme theory.
Nine online databases were searched for documents referring to social prescribing, community and evaluation, published between 1 January 1998 and 31 May 2020. A grey literature search was also undertaken in Wales for public evaluation documents and a document request was sent out through extant social prescribing networks. Details of the databases and search strategy can be found in online supplemental file 3.
Supplemental material
The formal published literature search yielded 2904 records and an additional 145 records were identified through the grey literature, a research network request and other sources. See figure 1 for a Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram detailing the search results.
PRISMA diagram of document selection. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Step 3: study selection
Documents included in the realist review were required to make some reference to the social prescribing/link worker process, but could focus on any component of the pathway. All evaluation and monitoring designs were included, but documents lacking description of evaluation design or not reporting findings (eg, protocols, editorials) were excluded.
Screening of titles and abstracts was undertaken by ME, with a random sample of 10% of citations reviewed by JD to check for consistency in application of the screening tool.13 Any disagreements were reviewed by CW and resolved through discussion.46 Following title and abstract screening, 160 full-text documents were screened for eligibility by ME, with 10% screened by CW. Disagreements were discussed and resolved within the team. As a result, 83 documents were included in the realist review.
Step 4: quality appraisal
All included documents were assessed for relevance to the initial programme theory and ability to contribute to CMOC. Documents were appraised and categorised as ‘high’ (n=16), ‘moderate’ (n=35) and ‘low’ (n=32) in usefulness and relevance. See table 2 for a description of the criteria. All documents were included in the review, regardless of their appraisal, as it was agreed that even documents with ‘low’ relevance may have the potential to contribute ‘nuggets’ of information.37 Documents were also appraised for rigour and trustworthiness of methods and quality of reporting. However, as this review focused on evaluation methods and designs, rather than evaluation findings, it was deemed inappropriate to exclude documents on the basis of low rigour, as these documents would still contribute to the programme theory, and the exploration of how social prescribing evaluations do and do not work.
Appraisal criteria for usefulness and relevance
Step 5: data extraction
Documents were split by methodology into four subcases for data extraction and management (figure 1, table 3); qualitative (n=21), quantitative (n=14), mixed-methods (n=38) and reviews (n=10). Data extraction was undertaken by ME, using a bespoke data extraction Excel file, which captured document characteristics and CMOCs and themes. Coding was inductive but guided by four questions which explored; whether the extracted data referred to a context, mechanism or outcome; whether a partial or complete CMOC could be identified; whether the data was relevant to social prescribing evaluation and the programme theory; and whether the data were sufficiently trustworthy and rigorous.43 As with screening, 10% of documents were reviewed and coded by CW. All preliminary CMOCs were coded and gathered under themes. If-then statements were developed for each CMOC to clarify the relationship between the three components, prior to data synthesis.
Summary of documents within each subcase
Step 6: data synthesis
Using the preliminary codes, CMOCs were reviewed and gathered into overarching themes for each subcase. A meta-matrix was used to identify common themes and codes across the four subcases. Using this, 77 codes were synthesised into 13 broader themes. These themes and corresponding preliminary CMOCs were mapped onto the initial programme theory. Diagrams were created and iteratively refined to depict our thinking and the contribution of different documents to different parts of the programme theory. The CMOCs and programme theory were iteratively refined through ongoing document analysis and discussions with the review team and advisory group.
Patient and public involvement
This realist review sits within the ACCORD study. The study was presented to the PRIME Centre Wales SUPER public and patient involvement group in its early phases of development. Comments from this group led to recruitment of two permanent patient and public involvement (PPI) representatives to the Wales School for Social Prescribing Research (WSSPR) steering group to specifically support the ACCORD study. An additional PPI representative joined the realist review advisory group and commented on ideas and findings.
Results
Document characteristics
Overall, 83 documents were included in this realist review (see figure 1).1 2 5 8 9 21–24 26–29 32 47–114 Documents were split by methodology into four subcases, with representation from both the published and grey literature, although the majority of grey literature documents employed mixed methods (table 3). Generally, studies lacked in-depth descriptions of the evaluation processes and methods. Most described evaluations of general, holistic social prescribing processes, including a link worker. Others included arts on prescription, nature-based interventions, welfare advice services, Time Credits programmes, Museum-for-Health programmes, National Exercise on Referral Services, Community Navigation programmes and nurse navigation. Documents were predominantly from the UK (England, n=44; Wales, n=26; Scotland, n=6; Northern Ireland, n=1), with few documents from Europe (n=4), Canada (n=1) and Australia (n=1). The formation of the research team varied between evaluations undertaken by independent teams, service providers and mixed-teams. A quarter of the documents provided no description of the composition of the research team. Online supplemental file 4 provides a table of studies included in the review and their characteristics.
Supplemental material
Main findings
The initial programme theory provided a linear explanation of social prescribing evaluation with no exploration of mechanisms (see figure 2). This provided a basis for exploring CMOCs which were identified through data extraction.
Initial programme theory for social prescribing evaluation.
When considering social prescribing evaluation as an intervention, identification of outcomes was challenging. Three outcomes were identified, first, that the social prescribing evaluation design was acceptable to all stakeholders. Second, that it was high quality, in that it employed rigorous evaluation techniques and was reported transparently. The final outcome was more distal; a nuanced understanding of the impact and effects of social prescribing. Through achievement of the first two outcomes, and the mechanisms discussed here, social prescribing evaluations extend our knowledge and understanding of the topic and identify areas for further research.
Data synthesis resulted in identification of five key themes which underpin our refined programme theory; coproduction, alignment, researcher agency, sequential design and integration.
Coproduction with mixed stakeholder teams
If social prescribing evaluations are coproduced by mixed-teams (C), then sharing of experiences, expertise and diverse perspectives (M), increases evaluation acceptability (o) and trustworthiness (o)
Twenty documents contributed to the development of this CMOC.5 9 23 24 32 52 54 55 57 67 68 71 76 78 82 85–87 95 102 In the early stages of the evaluation development, involvement of a breadth of stakeholders (eg, social prescribing practitioners, service providers, commissioners, community assets, individuals receiving social prescribing) facilitates the codevelopment of an acceptable and trustworthy evaluation design. Materials are coproduced, based on existing literature and experiences of stakeholders, who can then comment on acceptability of design features for prospective participants. Where these aspects are informed by the views of stakeholders, participant burden may be reduced, thus improving completion rates. Evaluations were frequently reliant on service providers for access to participants and data collection. Where those service providers were part of the research team, they held a sense of investment, and participant recruitment was more successful. While this does pose a risk of bias, randomised approaches to participant recruitment were not effective in yielding sufficient participant numbers. A balance must therefore be struck between data integrity and feasibility of recruitment strategy. Improved trustworthiness of the evaluation is also fostered through coproduction and sharing expertise and diverse perspectives. Reporting of public involvement in the included documents was sparse, with only 6 of the included 83 documents detailing their approach. However, those which did benefited from access to diverse perspectives, contextual information and insight. This was crucial in developing trusting relationships with the wider community who were subsequently more engaged with the research.
Alignment between the intervention and evaluation design
If evaluators have strong contextual knowledge about the intervention and its’ aims (C), then they can align the research question and design (M) to provide a coherent, cohesive evaluation (O).
Twenty-five documents provided evidence for this mechanism.8 9 22 27–29 32 52 55–57 60 62 64 65 67 68 71 76 81 87 105 110 113 In designing a social prescribing evaluation, the research team must develop a comprehensive understanding of the intervention and how it may be working. This may be achieved through stakeholder discussions, service mapping, service observation, applying a framework or developing an initial programme theory. This thorough knowledge about the intervention is used to inform the development of the research questions and evaluation design. By completing this step, the evaluation is poised to assess whether the intervention is achieving what it set out to. Where possible, corresponding validated tools can then be selected for data collection, although a lack of appropriate outcome tools for social prescribing evaluation was highlighted in multiple documents. Clear reporting and presentation of the alignment between intervention aims and context, evaluation aims, evaluation design and outcomes is critical for the evaluation user, allowing them to draw conclusions about the intervention and its impact. An important caveat to this mechanism is that evaluations should not be designed too narrowly, only focusing on the aims of the intervention, as this risks missing unanticipated benefits or outcomes which may arise. The benefit of mixed-methods designs which can capture outcomes aligned with the aims and undertake exploratory research is evident here.
Agency to make decisions
When there are predetermined aspects to an evaluation (C), the researcher does not have the freedom to make decisions regarding the execution of the study (M), which minimises the quality of the data and evaluation (O).
Fifteen documents contributed to this CMOC.2 26 27 32 51 53–56 60 62 68 82 88 97 Evaluations were rarely implemented alongside services and were more commonly commissioned and designed after service implementation. This often resulted in elements of the evaluation, for example, the outcome tools used, research questions or methodology, being pre-determined by service developers, commissioners or routine data monitoring systems. Lack of researcher agency during data collection was also common and negatively impacted on data quality, limiting insights and ability to draw conclusions. This was evident where data were collected by a third party, resulting in inconsistencies in time points when data were collected, incorrect completion of validated tools, incomplete datasets, insufficient data collected and self-reporting biases. Financial constraints and insufficient funding may be responsible for this lack of agency, impacting on researcher ability to collect data, use control groups, have sufficient follow-up periods and employ rigorous designs. We anticipate that the impact of funding on researcher agency and rigour is greater than that reported in the literature.
Use of a sequential, iterative design
If researchers use a mixed-methods sequential design for data collection (C), they can use existing data to inform subsequent design and data collection (M) to provide a nuanced, stronger understanding of the effects of social prescribing (O).
Thirteen documents provided evidence for this theme.22 23 32 50 55 57 60 68 81 95 105 107 108 Use of a sequential mixed-methods approach enabled researchers to use findings and insight from prior stages of the research to inform the design and development of subsequent stages. This was observed bidirectionally. Findings from quantitative components were used to inform the development of interview questions and areas of exploration in subsequent qualitative research. Datasets were used to develop purposive sampling strategies for qualitative research, including identification of different demographic groups and for individuals who responded differently to the social prescribing intervention. Exploratory qualitative research was used as a basis for designing quantitative research and selecting appropriate outcome tools. Qualitative observations were beneficial in identifying unanticipated benefits, particularly where these were not captured by selected outcome tools. This aided the researcher in developing a cumulative understanding of the social prescribing intervention and its effects.
Integration of findings to produce a full picture
This theme was heavily supported; forty-two documents contributed to its’ development and it is split into two CMOCs.2 5 9 21–27 32 52–54 56 60–62 66 68 69 71 73 76 79–81 85–87 95 100–103 105–108 110 113 114
When there are multiple sources of data (C), researchers can integrate and triangulate findings (M) to provide a nuanced, stronger understanding of the effects of social prescribing (O).
Social prescribing evaluations generate multiple sources of data. This includes data collected from different participant groups, using different methods and gathered at different time points. Triangulation of perspectives between different participants, particularly non-participant stakeholders, offers a more complete view of the broader impact of different dimensions of the intervention and the experiences of non-attenders, or hard to reach groups. A social prescribing evaluation does not sit in isolation, and the existing literature and previous research conducted about social prescribing must also be used for contextualising and explaining findings from their research, to contribute to the developing evidence base.
If qualitative and quantitative findings are reported separately (C), then there is a lack of integration (M), which results in a fragmented understanding of the effects of social prescribing (O).
Many of the documents included in the Review reported on single components of broader mixed-method, multicomponent studies. Despite this, findings and conclusions in different components of the same study were rarely integrated or triangulated. This lack of integration resulted in a fragmented, disjointed understanding of the intervention and its’ impact. Where studies are presented independently and not contextualised and integrated with existing knowledge, the evaluation user is unable to fully understand the intervention and unpick its inherent complexity. Studies which did successfully integrate their findings, either in the reporting of their results or in an overall interpretative analysis section provided the reader with an overarching understanding of the impact of social prescribing and a more nuanced understanding of the effects. Where possible, researchers should provide a commentary on the overall findings drawn from integrated mixed-methods research.
Development of the refined programme theory
The initial programme theory (figure 2) presented a logic model on which contexts, mechanisms and outcomes were placed as they were extracted from the literature. Initially, a linear relationship was proposed between the three identified components of social prescribing evaluation: preparation (1), conducting the study (2) and interpretation (3). During the interpretation component (3), identification of new research questions and proposals for future research occur. We, therefore, propose a cyclical relationship between the three components, although acknowledge this may not consistently occur (represented by the dotted line). The five identified themes discussed above and their corresponding CMOCs relate to each of these components. Elements of the overarching context within which the evaluation takes place; for example, funding, stakeholder involvement, service status, contextual knowledge, theoretical stance and the target population, were also considered relevant for inclusion in the refined programme theory.
The refined programme theory sought to represent the interplay between the overarching contexts, the themes and corresponding CMOCs in generating the outcome of an acceptable, high quality social prescribing evaluation, within the realm of the three components. The refined programme theory for social prescribing evaluation can be found in figure 3.
Refined programme theory for social prescribing evaluation.
Discussion
The present realist review sought to understand how and why social prescribing evaluations work. It included 83 social prescribing evaluation documents sourced from the international published literature and grey literature in Wales. A range of evaluation approaches and methodologies were employed, but documents lacked in-depth detail and descriptions of these approaches. Systematic reviews of social prescribing have also emphasised the poor reporting of their evaluations.29 Five themes were identified, with corresponding CMOCs through which the social prescribing evaluation worked to deliver an acceptable and high-quality evaluation.
The value of stakeholder involvement from the outset of the evaluation was evident, it yielded a sense of investment, offered insight and contextual knowledge and improved acceptability of the design through coproduction. Chatterjee et al55 also highlighted the benefit of stakeholder involvement, in integrating the views and perspectives of diverse groups and understanding their expectations. Utilisation-focused evaluation115 is evaluation undertaken with the intended users at the forefront. It posits that evaluations will be more useful and effective if intended users have a sense of ownership over the evaluation. The utility and design of the evaluation is therefore constantly informed and guided by the stakeholders. The lack of PPI in the included documents was surprising. Social prescribing is a person-centred intervention,116 and this should be reflected in the design of its evaluation. PPI is widely advocated for in research and its benefits are well known117 and were evidenced in the studies which involved the public in this Review. The UK Standards for Public Involvement118 provide guidance on good practice and must be followed to garner effective social prescribing evaluations.
Mixed-methods approaches were optimal for gaining a nuanced, in-depth understanding of the social prescribing intervention under evaluation, particularly when used sequentially and findings were integrated. Often this integration was missing from the evaluation documents, resulting in a partial view of how services were working.56 Even where each component of the mixed-methods study was reported separately, the depth and nuances were lacking. Going forward, evaluations must report on the integration of different study components and the relationship between their findings and the existing literature. This will result in cumulative development of the evidence base, minimising duplication and contributing to a cohesive understanding of social prescribing.
Given the inconsistencies in the evidence base, researchers have called for a coordinated framework for social prescribing evaluation.1 15 29 The refined programme theory presented here offers principles for good practice in social prescribing evaluation. These provided the foundation for the development of a series of evidence-based recommendations for social prescribing evaluation (box 1). These recommendations will directly feed into the development of the evaluation framework for social prescribing through the ACCORD study. Provision of such a framework will be particularly valuable given the limited evaluation capacity in practice.14 119 It will provide clear guidance and support for conducting monitoring and collecting data, which can be used in subsequent evaluations, mitigating the effects of low researcher agency and control. Similarly, the need for reporting standards was made clear through this Review. The sparsity and lack of detail in reporting the methods, alignment and findings of social prescribing evaluations has been identified elsewhere.8 29
Recommendations for social prescribing evaluation
Apply a mixed-methods design to produce an evaluation which captures the impact of social prescribing at multiple levels.
Where possible, design social prescribing evaluations iteratively, so that each stage can build uponon the previous stage so knowledge can be accumulated and the evidence base can continue to grow.
Undertake a mapping exercise to identify all stakeholders for a social prescribing evaluation. Involve stakeholders from the outset to coproduce the study design and materials.
Involve stakeholders in the interpretation, analysis and dissemination of findings so that the evaluation is grounded in the real world and findings can be translated back into practice, to make a difference to people involved in social prescribing.
Involve members of the public throughout social prescribing evaluation in a meaningful way. Follow the UK Standards for Public Involvement and report public involvement when disseminating findings.
Take time at the start of the evaluation, before the study design is determined, to understand the social prescribing intervention or service that is going to be evaluated. Identify the aims, objectives, participants, context, setting, activities, processes that are involved.
Align the evaluation with the social prescribing intervention so that the evaluation can answer questions that are relevant to the intervention and to stakeholders.
Seek advice or use an evaluation framework to inform evaluation decision making. This will maximise data quality, and ensure a consistent approach which can be compared with other similar evaluations.
For rigorous evaluations of social prescribing, remove the burden on link workers and use independent researchers to collect data at the appropriate time point.
Provide sufficient funding for social prescribing evaluations, to ensure that they can be undertaken rigorously, without bias, to address gaps identified by services or in the literature.
Integrate mixed-methods findings to generate a more in-depth, nuanced understanding of social prescribing, how it works, for whom and in what context.
Triangulate findings from multiple data sources and different perspectives to generate a more in-depth, nuanced understanding of social prescribing, how it works, for whom and in what context.
When using mixed-methods or conducting a multicomponent study, produce an overarching commentary or narrative, explaining the links between the different components and identifying remaining gaps for future research.
Provide in-depth descriptions of methods used and decisions made to facilitate judgements about the rigour and quality of the study, and to enable the study to be replicated in different contexts.
Report good practice, strengths, successes, failed approaches and methods to mitigate challenges in social prescribing evaluation to support future evaluators.
Finally, the need for sufficient funding and investment in social prescribing evaluation must be addressed. Evaluations to date have been criticised for lacking rigour and having a high risk of bias.1 15 An evaluation framework will only be useful if it is accompanied with funding to undertake high-quality, acceptable evaluations of social prescribing. Some evaluations included in this review alluded to the negative impact of limited funding, but the impact is anticipated to be much larger. Future research needs to explore the funding requirements for social prescribing evaluation and monitoring, and assess how this may change over time, as the evidence base for social prescribing develops, and the needs and priorities that it seeks to address change.
Strengths, limitations and future research directions
A strength of this realist review is its application of a realist logic of enquiry to a novel area; social prescribing evaluation. To our knowledge, this is the first realist review in this area, and the first exploring evaluation overall. Previous systematic reviews had provided descriptive commentaries about the social prescribing evidence base and evaluations to date.1 29 55 They critiqued the methods employed and highlighted low rigour and a high risk of bias. However, they did not seek to explore the reasons as to why this may have occurred, and explain the weaknesses in the evidence base, and what can be done to develop successful social prescribing evaluations. This review addresses this knowledge gap and highlights mechanisms through which evaluations may be acceptable, high quality and produce a nuanced understanding of social prescribing. A series of recommendations (box 1) for social prescribing evaluation have been generated based on the programme theory from this realist review, which will be useful for people conducting evaluations of social prescribing across the spectrum.
Another strength of this review is its placement within the ACCORD study. The findings from the realist review will be used in conjunction with two consensus studies, using Group Concept Mapping120 and a world café approach121 to explore social prescribing evaluation. Taken together, these studies will inform the development of an evidence-based, evaluation framework, reporting standards and training materials for people undertaking social prescribing evaluations. Direct application of the findings and their relevance to these outputs, which will be widely disseminated, fits with the translational model of research.122 It means that findings will be directly relevant and have a direct impact on the progress of social prescribing evaluations in the future.
As previously mentioned, the documents included in this realist review generally lacked in-depth information regarding the methods, design and processes used for their evaluations. Evidence syntheses are reliant on secondary data, and how findings are reported by authors.123 This proved challenging for this review, as documents rarely provided in-depth explanations of the mechanisms, causality or decision-making processes, which could contribute to CMOC. An example of this is the lack of information about how social prescribing evaluations were funded and the funding allocated to them. Funding is an important contextual factor, which will likely impact on how the rest of the evaluation is able to be undertaken. However, where studies lacked information about the funding, it was not possible to understand the full impact, and the mechanisms through which this may have impacted the outcomes. This highlights a clear need for transparent reporting and reporting standards for social prescribing evaluations so that evaluation users have access to the necessary information to make their own judgements about the quality and rigour of the evaluation.
Finally, the grey literature search was limited to documents from Wales due to differences in the models of social prescribing between Wales and other UK nations39 124 and differing models of health and social care due to devolution.125 Expansion of the grey literature search across the UK and/or internationally may have yielded more relevant documents which could have supported CMOC development.
Conclusions and recommendations
To our knowledge, this is the first attempt to apply a Realist logic of enquiry to the issue of evaluation, particularly in the context of social prescribing. This realist review offers insight into the current status of social prescribing evaluation, it identifies how and why social prescribing works, barriers to its success and examples of good practice. The review also clearly highlights the importance of a standardised evaluation framework and reporting standards for social prescribing going forward. A series of recommendations have been developed based on the findings, which will feed directly into the ACCORD study and are useful for practice and research in the undertaking of future social prescribing evaluations. The next stage of this programme of work is to develop and test an evidence-based evaluation framework and reporting standards for social prescribing, using the evidence from this review and consensus research.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
Ethics statements
Patient consent for publication
Acknowledgments
The authors would like to thank the Wales School for Social Prescribing Research steering group members and the members of the Realist Review advisory group; Lyndsey Campbell-Williams, Mair Edwards, David Humphreys, Prof Mark Llewellyn, Dr Mary Lynch, Dr Sally Rees, Dr Glynne Roberts, Andrew Rogers, Roger Seddon, Sara Thomas, Dr Josep Vidal-Alaball.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @MeganBElliott_
Contributors ME prepared the initial protocol, developed the search strategy, facilitated the advisory group, undertook the main searches and document screening at title, abstract and full-text level, carried out the coding and development of CMOCs and refined programme theory and prepared the final report. ME prepared the full manuscript. MD supported development of CMOCs and the refined programme theory, contributed to the interpretation of findings and revised the final report. MD reviewed and commented on the manuscript.JD contributed to the formal search strategies, carried out consistency checks on documents in screening and provided practice expertise and perspective. JD reviewed and commented on the manuscript. CW was the principal investigator and developed the research project. CW contributed to the development of the protocol and search strategy, carried out consistency checks on document screening and coding, developed and refined the programme theory and CMOCs and revised the final report. CW reviewed and commented on the manuscript. CW takes full responsibility for the overall content as the guarantor.
Funding This work was supported by Health and Care Research Wales Infrastructure funding for PRIME Centre Wales and the Wales School for Social Prescribing Research, from April 2020 to March 2023.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.