Article Text
PDF
Abstract
Introduction Healthcare is increasingly challenged to meet the demands of user involvement and knowledge mobilisation required by the 21st-century patient-centred and knowledge-based economies. Innovations are needed to reduce problematic barriers to knowledge exchange and improve collaborative problem solving. Living labs, as open knowledge systems, have the potential to address these gaps but are underexplored in healthcare.
Methods and analysis We will conduct the first systematic review of living labs across healthcare contexts. We will comprehensively search the following online databases from inception to 31 December 2020: Scopus, the Cochrane Library (Wiley), Medline (OVID), Embase (OVID), Web of Science, PsycINFO (OVID) and EBSCOhost databases including Academic Search Complete, Business Source Premier, Canadian Reference Centre, CINAHL, MasterFILE Premier, SPORTDiscus, Library & Information Science Source, Library, Information Science & Technology Abstracts, AgeLine, EconLit, Art Full Text, Women’s Studies International and Social Work Abstracts. We will search for grey literature using Google advanced techniques and books/book chapters through scholarly and bibliographical databases. We will use a dual-reviewer, two-step selection process with pre-established inclusion criteria and limit to English language publications. Empirical studies of any design examining living lab development, implementation or evaluation in health or healthcare will be included. We will use the Mixed Methods Appraisal Tool (MMAT) for methodological quality appraisal and Covidence software for review management, and we will extract data on pre-established variables such as lab context and technological platforms. We will create evidence tables and analyse across variables such as focal aim and achievement of living lab principles, such as the use of cocreation and multimethod approaches. We will tabulate data for descriptive reporting and narrative synthesis to identify current applications, approaches and promising areas for living lab development across health contexts.
Ethics and dissemination Ethical approval was not required for this review. This review will inform research into living labs in health environments, including guidance for a living lab in paediatric rehabilitation. Academic publications shared through collaborative networks and social media channels will provide substantive knowledge to the growing tech-health development sector and to researchers, practitioners and organisations seeking enhanced patient/stakeholder engagement and innovations in knowledge translation and evidence-based practice.
PROSPERO registration number CRD42020175275
- information technology
- health services administration & management
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This will be the first systematic review on living labs in healthcare and will incorporate a range of published and grey literature sources.
A dual-reviewer process will be used throughout to reduce risk of bias and increase likelihood of comprehensive study identification and inclusion.
Non-English literature will be excluded, which may under-represent the extent of living lab development and use across geographically diverse health contexts.
Introduction
First introduced in the early 2000s as a means to catalyse end-user engagement with innovation, living labs emerged as user-focused, open data gathering, sharing and exchanging platforms intended to facilitate collaborative problem solving in real-life settings.1 2 Despite widespread growth of living labs in urban planning and civil design, living labs have received comparatively little attention in healthcare.3 Living labs hold great potential as open knowledge exchange platforms in health settings3 creating opportunities to improve participant engagement in research and healthcare.1 4 Although research is emerging on living labs in healthcare, no systematic review (SR) exists on this topic. We attend to this gap by proposing the first SR on living labs in healthcare.
Innovation, engagement and knowledge translation
Innovation in healthcare practice and research increasingly involves the collective collaboration of multiple stakeholders.5 This sentiment is reflected in abundant practices and associated terminology, both emergent and established. Terms such as codesign, experience-based codesign, user-centred design, design thinking, patient-oriented research and integrated knowledge translation (iKT), for example, reflect at a minimum a commitment to participant/end-user engagement and concurrent recognition of the value of incorporating multiple perspectives through dynamic knowledge exchange processes (ie, interactive information sharing and problem solving occurring between researchers and knowledge users).6 Among these, knowledge translation (KT) and patient engagement are predominant narratives of contemporary western healthcare and health research; their value is affirmed through funding agency requirements and the continued emergence of ‘new’ communicative and collaborative approaches (eg, arts-based KT) appraised as more innovative, more engaging and more reflective of end-user perspectives of healthcare.7 Concurrent recognition of the value of end-user experiences, knowledge and expertise and knowledge that research evidence produced through patient-engaged strategies may be more relevant to knowledge users, better aligned with stakeholder priorities and more likely to improve the patient experience of healthcare further merit stakeholder involvement.8–10 These recognitions have also prompted attention towards the processes, systems and structures in place within healthcare to achieve such objectives.
Indeed, despite knowledge of the critical importance of KT and patient engagement, approaches to patient engagement have not uniformly followed suit, often tokenising involvement and expending extensive resources in the process.8 Similarly, approaches to KT have inconsistently achieved their desired results, with research-centric agendas, an emphasis on rationality and linearity in decision-making and a lack of sensitivity to local practice context among the many factors cited as barriers to this attainment.7 10–12 Identifying alternative mechanisms and systems to support end-user engagement and knowledge exchange within health systems is needed. Such a paradigm, capable of entwining patient engagement with KT and embedding it within healthcare settings, could enable the living generation, exchange and mobilisation of research evidence sensitive to local context. One such approach that has historically been underexplored in healthcare but is pertinent to the collaborative agenda of present-day health research and care practices is that of the living lab.
Living labs
Since first documented in the literature in the early 2000s, living labs have been emerging as highly promising, user-centred, open innovation systems that integrate research cocreation and knowledge exchange in real-life settings.4 13 14 Living labs involve a designated space (ie, virtual or physical), generally with the aim of leveraging stakeholder collaboration and shared ideation in order to solve social problems.15 While often created to foster collaborative innovation and bidirectional information sharing, organisationally embedded living labs can inform problem identification, data gathering, creative data sharing and rapid prototyping of concepts towards sustainable knowledge sharing systems.1–3 13 14 Despite the myriad of possibilities for living labs in catalysing innovation and their global emergence over the past 20 or so years, research into living labs was still classified as nascent as recently as 2015, and in 2016, research into living labs was classified as ‘insignificant’ in relation to quality.16 Among the calls for further research development in the field has been the need for more comprehensive reviews of existing literature.15 17 18 In this regard, no SR has been conducted on the development and use of living labs within health contexts.
However, previous reviews on living labs in other sectors provide insight into their theoretical foundations, methods, aims, domains of applications and outcomes,19 as well as the general state of the theoretical and practical domains of the field. Folstad and colleagues19 conducted what appears to be the earliest review on this topic, specific to information communication technology. Reviews have explored the evolution of the living lab concept20 and in reference to ‘grassroots’ innovation.21 Two SRs of living lab literature not specific to health15 16 were also identified. These reviews contribute to an understanding of living lab principles and illuminate useful considerations pertaining to sustainability and governance, for example, without meaningful engagement with health-related concepts. Specifically, Hossain and colleagues15 identify technological infrastructure, a stakeholder infrastructure, community involvement and end-user community positioning as principle characteristics of living labs. Similarly, the living lab methodology handbook identifies multimethod approaches, user-engagement, multiple stakeholders, real-life settings and cocreation of an environment for innovation as the five key elements of the living lab approach.2 With an emerging global presence with predominant European concentration and with inherent inclusion of multiple disciplines and stakeholders, living labs hold promise in multiple sectors beyond sustainability and development wherein they are more commonly employed. Within the health context, an integrative review of living labs in health3 was located, which identified some theoretical components of living labs pertinent to inpatient and community-based health services. However, this review was limited by the exclusion of grey literature and a lack of reporting on the date of search, both critical components for inclusion in a review of a rapidly growing field of study.3 It is clear that despite burgeoning interest in living labs for innovation, living labs remain underexplored and underdeveloped within health systems, and there appears to be an overemphasis on user roles as ‘testers’ of a prototype or service, rather than alternative roles, such as developers or designers of innovation.5 As such, the valuable cross-sectoral learning theoretically possible through living lab integration within the health sector has not yet been realised or fully explored.3 4 13 14 Establishing an understanding of how living labs have been developed and used in healthcare environments is a critical first step towards maximising the potential of living labs in healthcare. The proposed study will respond to this gap by conducting the first SR on living labs in healthcare environments internationally, in order to identify promising applications and catalyse an emerging research programme exploring novel knowledge generation and exchange using living labs in healthcare as well as supporting the development, implementation and evaluation of living labs across receptive health contexts. In addition, this SR will provide specific guidance for an innovative knowledge mobilisation project leveraging identified living lab principles and methods a clinical paediatric rehabilitation environment, led by the lead author (MA).22
Study purpose and objectives
The objective for this SR is to systematically locate, identify and synthesise existing literature on living labs within health contexts. This project will assess and report on empirical studies describing the development, implementation and/or evaluation of living labs in healthcare environments irrespective of setting (eg, community, clinical and administrative) to identify current aims, processes, methods, frameworks, tools, user roles and disciplinary and geographical domains of application, as well as to document existing knowledge gaps. To this end, this SR will (1) identify, assess and report on empirical studies of any design that describe the development, implementation and/or evaluation of living labs in healthcare environments; (2) identify current uses, purposes and methods in the development of living labs; (3) describe the characteristics of living labs in relation to existing frameworks and typologies, including those sensitive to theory and user involvement; (4) identify factors related to living lab implementation and sustainability; (5) report on the effectiveness, meaningfulness and appropriateness of living labs for specific healthcare outcomes and populations; (6) identify strengths and shortcomings in current applications and empirical literature; and (7) identify strategies to facilitate the future development and implementation of living labs in healthcare. In accordance with these objectives, our predominant research questions are as follows: (1) What are the uses, purposes and methods of living labs in health contexts? (2) What are the characteristics of living labs as executed within health contexts, in relation to core characteristics, tools, frameworks and user typologies? (3) What are the factors related to living lab implementation and sustainability across health contexts? (4) What are the reported outcomes of living labs related to feasibility, appropriateness or effectiveness across health contexts? (5) What is the effectiveness, meaningfulness and appropriateness of living labs for specific healthcare outcomes and populations? (6) What are some strategies to facilitate the development, implementation and evaluation of living labs in healthcare?
Methods
Methodology
We will conduct a SR guided by the rigorous methodological guidelines developed by Mays et al to support the synthesis of evidence from multiple evidence sources.23 The inclusion of multiple evidence sources (published and unpublished/grey) and methodologies (qualitative, quantitative and mixed methods) of any design (eg, exploratory, descriptive, experimental) is necessary to accommodate the nature of the research being conducted on living labs. Extant reviews confirm that literature on living labs extends beyond the use of randomised controlled trials (RCTs), controlled before and after; studies; and interrupted time series—study designs of frequent interest to researchers conducting SRs of effectiveness. Growing recognition of the multiple evidence sources used in healthcare decision-making and implementation supports the methodological principle of inclusivity guiding this review.24
Search strategy and information sources
We will consult a health research librarian (JL) to develop a strategy for information retrieval and who will carry out comprehensive searches of bibliographical databases to identify scholarly articles, books and book chapters and grey literature writing in the English language on this topic from inception to 31 December 2020 (online supplemental file 1). Restricting to English language evidence is informed by data demonstrating no empirical bias when this restriction is in place.25 Tailored keyword searches will be executed across the following biomedical and interdisciplinary databases: Scopus, Medline (OVID), Embase (OVID), Web of Science, PsycINFO (OVID) and EBSCOhost databases including Academic Search Complete, Business Source Premier, Canadian Reference Centre, CINAHL, MasterFILE Premier, SPORTDiscus, Library & Information Science Source, Library, Information Science & Technology Abstracts, AgeLine, EconLit, Art Full Text, Women’s Studies International and Social Work Abstracts. Grey literature will be searched using keywords pertinent to living labs and health as identified in consultation with the research librarian and using Google advanced techniques until data saturation is reached. Reference lists will be mined to ensure pertinent articles are identified.
Supplemental material
Inclusion and exclusion criteria
Study design
We will include empirical articles, books and book chapters of any methodology (eg, qualitative, quantitative and mixed methods) and design (eg, descriptive, experimental). We will include published resources wherein the full text is available to us, pending a maximum of two requests made to corresponding authors. Conference proceedings reporting full or interim results pertinent to living labs in health will be included. If empirical data are not available, conference reports will be used to analyse trends in emerging literature versus existing works, citation and literature mapping and flagging for forthcoming publication of the work. Editorials, commentaries, opinion pieces, non-empirically based book chapters, review and methodological or theoretical articles will be excluded but will be used to identify original studies and situate the findings within the broader context of the field.
Participants and setting
We will include studies where participants are recipients or providers of healthcare or stakeholders or knowledge users identified as pertinent to the living lab in the primary study.
Intervention
Studies that examine or discuss the development, implementation and/or evaluation of living labs for any health-related purpose and in any population or context will be included. Empirical articles that do not specifically refer to or self-classify as living labs but that may demonstrate overlap in terms of innovative and user-centred principles will be excluded.
Outcomes
We will extract outcomes related to how living labs have been used and for which purposes within the context of health in relation to the development, evaluation and achievement of living lab principles—including outcomes pertinent to feasibility, appropriateness and effectiveness of living labs, in reference to their stated aims.
Data management
We will use Covidence review management software (Veritas Health Innovation, Melbourne, Australia) to manage all stages of the SR. Covidence is an internet-based platform reported to reduce SR times by 35%26 and improve adherence to SR methodology.27
Study selection
A blinded dual-reviewer, two-step selection process with pre-established inclusion criteria will be used to reduce risk of bias. During the primary screening, two trained reviewers will independently screen for relevancy of the titles and abstracts of all articles retrieved through database searches. Reviewers will include trained research assistants and research technicians, who will be blinded from each other’s decision for inclusion. Inclusion criteria for the primary screening will trend towards inclusivity; broad understandings of health and health contexts will be used and operationally defined, a priori. Each article will be classified as ‘relevant’, ‘irrelevant’ or ‘unclear’; articles designated as ‘unclear’ decision will be re-examined during secondary screening. Inter-rater reliability will be assessed using kappa coefficient, with a 0.80 coefficient recognised as appropriate. Inter-rater reliability will be checked after the first ten articles are screened in order to identify early challenges with the screening process and rechecked at the end of primary screening to support transparency and rigour of decision-making and reporting. For the secondary screening, two trained reviewers (eg, research assistants and research technicians) will independently review the full texts of articles previously identified as ‘relevant’ or ‘unclear’ against pre-established inclusion criteria (online supplemental file 2), delineated within the Covidence software in a series of yes/no criteria. Discrepancies will be resolved through consultation with a third reviewer (lead author, MA), who will be blinded from which reviewer did not vote ‘relevant’ to reduce bias. If the full texts of potentially relevant studies are not available, we will contact the listed corresponding author twice with a wait period of 1 week between contacts. If this approach is unsuccessful, the article will be excluded (with justification noted) from the review.
Supplemental material
Data extraction
We will use a two-step, two-reviewer hybrid process, wherein a reviewer will independently extract data of included studies and a second reviewer will verify all extraction data points. Reviewers will extract only those data reported in the studies directly into data extraction forms managed in Covidence. The forms will be pilot tested on a random sample of five studies and revised if necessary. Discrepancies in data extraction will be resolved through consultation with a third reviewer (lead author, MA). We will extract from each article the following information: title; authors (first and corresponding); author contact details; year of publication; journal, website, publication or conference title; study title; study purpose; study design; population; predominant health disciplines involved; methods and achievement of living lab principles (eg, cocreation, multimethod approaches, user engagement, multistakeholder perspective and real-life setting application); theoretical framework; technological platform; user roles; tools supporting innovation; contextual attributes; ethical considerations; outcomes assessed (including process outcomes); main findings with themes and subthemes relevant to our review questions; significance, non-significance or unclear significance for main outcome variables; reference to meta-inferences from integration when applicable; and sustainability. Multiple reports from a single study will be collated as a single resource in the SR. Corresponding authors will be contacted in the event of inconsistent or incomplete reporting of findings. Data extraction will commence concurrently with study selection after agreement on each resource (eg, article and chapter) has been attained (table 1)
Summary of review timeline
Methodological quality appraisal
We will use the MMAT V.201828 (online supplemental file 3) to appraise methodological quality of the included studies within Covidence, using the hybrid process described above. As this SR will be inclusive of qualitative, quantitative and mixed method studies, the MMAT was selected given that it is a single tool that can be used to appraise the methodological quality of different study designs.29 30 This tool emerged out of a need to mitigate against the time and logistical challenges of locating and using multiple appraisal tools for different types of studies when conducting a SR inclusive of multiple study designs.29 30 A user guide is included with the tool, which aids in consistent quality appraisal between multiple reviewers.30 In addition, the MMAT provides specific criteria for mixed method studies, which are often not found in other tools.29 The MMAT includes criteria of five categories of study designs that are the most commonly found in mixed method SRs29 30: (I) qualitative, (II) RCT, (III) non-randomised, (IV) quantitative descriptive and (V) mixed method studies. The MMAT focuses on five core relevant methodological criteria for each study design, which are rated on a scale of ‘yes’, ‘no’ and ‘can’t tell’, in order to provide time-effective and consistent appraisals.29 30 To date, there have been three versions of the MMAT, including 2006, 2011 and 2018, respectively. Revisions to the tool have focused on improving content validity29 30 with the 2018 revision specifically focusing on improving the content validity for the qualitative, survey and mixed method criteria.30 Validation studies have demonstrated sensitivity to the unique methodological characteristics of diverse qualitative, quantitative and mixed method study designs,31 which is critical to a SR of the proposed breadth. Any discrepancies in assessing methodological quality using the MMAT will be resolved through consultation with the lead author (MA). Given the emerging nature of this field, studies will not be excluded on the basis of methodological quality, to ensure a rich and inclusive representation of activity in this field.31 Data on methodological quality will be reported to provide context to the body of literature identified.
Supplemental material
Data analysis and synthesis
Previous reviews on living labs suggest extant research will be predominantly qualitative or descriptive in nature.15 16 Given this, and the extent of expected heterogeneity, we do not anticipate conducting a meta-analysis or risk-of-bias assessment as would be expected with a review of RCTs, for example, or a formal evaluation of the quality of evidence using Grading of Recommendations, Assessment, Development and Evaluations. We will construct evidence tables to facilitate the categorisation and analysis of domain information across core extraction variables. We will use narrative synthesis23 to explore relationships within the data, and when necessary, a combination of inductive thematic analysis to identify categorical domains not reflected in existing living lab literature and deductive content analysis to code narrative data and enable descriptive statistical analysis. For quantitative studies, we will classify main and process outcomes as significant, non-significant or unclear. Reported theoretical frameworks will be examined comparatively in relation to operationalisation of living lab principles and achievement of intended outcomes. We will conduct descriptive and narrative subgroup comparisons to consider differences in innovation tools, user roles and other living lab principles in reference to populations, context, identified outcomes and attainment of outcomes. The lead author (MA) will direct this analytical process, which will also involve research assistants and technicians as well as discussion with collaborating researchers, to ensure comprehensive and accurate interpretation of evidence tables by which narrative summaries and recommendations will be built.
Patient and public involvement
Patients will not be involved in the design, conduct, reporting or dissemination of the SR. This decision was made given the nature of this review as highly complex and broad; the purpose of the review, which was to provide an overview of living lab research in healthcare; and that resources for this project rendered participatory approaches unfeasible. However, opportunities for patient and public involvement will be more fully realised when the SR findings are used to operationalise a clinical living lab in paediatric rehabilitation and members of our family coresearch and parent advisory group will have opportunities to confer on the operationalisation of the SR findings.
Registration and reporting
This protocol was submitted for registration with the International Prospective Register of Systematic Reviews (CRD42020175275) on 19 March 2020 (table 1). We used the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols while preparing this protocol (online supplemental file 4).
Supplemental material
Ethics and dissemination
No ethical review was required. We will disseminate findings from the SR findings through summary reports and findings publication(s), presenting our findings at conferences to facilitate research reach, listing of project description and associated outputs on investigator-affiliated websites and professional web-based accounts, concurrent dissemination through professional social media accounts and clinical and parent/family advisory coresearch and collaborative channels established for the operationalisation of the SR findings through the clinical living lab project.22 The proposed review has the potential to inform user-centred innovation through living lab methodology in health contexts worldwide, by highlighting current practices and opportunities for further development. Findings will inform the development of future projects with the lead author’s research programme involving developing, implementing and evaluating living labs and associated projects in various receptive health contexts. Findings will directly inform a concurrent, independently funded project by lead investigator MA (Archibald M, unpublished data, March 2020) focused on the codesign of a living laboratory in paediatric rehabilitation, wherein stakeholders (eg, families, health providers, administrators and researchers) will engage in a robust collaborative and coresearch process and operationalise the SR findings.22
Discussion
Existing SRs on living labs are not specific to health and do not engage with health-related concepts or the diverse range of possible applications in any significant manner. Existing literature reviews on living labs in healthcare demonstrate design limitations including shortcomings in reporting and comprehensiveness that limit the utility and trustworthiness of findings. In addition to its rigorous and transparent methodology and unique knowledge contribution, this review will depart from existing SRs in its warranted health focus, which was identified as an underexplored yet promising domain for collaborative, multistakeholder and innovative knowledge mobilisation innovations. Indeed, while we expect that study heterogeneity and a predominance of qualitative and descriptive research will prevent meta-analysis but enable tabular descriptions and thematic analysis, as the first SR on this topic, this work will provide the basis for informing subsequent user-centred innovation in healthcare using living lab methodology. Given the overlap between the principles of living labs and other concepts and practices incorporating user involvement, collaboration and knowledge exchange (eg, codesign and iKT), researchers and practitioners working in these areas may also benefit from the forthcoming SR findings.
Numerous concurrent influences, fields of study and implementation considerations in healthcare highlight the potential timeliness of the living lab approach. A sustained interest in patient-centred and family-centred care, a user-centred healthcare economy and growing emphasis on transdisciplinary and cross-sectorial collaboration32 33; a movement towards cutting-edge technological integrations (eg, artificial intelligence and machine learning)34 35; and long-standing challenges in ensuring the timely mobilisation of research evidence (including evidence of lived experience) into health contexts36 37 including related issues with the sustainability and context specificity of such approaches suggest that new conceptualisations and approaches are needed. Living labs, as a boundary-spanning innovation model with untapped potential in healthcare, demonstrate potential to support these interrelated aims. However, maximising their utility is contingent on a more thorough understanding of their principles as well as the lessons gained through living lab development and operationalisation across health contexts.
Acknowledgments
RW is supported by a Tier 1 Canadian Research Chair (CRC) in Child and Family Engagement in Health Research and Healthcare (CIHR—Canadian Research Chair—950-231845). We acknowledge the input of CK for his editorial support on this manuscript.
References
Footnotes
Twitter @Mandy_Archibald
Contributors MA conceptualised the study, including leading the associated grant application and drafting the study protocol. KR, RW, JL and SW contributed to the protocol and will be involved in conducting the proposed research. JL provided guidance on the search strategy. SW is the project coordinator for the proposed work. All authors provided final approval of the manuscript prior to submission and agree to be accountable for all aspects of the study described here.
Funding The University Research Grants Programme (URGP) (UM Project no.52519) at the University of Manitoba provided funding for this research. Funders provided financial support for this forthcoming project but are not involved in the SR.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Author note MA is an applied research scientist interested in technological and methodological innovations to support knowledge exchange between diverse stakeholders, often using arts-based and mixed method research approaches and methods of application focused on the integration of lived experience research with evidence of effectiveness to reduce evidence misalignments.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.