Article Text

Communication
Community-Partnered Research appraisal tool for conducting, reporting and assessing community-based research
  1. Avery Park1,
  2. Jenna van Draanen2
  1. 1University of Washington - Seattle Campus, Seattle, Washington, USA
  2. 2Child, Family, and Population Health Nursing; Health Systems and Population Health, University of Washington, Seattle, Washington, USA
  1. Correspondence to Dr Jenna van Draanen; jvandraa{at}uw.edu

Abstract

Objective The aim of this study was to develop an appraisal tool to support and promote clear, accurate and transparent standards and consistency when conducting, reporting and assessing community-based research.

Design Current recommendations for developing reporting guidelines was used with three key differences: (1) an analysis of existing guides, principles and published literature about community engagement, involvement and participation in research using situational and relational maps; (2) feedback and pilot-testing by a community-based research team; and (3) testing the utility and usability of the appraisal tool.

Results After a series of iterative revisions, the resulting Community-Partnered Research (CPR) appraisal tool emerged into three products: an elaborate prospective format, a basic retrospective format, and a supplemental checklist format. All three versions of the CPR appraisal tool consist of 11 main question items with corresponding prompts aimed to facilitate awareness, accountability, and transparency about processes and practices employed by professional researchers and community co-researchers throughout four phases of research: (1) partnership and planning, (2) methods, (3) results and (4) sustainment.

Conclusion We hope that introducing this tool will contribute to shifting individual and systematic processes and practices towards equitable partnerships, mutual trustworthiness and empowerment among professional researchers and community co-researchers and, in turn, improving the quality of co-created knowledge that benefits communities and creates social change.

  • Community-Based Participatory Research
  • PUBLIC HEALTH
  • STATISTICS & RESEARCH METHODS
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Driven by social movements to include marginalised and vulnerable voices in the scientific processes since the 1940s, models of public involvement, engagement and participation in scientific research emerged across disciplines in hopes of challenging the traditional, unidirectional, top-down practice in research and shifting the relationship between science and society towards a more integrated ‘science in society’ approach.1 2 This ‘participatory turn’ became a collective response among professional researchers and communities seeking to address the ongoing colonisation of science. Engaging the public as co-researchers promoted the vision of democratising science by redefining hierarchical power relations between the researcher and those they researched to reduce health disparities and inequities, improve population health management and public health initiatives, and collaboratively change the healthcare continuum in communities (To practice inclusivity and balance positionality through language, ‘community co-researchers’ will be used to acknowledge the variety of involvement community members may participate in when contributing to the scientific inquiry process).1 3–7

Common community-based frameworks range from contractual models, where communities request professional researchers to conduct specific scientific investigations; collaborative models, where experienced researchers generally design the study while community stakeholders support the design and implementation of the project by contributing their ‘community voice’ at various points of the scientific inquiry; to co-created projects, where professional researchers and community co-researchers partner at all stages of the research lifecycle and co-design research questions driven by community needs, co-learn and build local capacity, and co-create knowledge that has a transformative impact in communities and address real-world problems.1 8–16

Although these models of public participation in scientific research have made substantial scientific contributions in many fields, current standards for conducting traditional research involving human subjects are seen as an ‘imperfect fit’ for community-based research due to each partnership’s unique nature and variation.1 3 9 17 18 (Given its widespread use and recognition, the term ‘community-based research’ will function as an all-encompassing terminology to represent the extensive breadth of models using public participation in the scientific inquiry process.) For that reason, public and private organisations and associations worldwide have discussed and determined principles and guidelines for community-based research. Examples of prominent resources in this area include the ‘Principles and Guidelines for Community-University Research Partnerships’, developed by a committee of academic professionals at the Yale Center for Clinical Investigation to reflect an ideal community-university research partnership.19 Another guide known as ‘Community-Based Participatory Research: A guide to ethical principles and practices’ emerged from a research project in the UK, primarily focusing on the lessons learnt by those involved to ensure ethical research conduct when using unique approaches that reveal insights into the lived experiences of communities.20 Subsequent guidelines continued to build on prior work and produced the ‘Participatory Health Research: A guide to ethical principles and practice’.21 Additionally, other professional associations created standards representing their belief in what underlies best practices in community-based research, such as the ‘Ten Principles of Citizen Science’.22 Presently, workshops and think tanks like those hosted by the US National Institutes of Health facilitated discussions to identify and explore the ethical, legal and social implications associated with citizen science research in hopes of eventually establishing criteria and guidance for community-based research.23

However, such principles and guidelines stop short of providing tools for reporting requirements of findings and assessing the quality of existing evidence, thus resulting in suboptimal reports and reviews of community-based research that fail to produce robust, usable and reproducible knowledge for academic and non-academic communities alike.24 Furthermore, there is often a lack of accuracy, clarity and transparency in scientific reporting about how professional researchers and community co-researchers partner and collaborate when conducting community-based research, leading to discrepancies in the variety of methods that are referred to as community-based and risking dilution of the core tenets that characterise community-based research. The current state of the science in community-based research reveals a critical need for an explicit methodology to improve the quality of conducting, reporting and assessing community-based research and uphold researcher responsibilities, protection of participants, and scientific rigour. In this paper, we introduce prospective and retrospective systematic guidelines and a supplemental checklist known as the Community-Partnered Research (CPR) appraisal tool to promote clear, accurate and transparent standards and consistency when conducting, reporting and assessing community-based research while minimising the influence of bias, power and privilege within and across studies.

Methods

Development of the CPR appraisal tool

Our team followed many of the steps indicated by current recommendations for developing reporting guidelines,24 with three notable and intentional differences: First, we allowed our process to be abductive and reflexive, where we extracted data from a literature review and grouped constructs through situational and relational maps. Second, instead of having our team conduct a Delphi process, a top-down approach, to determine question items and prompts for inclusion in the guidelines, we went back and forth with a community-based research team who gave suggestions for refinement based on utility, comprehension and practical needs, finalised the development of the tool, and pilot-tested the prospective CPR appraisal tool. Lastly, we tested the utility and usability of the retrospective CPR appraisal tool on publications25–27 to identify functional problems in the design of the tool.

Literature search and review

Six inclusion criteria were applied to the literature review to identify relevant papers for consideration24 and resulted in a review containing the following paper types: existing principles, guides and frameworks; original articles reporting the experiences of conducting community-based research at various stages; and commentary papers discussing the social, legal and ethical implications of public participation in scientific research. The literature search included international work with restrictions to publications written in English between the years 2000 and 2022. This time period was chosen as it reflects the advanced expansion of well-established community-based models across multiple fields and disciplines. Lastly, the literature search did not apply specific age ranges, populations, and characteristics of interest.

Exclusion criteria comprised studies that did not explicitly state a relevant model or type of community-based research, such as community-based participatory research, participatory action research and participatory health research. In addition, papers that reported conducting community-based research but did not adequately describe the procedures and experiences throughout the scientific investigation were excluded, as well as those that presented findings on non-human populations of interest, virtual methods (eg, crowdsourcing), single case studies and unpublished articles, including dissertations and conference abstracts.

The authors first met regularly to achieve consensus on which papers would meet the inclusion criteria and how to screen consistently. Once AP and JvD achieved reliability, AP conducted the search and screened the records retrieved from the literature review using recommended guidelines28 and extracted empirical data from scholarly publications of community-based research. We focused on a purposive rather than an exhaustive sample to achieve data saturation.29 Decisions were made not just based on the existence of collaboration between professional researchers and community co-researchers but also on the centrality of partnership to the study and the relative value of the information about partnership and co-creation of knowledge presented in each study.30 As a result, the strength of our findings reflects a high degree of diversity and information relevance, richness and ‘thickness’ of detail.31

Papers were systematically identified and screened by using one electronic database, PubMed. Search terms, combined with ‘AND/OR’ operators depending on the term, were developed with assistance from a university librarian. After an initial rapid review and pilot searches, the final iteration of search terms included ‘community based’, ‘community led’, ‘community centred’, ‘community informed’, ‘community participation’, ‘citizen science’, ‘community-based participatory research’, ‘participatory research’, ‘stakeholder participation’, ‘questionnaire validity’, ‘surveys and questionnaires’ and ‘data collection’. Furthermore, additional hand-searching methods were employed by identifying potential papers from the reference lists of the included studies. The primary search strategy produced 2000 relevant documents that met the initial screening criteria. After compiling the results, the inclusion and exclusion criteria were applied to the remaining titles, abstracts and full text. For details on the search process, see the Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart (figure 1).32

Figure 1

PRISMA flowchart of literature search procedures. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

The review and screening process resulted in 54 papers as the final dataset. Among the papers that were included, 35 original articles incorporated constructive information about their experiences, reflections and lessons learnt when conducting community-based research.33–66 13 papers represented commentaries examining the social, legal and ethical implications of public participation throughout every research stage.1 3 9 10 18 67–74 The remaining six papers were principles, guidelines, frameworks and workshops to inform community-based research.19–23 75

Situational and relational maps

Throughout the literature review, situational mapping was used to organise and reorganise emerging human and non-human, material and symbolic/discursive elements within each phase of the research process as framed by those involved in community-based research.76 Upon reaching saturation in the data-based situational map, relational analysis was applied to pursue relations systematically and empirically among the various elements and capture the dynamic inter-relationships across multiple levels and models of public participation. The most salient outcomes of the relational analysis using situational maps were consolidated into question items and prompts and included in the CPR appraisal tool. In addition, AP and JvD practiced reflective and reflexive strategies to minimise the impact of bias throughout the project, especially when seeking a consensus for an item. For example, AP and JvD debated on a potential item concerning the environmental impact and sustainable future of research programmes. However, upon further examination, AP and JvD rejected the potential item because the element of environmental sustainability was not present as an item directly related to partnership in the data but rather emerged as an insight into a possible trend for future considerations within community-based research that the authors personally identified with.

Feedback and pilot testing

The first version of the prospective CPR appraisal tool was evaluated, refined and then pilot-tested by a group of professional researchers and community co-researchers grounded in community-engaged research principles,77 formally known as the Research with Expert Advisors on Drug Use (READU) team. This team consisted of five formally trained researchers and five community members with lived and living experiences of drug use. Some formally trained researchers on the team also identified as having lived experience. Together, the READU team actively read through the tool and gave feedback on the constructs, advice for refining language and ways to make the tool directly applicable to the work done by community-based research teams. After iterative tool refinement, a near-final draft of the prospective CPR appraisal tool was then piloted by the READU team to guide the description of their community-based research model. The tool was examined and used as a group, and all comments, especially disparate ones, were discussed extensively by the READU team until consensus was reached on what recommendations to provide to strengthen the CPR appraisal tool. Further improvement was suggested in three main domains: (1) rewording items for clarity on the final version of the prospective CPR appraisal tool; (2) the recommended creation of a more succinct version of the tool to guide descriptive reporting and critical appraisal, later known as the retrospective CPR appraisal tool; and (3) the recommended creation of a supplemental checklist version of the CPR appraisal tool for journal article submission. More details on the final three products of the CPR appraisal tools and their intended use are provided in the Results section.

Then, we tested the utility and usability of the retrospective CPR appraisal tool on several published articles.25–27 JvD chose the published articles representing community-based research projects and verified that the tool would sufficiently capture the process. Subsequently, AP used the retrospective CPR appraisal tool without any priming or added explanation to ensure accurate attributions to the functionality and sufficiency of the tool’s 11 question items and corresponding prompts. This testing process revealed that it was possible to quickly identify the presence or absence of community partnership in articles that correlated with the question items and corresponding prompts in the retrospective CPR appraisal tool. Some question items with their corresponding prompts from the retrospective CPR appraisal tool were more challenging to locate than others, specifically 2D, 5C, 6C, 7B, 8B, 8D and 10B, as most of these question items and corresponding prompts would typically be internally discussed between professional researchers and community co-researchers and not always made explicit in scientific reporting. These question items and corresponding prompts may be essential, then, for authors to report (perhaps even by using the retrospective CPR appraisal tool and/or the CPR supplemental checklist as an appendix, supplemental file or result table) when submitting a manuscript to ensure that the word limit requirements of journals are met while upholding transparent and consistent reporting standards for community-based research publications.

Results

Aim and scope of CPR appraisal tool

After a series of iterative revisions, the final CPR appraisal tool emerged into three products: an elaborate prospective format, a basic retrospective format and a supplemental checklist format. All three versions of the CPR appraisal tool consist of the main 11 question items aimed to facilitate awareness, accountability and transparency about processes and practices employed by professional researchers and community co-researchers throughout four phases of research: (1) partnership and planning, (2) methods, (3) results and (4) sustainment. The three versions of the CPR appraisal tool are designed to serve distinct purposes:

The elaborate prospective CPR appraisal tool is intended to act as an in-depth, practical guide for new and existing academic-community partnerships to use when preparing, planning and conducting research as it explains the meaning of each question item and its corresponding prompts. For instance, the READU team used the elaborate prospective CPR appraisal tool to descriptively guide their discussion about their partnership model. Each question item and its corresponding prompts enabled the READU team members to reflect personally and relationally on how things were done in the past and how they could improve their partnership for future projects. The elaborate prospective CPR appraisal tool can be found in online supplemental appendix 1.

The basic retrospective CPR appraisal tool is intended to be more user-friendly, with concise corresponding prompts, to serve as a systematic guide to ensure that information is descriptively and accurately reported or appraised. When reporting the results of a study, a community-based research team may opt to describe how each question item and corresponding prompts are met either throughout the article itself or in the space provided in the retrospective CPR appraisal tool. For critical appraisal, external reviewers may use the question items and corresponding prompts as an analytical method for evaluating the trustworthiness, value and relevance of the processes and results reported in community-based studies. Then, the completed tool can be referenced or linked to the main article and uploaded as an appendix or supplementary file for publication submission. The basic retrospective CPR appraisal tool can be found in online supplemental appendix 2.

The CPR supplemental checklist was designed to serve two purposes after using the prospective or retrospective CPR appraisal tool: (1) for transparency, much the same way existing checklists78–82 are used at the time of article submission to designate the location(s) of each reported item in the manuscript, and (2) for quality assessment purposes, where reviewers or those extrapolating evidence from community-based research studies can record the extent to which studies have incorporated essential elements of partnership based on the question items and corresponding prompts and provide the completed supplemental checklist as an appendix, supplementary file or results table.

When using the CPR supplemental checklist to transparently report the presence or absence of community partnership throughout each phase of the research, we recommend applying ‘Not applicable’ (N/A) under the ‘Reviewer’s rating’ and identifying the page number(s) where each question item and corresponding prompt are described in the manuscript under the ‘Item’s location’ column. When using the CPR supplemental checklist to critically appraise the elements of partnership in a study, we recommend completing both the ‘Reviewer’s rating’ and ‘Item’s location’ columns. Reviewers can rate each question item as ‘Yes’, ‘No’, ‘Unclear’, or ‘Not applicable’ and record observations and reasons for ratings in the spaces provided. It is recommended that the rating be discussed by all members involved in the critical appraisal process to develop a shared understanding of the study. The CPR supplemental checklist can be seen in table 1.

Table 1

The CPR supplemental checklist for reporting community partnership or critical appraisal

Of note, the question items and corresponding prompts were intentionally structured with compound conditions that should be fully met for the reviewer to rate the entire item as a ‘Yes’. For example, in the Methods phase, question item 4 states, ‘Did the study design align with the nature of the research question(s) and meet the needs of the community of interest’? Disregarding either condition, (1) the study design aligning with the research question or (2) meeting the needs of the community interest, may signify a need to critically re-examine and adjust the foundational elements that constitute a mutual partnership between professional researchers and community co-researchers. Since the scope of the CPR appraisal tool primarily focuses on the nature of partnership and co-creation of knowledge in community-based research, it is recommended to use the CPR appraisal tool with other study design-specific tools83 84 when conducting quality assessment to ensure balanced alignment between methodological rigour and the needs of the community of interest.

Discussion

Discrepancies in how community-based research methods are used, reported and assessed threaten to impede the forward movement of integrating knowledge and action that benefit communities, democratise science and bring social change.1 8–16 Therefore, the CPR appraisal tool was designed to reflect the current community-based research literature as well as the needs and experiences of research partners. The systematic application of this tool may assist in increasing awareness, transparency and accountability among professional researchers, community co-researchers and other scientific community members. Furthermore, conducting explicit, methodical and accurate reporting and assessment of community-based research can encourage more effective critiques and peer review processes. The collective approach in strengthening the rigour and quality of reliable knowledge85 generated by community-based research can help reduce the gaps between theory, research and action to address social, structural and environmental inequities7 and create sustainable solutions to improve community health endeavours.9 44 Although the CPR appraisal tool has the potential to facilitate a paradigm shift in how community-based research is conducted, reported and assessed, fundamental changes must start with accurate and transparent reporting to improve the reliability, utility and impact of public participation in science and communities.86

To our knowledge, only one systematic review has been completed to evaluate the quality of published studies using community-based research methods.87 Although included studies reported improvements in recruitment, research methods, dissemination and other outcomes, information about partnership development and collaboration among professional researchers and community co-researchers was typically condensed into a few words or completely missing.87 Such publications with minimal transparency of how community-based research is conducted persist and contradict existing guidelines on writing manuscripts about community-based research for peer-reviewed journals.88 This lack of transparency is likely due to limitations related to funding and page length of most peer-reviewed journals that restrain the descriptive writing needed to thoroughly situate the range and depth of agreements and relationships established among all partners throughout the research lifecycle of co-designing, co-learning and co-creating knowledge.87 Changing these norms is critical because how information is shared and withheld can influence power dynamics at the individual and structural levels, specifically diminishing the capacity of individuals to maintain autonomy, self-determination89 and their trust in science and scientific institutions.90 These existing norms can force those partnered in community-based research into a conundrum of navigating between traditional ways of reporting on their scientific work that may potentially mute the voices of communities they included in their projects,91 92 and reporting in an authentic way that is seen as not normative may challenge scientific convention or be viewed as less rigorous. We hope that the CPR appraisal tool can help mitigate this challenging position so that community-based research can continue to refine the principles, methods and practices of co-created knowledge. In doing so, the commitment to building and maintaining equitable and trustworthy partnerships has shown to contribute a ripple effect of longitudinal outcomes, including partnership longevity, spin-off projects and systematic transformations that manifest as cultural shifts, implementation of evidence-informed policies, and advancements in health equity.44

The widespread application of the CPR appraisal tool has several future implications. For instance, following the recommended guidelines of the CPR appraisal tool may increase investment and commitment in the project and relationships, increase clarity about roles and responsibilities, and help articulate a shared vision and goal during the partnership and planning phase when conducting community-based research. Furthermore, similar to the endorsement of other recommended guidelines, using the CPR appraisal tool may increase the word count and length of manuscripts and review reports while conversely improving the clarity, transparency and completeness of community-based research publications,78–82 strengthening the systematic evaluation of research evidence,93–95 and encouraging more authentic and trustworthy partnership among all parties involved in community-based research.54 96 97 However, due to the current state of community-based research publications, appraisals from external reviewers may initially have trouble identifying all the items in the retrospective CPR appraisal tool. So, we implore readers and future users of the tool to approach the new standards of community-based research with patience, persistence and pragmatic adaptation as we dynamically and reciprocally interact and adapt into a more integrated, transparent and rigorous ‘science in society’.2

The development of the CPR appraisal tool benefited from a wealth of prior data published in studies using community-based methods from many different geographical and cultural contexts, thus representing the widespread utilisation of community-based research across diverse populations. However, limitations to the current tool include the adaptation of question items and corresponding prompts from included papers, which was mediated through the positionality of the researchers, authors, peer reviewers, and AP and JvD. While the abductive and reflexive processes increased the appropriateness and refinement of the included empirical elements and relations, they were altered to an extent each time76 and would be challenging to replicate identically. Another limitation that impacted the scope of the current tool was the constraints of the literature review. Although interventions/clinical trials were not intentionally excluded from the literature review, the identified source documents were predominately observational studies, thus biasing the development and limiting the utility of the CPR appraisal tool towards specific study designs. Nevertheless, situational and relational maps are a widely used process where the element and relations are designed to be loosely bound and ever-changing,98 enabling the CPR appraisal tool to continue to evolve alongside the expansion and refinement of community-based research approaches and publications. Lastly, the relevancy and application of the CPR appraisal tool in other contexts and communities may also be limited as it reflects the current literature and the needs and lived experiences of the community-based research team who offered feedback and pilot testing. However, the CPR appraisal tool is expected to be transferrable and promote further exploration and reflection for other situations of inquiry.

To facilitate wider dissemination and uptake of this innovative tool in the scientific community, we encourage endorsement and widespread utilisation of the CPR appraisal tool by those participating in, assessing and publishing community-based research. Our team plans to follow the post-publication activities indicated by current recommendations for developing and disseminating reporting guidelines.24 We hope to present the CPR appraisal tool in various formats and translations to accommodate our audiences’ varying needs for and responses to scientific information.99 By doing so, we invite community-based research teams, external reviewers and community members to provide feedback and constructive criticism about the CPR appraisal tool.100 In this open dialogue, we will suggest refinement or revision, as appropriate, to the question items, prompts and guidelines of the CPR appraisal tool to align with evolving community-based research best practices. In addition, we also aim to evaluate the impact of the prospective and retrospective CPR appraisal tool as well as the CPR supplemental checklist in future research projects. For example, we plan to subsequently test the utility, usability and user experience of the retrospective CPR appraisal tool by recruiting community-based research teams across disciplines. We hope introducing the CPR appraisal tool can function to bring together professional researchers and community co-researchers as partnered explorers97 with a shared explicit language96 101 102 who co-learn, co-design and co-create robust, usable and reproducible knowledge that propels community-based research forward towards social, structural and environmental equity and sustainability for individual, family and population health.

Ethics approval

Not applicable.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors All authors contributed substantially to the design and interpretation of the tool and guidance, writing sections of drafts, revising based on feedback received and approving the final version. AP conducted the analysis of the literature review, managed and revised the checklist items and drafted the paper. JvD mentored and offered feedback, reviewed and revised the checklist items and drafted the paper. The READU team reviewed, provided suggestions and pilot-tested the tool. AP is the guarantor for the project and accepts full responsibility for the finished article. All authors attest that the listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests The views expressed in the submitted article are the authors’ own and not an official position of the institution or funder.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.