Review of reviews of RCTs and observational studies review of most recent controlled trials

As presented in table 1 which reviews specific inclusion and exclusion criteria, our search strategy has been developed to identify (1) reviews of RCTs and controlled trials of primary and/or community care interventions to improve long-term outcomes after stroke in stroke survivors and/or informal carers, (2) recent RCTs and controlled trials published in the past 5 years (since January 2011); and (3) reviews of observational studies. We will focus on systematic reviews and meta-analyses as they provide a quantitative review of findings. The reviews of observational studies will be identified together with the reviews of RCTs and controlled trials using an additional inclusion criterion specific to observational designs (eg, including cross-sectional studies, longitudinal studies, retrospective cohort). We will search the five databases as recommended by the Centre for Reviews and Dissemination53 as the basis for systematic reviews in healthcare research: MEDLINE, EMBASE, PsycINFO, CINAHL and either Cochrane Reviews (to search for review literature) or Clinicaltrials.gov (for recent trials).

Grey literature

Little guidance exists on methods to review grey literature. Some publications focus on systematic review methods adapted for grey literature search.54–57 However, their purpose was to complement systematic reviews of published research in their respective areas. In contrast, our aim is to complement the scoping review of the trial and observational literature by identifying potential examples of good practice to inform new models of primary care. Consequently, our search strategy focuses on examples of good practice by searching the websites of professional bodies relevant to healthcare and/or stroke. Grey literature will be identified using an advanced option in google.com search engine, with limits set to site (eg, http://www.nao.org.uk), language (English) and the past 15 years to keep the search manageable and relevant to current healthcare. This strategy offers consistency compared with the reliance on varying search algorithms employed by specific websites. We will search the content of major health organisations, clinical professional bodies, and stroke charities in the UK and internationally, including but not exclusively the: (1) Stroke Association, (2) Department of Health in the UK, (3) WHO, (4) National Institute for Health and Care Excellence, (5) UK National Audit Office, (6) National Stroke Association (USA), (7) World Stroke Organisation, (8) European Stroke Network and (9) Internet Stroke Centre (USA).

One reviewer will screen the titles and the brief text underneath for relevant documents. Links identified as relevant will be screened at the same time as titles for relevant content (figure 1). Potentially relevant documents will be saved for further screening. To keep the search manageable, we will limit the screening to the first 10 pages of results (∼100 results) relying on Google's PageRank algorithm which prioritises relevant sites.57 We chose to limit our search to the past 15 years to both manage the number of retrieved records and to focus on the results which are most relevant to the current organisation of healthcare system(s) and the interaction between different components of care.58

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

A flow diagram of search strategy for grey literature.

Identification of trial and observational literature: reviews

We have developed a set of key terms organised according to the study type and Population Intervention Comparison and Outcome (PICO) framework: (1) Population: terms related to stroke (eg, stroke*, ‘cerebrovascular accident*’, CVA) were triangulated with Boolean Operator ‘AND’ with those related to a stroke survivor (eg, survivor, patient*) or caregivers (eg, carer*, caregiver*, family), (2) Intervention: primary care and community health services (eg, primary care, general practice, community care/healthcare, postdischarge services, long-term care/rehabilitation), (3) Comparison: we will include any control group, (4) Outcome: any modifiable long-term outcome after stroke except for medication efficacy. The terms across the PICO categories will be triangulated with Boolean Operator AND. We will focus on review literature searched using a filter adapted from the Scottish Intercollegiate Guidelines Network for systematic reviews59 and adapted to include non-systematic reviews. Text within the title or abstract, and Medical Subject Headings (MeSH) term (or equivalent) searches will be conducted. An example of a search strategy developed for MEDLINE is included in online supplementary appendix 1. We will only include studies which focus on the population of community dwelling stroke survivors and on care delivered within ambulatory primary care and community healthcare settings (table 1). Studies conducted in nursing homes, residential care settings or in inpatient settings (eg, community hospitals) will be excluded. No time or language limits will be set but due to pragmatic reasons only texts in English will be included. We will only include published and peer-reviewed reviews. Reviews of observational studies will be identified together with the reviews of trial literature and sorted based on the design of included studies.

The development of the search strategy was iterative and followed three steps. In the first step, a preliminary set of key terms was developed based on previous literature and key words identified in a qualitative review protocol conducted within our team.49 In the second step, we conducted a preliminary search in PubMed. The search strategy and key terms identified were further reviewed by our research team. Key terms were expanded (eg, we added ‘general practice’ and ‘general practitioner’), terms related to specific outcomes were excluded, and those related to the population of interest (eg, stroke survivors, patient, caregiver) were included. In the third step, the final search strategy was employed and adjusted for each of the specific databases. Search strategies for each of the five databases were reviewed by a medical librarian experienced in systematic review methods. Further eligibility criteria for the studies to be included in the scoping review are described in table 1.

Identification of trial literature: most recent trials

Using the same set of key terms triangulated with a filter to identify controlled60 and RCTs61 (see online supplementary appendix 2), we will search five databases (MEDLINE, EMBASE, PsycINFO, CINAHL, Clinicaltrials.gov) for trials published in the past 5 years (since 1 January 2011). The combination of filters developed by the Cochrane Collaboration for RCTs and Robinson et al60 was used to maximise sensitivity while also preserving specificity.62 The search strategy has been tested in MEDLINE and reviewed by a medical librarian. Following the screening of abstracts, we will cross-reference the identified titles against the reference lists of the included reviews. Such a strategy will help identify trials assessing a wide range of long-term outcomes after stroke, thus supplementing the review of reviews. Given that we are interested in a broad range of long-term outcomes after stroke, we made a pragmatic choice to narrow the search of the most recent trials to the past 5 years. As with review literature, we will only include published and peer reviewed articles written in English.

Identification of grey literature

A search strategy for the review of grey literature was developed iteratively. First, initial key terms were identified based on the peer reviewed trial literature, which was then tested against specific websites (Department of Health and Stroke Association) as well as Google search using advanced settings. The hand search of the Stroke Association and Department of Health websites helped narrow the terms related to stroke and primary care. The strategy was then reviewed by an interdisciplinary group of experts (allied healthcare professionals, academics specialising in general practice and stroke and a stroke survivor) who suggested adding key terms of ‘best practice’ and ‘effective service delivery’, and the inclusion of additional websites: for example, National Audit Office and Royal College of Physicians.

We developed three broad groups of search terms: (1) examples of good practice (eg, ‘good practice’, ‘best practice’, ‘effective service delivery’, ‘practice example’), (2) the context of primary care and community services (eg, ‘general practice’, ‘community health services’, ‘voluntary service’, ‘charity worker’, ‘social services’) and (3) stroke. A set of key terms for each of these categories will be triangulated using Boolean Logical Operators AND/OR. We reduced stroke-related search terms to a single term ‘stroke’, as this term was used most frequently in relation to healthcare services in general Google search. An example of the grey literature search strategy developed for the National Audit Office is included in online supplementary appendix 3. We set language limits to English only and, depending on the breadth of the website, set a pragmatic time limit to 1 January 2000.

Review of reviews of RCTs and observational studies, review of most recent controlled trials

Data will first be downloaded to EndNote X7.1 (EndNote X7.1, Thomson Reuters). Titles will be exported to an Excel spreadsheet where the screening results will be recorded. In phase 1 (review of reviews of trials), two reviewers will independently screen the titles of reviews identified by our search strategy (figure 2). Next, we will screen abstracts of potentially relevant reviews using our inclusion/exclusion criteria (table 1). In the final stage of phase 1, two reviewers will screen the full texts using the data extraction spreadsheet (available on request) to extract the information listed in table 2 onto an Excel spreadsheet. Any inconsistencies across all stages will be resolved by discussion between the reviewers. An experienced third reviewer will be consulted should no consensus be reached. The second phase (RCTs and controlled trials published in the past 5 years) will follow the same process as described above. The review of reviews of observational studies will follow the strategy as outlined for the trial literature except for the inclusion criteria related to study type and control groups. Data will be managed in the same manner as the data for the trial literature. We will include observational studies (eg, cross-sectional, longitudinal, retrospective cohort), which focus on primary and community care after stroke. We will use the PRISMA flow diagram to report the number of screened and included studies.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

A flow diagram of search strategy for trial literature and observational studies. RCTs, randomised controlled trials.

Grey literature review

Grey literature search will be performed by one reviewer. A reviewer will screen the results of the Google search (titles and brief text underneath the title) to identify potential documents with examples of good practice within primary and community care relevant to long-term outcomes after stroke. Documents deemed relevant will be downloaded for data extraction and record keeping. We will operationally define the examples of good practice as case studies in guides, guidelines, action plans, consultation papers developed by government agencies, local and national health authorities and professional bodies (eg, National Audit Office Good Practice Guide, policy documents on transforming community services from the UK Department of Health, Royal College of Physicians Sentinel Stroke National Audit Programme (SSNAP) Report). The documents identified as relevant will be saved for further screening using the following inclusion criteria: (1) the practice example has to focus on care after stroke (for a survivor and/or informal carer), (2) on improving long-term outcomes, (3) it has to be delivered by general practice, (4) practice components have to be clearly described including information on what was delivered, by whom and to whom, and its outcomes, and (5) the justification for why it was considered good practice has to be provided. The identified examples of good practice will be reviewed and discussed within the research team and as part of the consultation exercise to assess their utility for a new model of primary care.

RCTs and controlled trials

We will summarise the information from reviews and most recent experimental research using descriptive methods. On the basis of the scoping review objectives, we have developed a data extraction framework to help visualise the key outcomes to answer our research questions (table 2). The review data chart will include: (1) bibliographic information, (2) characteristics of the review (eg, setting, population, intervention characteristics, outcomes), and (3) main results (summary of the reported effects and effects sizes). The data chart will summarise the key characteristics of interventions (timing, duration, frequency, ‘active ingredients’, mode of delivery and who delivered the intervention) in relation to specific outcomes. It will also help identify potential gaps in research in relation to the type and timing of interventions (eg, within the first year since stroke or beyond), its scope and the long-term outcomes it addressed (eg, psychological outcomes, those related to physical function, participation, activities). Since the methodology of a scoping review is guided by iterative approach to data synthesis,48 we will first develop a preliminary data extraction sheet based on the chart presented in table 2. The data extraction form will then be pilot tested by one reviewer, consulted with the second reviewer and, on reaching the consensus, finalised. Since no formal quantitative synthesis will be attempted, we will rely solely on the data reported by the authors in publications. Tabulation of results from the RCTs and controlled trials using similar categories and following the same process as that described for the reviews will aid comparative analyses of the evidence from phases 1 (reviews) and 2 (most recent RCTs and controlled trials).

Observational studies and grey literature

First, the results from the reviews of observational studies will be charted using an adapted data extraction framework developed for the reviews of RCTs (table 2). As in the review of trial literature, the development of a data extraction sheet will be iterative. The characteristics of primary and/or community care services (eg, service provider, target population, service availability and/or use, timing of service and, where appropriate, its duration, frequency and intensity) will be extracted. Information on the strength and the direction of the associations between service characteristics and long-term outcomes after stroke as well as study design (eg, prospective, retrospective, cross-sectional) will also be recorded.

Second, we will extract the data from grey literature focusing on the examples of good practice (eg, new service or assessment addressing a particular long-term need after stroke, refinement of existing services), the description of practice or service (eg, what was delivered and how?), service provider, timing of delivery (in relation to stroke onset), its impact on long-term outcomes and justification as to why it was considered ‘good practice’ (eg, benefits to stroke survivors/informal carers, larger community, adherence and/or implementation of guidelines). The data will be extracted using a data extraction form (an example is presented in online supplementary appendix 4; the form will be tested and amended once relevant literature is identified) and recorded in an Excel spreadsheet.

Review of reviews of trial and observational literature, most recent trials and grey literature

In the first instance, the results of phase 1 of the review will be summarised. We aim to report which long-term outcomes have been addressed by the interventions included in our review, which areas of research (and care) are well developed and could therefore inform models of primary care services to support the long-term care of stroke survivors and carers, and which areas need further research. The discussion of findings from phase 2 will complement findings from phase 1 by mapping the areas of current focus within research and practice. We will reflect on whether and how the trials address the gaps identified in phase 1. On the basis of findings from phases 1 and 2, we will discuss the characteristics of interventions found effective and contrast them with those which were found less effective. We will also outline the areas where further research is most needed to address needs reported by stroke survivors and caregivers in the literature. Further, using the findings from grey literature, we will provide an overview of potential components of good clinical practice to further inform pathways within a new model of primary care services. The components of clinical practice with supporting evidence will be charted against the gaps identified by the trial literature. For example, if one of the gaps identified is training for carers, we will provide details of how this might have been addressed in observational research and/or practice exemplar(s) from grey literature. On the basis of this comparison, we will be able to make specific recommendations for future research.