The dissemination into clinical practice of knowledge gained from research can improve patient care and outcomes.1 However, research suggests that 30–40% of patients do not receive treatments shown to be effective, and that 20–25% receive treatments that are either unnecessary or potentially harmful.23

Clinical guidelines aim to improve the quality of care, reduce practice variation and/or reduce cost by providing clinicians with graded recommendations that reflect best practice. Research has shown that they can improve health outcomes and influence the processes of healthcare.4 However, they can have little impact on clinical practice unless they are successfully integrated into the clinical setting.5 The extent to which this occurs depends on primarily two factors: the quality of the evidence on which the guideline is based, and the effectiveness of the guideline implementation strategy used.6

To date, only one published systematic review has investigated the effect of guideline implementation in professions allied to medicine.7^–9 That review included allied health and nursing professions, and of the 18 included studies only two focused on allied health professionals. One study evaluated implementation of guidelines for dietitians and the other evaluated implementation of guidelines in a multi-disciplinary team.1011 Given the small number of studies involving allied health professionals no conclusions can be drawn from this review about the effectiveness of guideline implementation strategies for allied health.

To optimise effectiveness of implementation strategies it is important that an evidence-based approach is taken towards implementation of evidence-based guidelines. Therefore, the current review aimed to identify studies that have evaluated the effect of the introduction of clinical guidelines in allied health professionals and to estimate the effectiveness of the strategies used to disseminate and implement these guidelines.

METHODS

Study identification and selection

We searched electronic databases (MEDLINE 1966–June 2006, CINHAL 1985–June 2006, EMBASE 1988–June 2006, PsychINFO 1985–June 2006, AMED 1985–July 2006, the Cochrane Controlled Trials Register Issue 3 2006, DARE Issue 3 2006) using a search strategy built on the Cochrane Effective Practice and Organisation of Care (EPOC) Group strategy.12 The strategy combined terms relevant to guideline implementation, allied health professionals, outcomes and trial design (see web only appendix A). In addition, the EPOC trial register was searched, and the reference lists of all relevant articles identified were hand searched.

Types of studies

Randomised controlled trials (RCT), controlled clinical trials, controlled before and after studies and interrupted time series studies were eligible for inclusion. In addition, only articles in English (resources were not available for translation) and published in peer-reviewed journals were included.

Types of participant

We included studies in which allied health professionals were a part of the population. For the purposes of this review, allied health included the following professions: audiology; dietetics; occupational therapy; orthoptics; orthotics and prosthetics; pharmacy; physiotherapy; podiatry; psychology; radiography; speech pathology; and social work.13 Studies involving other professionals (eg, medical, nursing) in addition to allied health professionals were included if the processes or outcomes for the allied health profession(s) were evaluated separately, or if 50% or more of the participants were allied health professionals.7

Types of intervention

Clinical guidelines were defined as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances”.14 All types of dissemination and implementation strategies were included.

Types of outcomes

Studies were included if they reported objective measures of change in the allied health practitioners’ behaviour or patient outcomes. Studies measuring other outcomes, such as change in knowledge or attitudes of the allied health practitioner and costs were only included if change in practitioner behaviour or patient outcome were also measured.

Methods of the review

Two authors independently reviewed the references identified by the literature search for relevance. Full text articles of potentially relevant studies were obtained and assessed for inclusion. Disagreements between the reviewers were settled through discussion. One reviewer (SH) extracted data using a standardised form. This was checked by a second reviewer (KD). Discrepancies were resolved by discussion. The quality of included studies was assessed using criteria set by the Cochrane EPOC Review Group with each criteria scored as done, not clear or not done.12

To aid comparison between studies, the implementation and dissemination strategies used were classified according to a taxonomy developed by the EPOC group.15 The taxonomy provides criteria for different interventions using three major categories: professional (eg, educational outreach visits), financial (eg, fee for service) and organisational (eg, revision of professional roles). Outcomes were categorised as either “patient”, where the measurement was related to a change in the patient health status (eg, blood pressure measurements), or “process” where the measurement was related to a change in the process of care (eg, prescribing behaviour). Cost data were presented under process outcomes.

For dichotomous measures, effect sizes were calculated as the differences between groups with respect to the post-intervention percentages.16 Risk and odds ratios and their 95% confidence intervals were also calculated where sufficient information was provided.1718 The standardised mean difference (Hedges g) for the intervention was calculated for continuous outcomes. In this review an effect size of 0.2–0.49 was considered small, 0.5– 0.79 moderate and above 0.8 large.19 It was not feasible to pool results of trials for meta-analysis because of the variation between trials in implementation strategy, targeted behaviour and practitioner.

RESULTS

The search yielded 4569 citations. Following exclusion based on citation and abstract review, 240 full articles were obtained. Fourteen studies (27 papers) were included.1020^–45 The main reasons for excluding studies are summarised in the Quality of Reporting of Meta-analyses (QUOROM) flow diagram (fig 1).46

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 QUOROM flow diagram.

Two of the papers reported different subsets of results from the same study.3241 It was not possible to determine if the results were independent of one another and the results in both papers were similar. Therefore, data were only included from one of the reports.32

The characteristics of the included studies are shown in the supplemantary online web only table 1. Of the 14 studies, 12 were RCTs. The non-randomised studies comprised controlled clinical 20 and controlled before and after42 designs. Eight of the studies were conducted in the USA and two each were conducted in Australia, the Netherlands and the UK. Most (eight studies) were aimed at pharmacists. The remainder focused on physiotherapists (three studies), dietitians (two studies) and speech therapists (one study).

The methodological quality of studies varied, with the proportion of quality criteria met ranging from 0/729 to 6/7.23 The median score was 3. Half of the studies used blinded assessors to measure at least some of their outcomes. Allocation to group was concealed in only five studies. In most (nine) studies there were no significant differences between the groups at baseline (web only table 1).

Table 1 summarises the types and frequency of implementation strategies used, and the results of the individual studies are summarised in the supplementary online web only table 2. Twelve studies examined the effects of professional interventions, with 10 of these using one or more educational interventions. Only three studies evaluating the effects of professional interventions reported significant between-group differences in patient outcomes.20293042 Four showed significant between-group differences in changing practitioner behaviour.22333539 In addition to providing professional interventions of education and reminders, one study used a financial intervention comprising an incentive to the provider. No significant between-group differences were found in practitioner adherence to guidelines and in most patient outcomes.29 In the study that used the organisational intervention of revising practitioner roles, significant differences were found for all practitioner behaviours and some patient outcomes. However, there were significant between-group differences at baseline for one of the two behaviours measured.37

Six studies focused on a single intervention strategy103035373942 and seven on a multi-faceted strategy.2022242529343643 One study compared both single and multi-faceted implementation strategies.32 Multi-faceted interventions were found to be no more effective than single intervention strategies.

Six studies measured both patient and process outcomes.232930323637 Three studies measured only patient outcomes102042 and five studies measured only process outcomes.2534353943 In addition, five studies performed a formal cost analysis.1020233236 The effects reported for patient outcomes were small and in favour of the intervention group in most studies. Approximately half of the studies reported a significant improvement for at least some of their patient outcome measures. Similarly, the effects reported for most of the process outcomes were small to moderate and in favour of the intervention group. These between-group differences in process measures achieved statistical significance in fewer than half of the studies. Most studies that performed a formal cost analysis found small non-significant effects.

DISCUSSION

In contrast with the research effort that has assessed the impact of guideline implementation and dissemination strategies for medical practitioners this review identified only 14 studies investigating the effects for allied health professionals, eight of these focusing on pharmacists. Most studies showed small to moderate effects with the results varying widely both within and between interventions. These findings are consistent with the findings of previous reviews of guideline implementation and dissemination in other health professions.1648

The lack of consistency of effect in these studies may be related to the intervention strategies used. A review of the literature suggests that implementation strategies are only effective if they address identified barriers to change,49 and that the effectiveness of strategies depends on the organisational context in which they are implemented.45 Our results support these findings. Only one of the included studies reported developing their implementation strategy based on identified barriers to change.22 This study was of high methodological quality and the results indicated a significant difference between the two groups in favour of the active implementation group for all measures of compliance with the guideline recommendations. In addition, simply adding more strategies without a clear rationale for the selection of the strategies does not improve effectiveness because as the current review suggests, multi-faceted interventions are no more effective than single interventions.

Recently, Michie et al50 used psychological theory to develop a framework to identify barriers to implementation of clinical guidelines and evidence-based practice. In this framework, 12 different domains of behaviour change were identified (knowledge; skills; social/professional role and identity; beliefs about capabilities; beliefs about consequences; motivation and goals; memory, attention and decision processes; environmental context and resources; social influences; emotion; behavioural regulation; nature of the behaviours). Our results show that all except four of the included studies focused on the effects of educational interventions.10373942 This assumes that the reason allied health professionals do not implement clinical guidelines is because they do not have knowledge. However, educational interventions address only one (ie, knowledge) of the 12 domains identified by Michie et al. Therefore further research into guideline implementation in allied health professions addressing identified barriers to change and using emerging theoretical frameworks for understanding professional and organisational behaviour change may assist in determining which strategies are more likely to be effective under different circumstances.

The level at which outcomes are measured and the choice of these measures needs consideration in future studies of guideline implementation in allied health. For example, although the main aim of implementing guidelines is usually to improve patient care and outcomes, five of the included studies only measured process outcomes (such as change in practitioner behaviour). Measuring outcomes at the process level assumes that a change in process leads to a concurrent change in patient outcome. This is not always true, as shown by Bekkering et al2223, who examined the effects of interactive education, discussion, feedback and reminder strategies on physiotherapists’ adherence to guideline recommendations, patient outcomes and costs. These authors found significant between-group differences in the physiotherapists’ compliance with the guideline recommendations; however, no differences were found in patient outcomes or costs. Similarly, measuring change only in patient outcome can also be problematic as this may preclude conclusions being drawn about the effectiveness of the implementation strategy in changing practitioner behaviour.

The methodological quality of the included studies varied widely. In particular, issues with concealment of allocation and contamination between groups may have resulted in overestimation or underestimation of the effect of the intervention, respectively. A cluster randomised trial is often the most appropriate study design where there is a threat of contamination between intervention and control patients.51 Although the use of this design reduces the risk of the control group receiving the intervention, an increased sample size is required as members of each cluster cannot be treated as independent. The choice of trial design in future studies of guideline implementation in allied health should take into account issues of contamination and power. At a minimum, whenever possible, designs incorporating baseline measures and allowing direct comparisons should be used.51

There are some limitations of this review, and, in light of these, the results should be interpreted with caution. The broad nature of the definition of clinical guideline used in this review, although widely used in the literature (including previous systematic reviews of guideline implementation and dissemination strategies) introduces the possibility that the authors’ interpretation may differ from other people’s interpretation.716 This may have resulted in the inclusion of studies that others may see as not fulfilling the criteria. Use of a more specific definition would ensure uniformity of inclusion across systematic reviews. Another limitation is that this review was restricted to studies published in English. Therefore, it is possible that potentially relevant studies were missed. However, it has been suggested that results of systematic reviews restricted to English language articles produce similar results to those without language restrictions.52 Finally, the taxonomy developed by the Cochrane EPOC group was developed to provide a framework for classifying the interventions use in implementation research, thereby aiding comparison between the studies. However, this taxonomy does have some limitations. In the light of the emerging emphasis on the use of theoretical frameworks for implementing change and innovation in healthcare it does not link the intervention methods to barriers identified based on these theories of change. It also does not account for discipline-specific differences in interventions. As the work of allied health practitioners differs from that of medical and nursing practitioners different implementation strategies may be required for implementation of change.

SUMMARY

This review identified 14 studies investigating the effects of guideline implementation and dissemination strategies in allied health professions. At best, the effects were small to moderate with results varying widely across studies. Allied health professionals aiming to improve the quality and safety of healthcare through the implementation of clinical guidelines need to identify specific barriers to change in their target group and develop a strategy that aims to address these barriers and is suitable for the particular organisational context. When measuring the effectiveness of these implementation strategies professionals should consider the use of both patient and process outcomes and choose outcomes that reflect their aims.