Table 1

Effectiveness of toolkits in strongly and moderately rated studies (N=8)

	Study design, participants	Toolkit and intervention	Evidence informing toolkit development	Outcomes measured	Results	Quality
Cavanaugh et al26	Two RCTs at 2 academic medical centres N=198 adult patients with diabetes n=99 (control) n=99 (intervention) Country: USA	Intervention components: enhanced diabetes care programme; training sessions; DLNET Control components: enhanced diabetes care programme Toolkit target: health professionals Toolkit contents: customisable 24 instructive modules about diabetes self-management activities, including blood glucose monitoring, nutrition management, foot care, administration of medications	Incorporated communication principles	Glycaemic control (A1c) Patient-reported self-efficacy of diabetes self-management Self-management behaviours Treatment satisfaction Sustainability	Significant improvements in A1c levels in intervention and control groups at 3 months (adjusted analyses showed greater improvement in the intervention group (p=0.03)) Significant improvement in self-efficacy from baseline in both groups (p=0.01,0.02) (but NS differences between groups in adjusted analyses) NS differences between intervention and control for self-management behaviour or treatment satisfaction NS differences between intervention and control groups at 6 months Implementation outcomes: sustainability of outcomes measured at 3 and 6 months Toolkit effectiveness: partially effective	M
Dykes et al 30	Cluster RCT N=8 units in 4 urban US hospitals, N=10 264 patients n=5160 (intervention) n=5104 (control) Country: USA	Intervention components: FPTK; local champions Control components: usual care Toolkit target: health professionals Toolkit contents: Morse Falls Scale to assess fall risk; interventions tailored to patient-specific areas of risk; bed poster, patient/family education handout, fall prevention plan (tailored for each patient)	Literature review, focus groups with nurses+nursing assistants, assessment of barriers and facilitators to optimal practice	Patient falls per 1000 patient-days Fall-related injuries Fidelity	Significantly fewer patients with falls in intervention versus control units (p=0.02) Significantly lower adjusted fall rates in intervention versus control units per 1000 patient-days (p=0.04) NS difference in fall-related injuries Implementation outcomes: protocol adherence >89% Toolkit effectiveness: mostly effective	M
Majumdar et al42	Controlled clinical trial N=14 managed care practices Country: USA	Low-intensity intervention components: evidence-based guideline on Helicobacter pylori; Toolkit High-intensity intervention components: evidence-based guideline on H. pylori; Toolkit; academic detailing of guideline dissemination by a PCP champion using persuasive educational session; 1 month reinforcement of guideline message; and reminder about eligible patients by a pharmacist Control components: usual care Toolkit target: health professionals Toolkit contents: customised list of eligible patients from participating practice; educational materials for patients; patient letters used to arrange for test or follow-up appointment; pre-printed materials including: (A) H. pylori serology test requisitions, (B) preapproved prescriptions, (C) progress notes for patient charts	Not specified	Rate of testing Rate of continued use of acid suppressing medications Sustainability	Significant increase in H. pylori test-ordering in high-intensity intervention versus usual care at 12 months (p=0.02)Significant decrease in proton pump inhibitor use by 9% per year in high-intensity intervention versus usual care (p=0.028) Implementation outcomes: sustainability of outcomes measured at 12 months Toolkit effectiveness: mostly effective	M
Menchetti et al44	Cluster RCT N=15 primary care groups with 223 PCPs n=8 intervention (128 patients) n=7 control (99 patients) Country: Italy	Intervention components: 2-day intensive training for PCPs; implementation of a stepped care protocol; dedicated consultant psychiatrist; Depression Management Toolkit Control components: usual care Toolkit target: PCPs Toolkit contents: issues discussed during training with PCPs; diagnostic procedure based on the PHQ-9; treatment algorithm	Based on training program developed by project steering committee	Clinical remission of depression Treatment response PCP behaviour Sustainability	NS differences between groups in remission of depression at 3, 6, 12 months; however in patients with minor/major depression, intervention was more effective than usual care at 3 months (p=0.015). Intervention group showed significantly higher treatment response rates at 3 (p=0.016) and 6 months (p=0.049). PCP increased use of appropriate antidepressants and decreased use of sedatives, hypnotics at 3 months. Implementation outcomes: sustainability of outcomes measured at 3, 6, 12 months Toolkit effectiveness: partially effective	M
Shah et al51	Pragmatic Cluster RCT n=933 789 adult patients with diabetes (administrative data study) n=1592 patients with diabetes at high risk for cardiovascular disease (clinical data study) Country: Canada	Intervention components: toolkit Control components: usual care Toolkit target: family physicians Toolkit contents: introductory letter; tailored 8-page summary of practice guidelines; 4-page synopsis of key guideline elements pertaining to cardiovascular disease risk; small laminated card with simplified algorithm for cardiovascular risk assessment, vascular protection strategies+screening for cardiovascular disease; tear-off sheets for patients with a cardiovascular self-assessment tool; list of recommended risk reduction strategies. Toolkit was packaged in a brightly coloured box with Canadian Diabetic Association branding)	Clinical experts (family physicians, endocrinologists), clinicians with KT expertise	Composite end point of death or non-fatal MI Processes of care	NS difference between groups in death or non-fatal MI (p=0.07) and use of a statin (p=0.26). Decreased use of ECG (p=0.02) and cardiac stress tests (p=0.04) in intervention group Implementation outcomes: not specified Toolkit effectiveness: not effective	M
Wright et al57	Cluster RCT N=42 Ontario hospitals (616 patients with stage II colon cancer) Country: Canada	Intervention components: standardised lecture from expert opinion leader; toolkit; academic detailing of local opinion leader; 6-month follow-up reminder package Control components: standardised lecture from expert opinion leader Toolkit target: physicians Toolkit contents: pathology template; poster and pocket cards emphasising 12 LNs to be assessed in colon cancer	Not specified	Mean number of LNs assessed Proportion of cases staged with a minimum of 12 LNs before and after intervention	Significant increase in mean number of LNs assessed and the proportion of cases with 12 or more LNs retrieved for both groups after standardised lecture (p<0.001). No additional increase noted with academic detailing and toolkit. Implementation outcomes: not specified Toolkit effectiveness: not effective	S
Goeppinger et al33	4 months RCT and 9 months longitudinal study N=921 adults with osteoarthritis, rheumatoid arthritis, fibromyalgia or chronic joint symptoms n=463 (control), n=458 (intervention) Country: USA	Intervention components: Arthritis Self-Management Toolkit Control components: no intervention Toolkit target: patients with arthritic conditions Toolkits contents (available in English and Spanish): ‘self-test’ to self-tailor the toolkit; information sheets on arthritic-related health issues and on key process components of the Arthritis Self-Management Program (eg, decision-making); Arthritis Help Book; audio relaxation and exercise CDs; audio CD of all material from information sheets	Not specified	Health status Health-related behaviours Medical care utilisation Perceived self-efficacy Sustainability Adoption and appropriateness of toolkit	Statistically significant improvement in 6/7 health status measures, all health-related behaviours and self-efficacy but not in medical care utilisation variables at 4 months postintervention (p<0.01) in intervention versus control groups Results maintained at 9 months compared with baseline Implementation outcomes: sustainability of outcomes measured at 9 months 97% of participants reported use of the toolkit and found it useful. The Book was rated the most useful part Toolkit effectiveness: mostly effective	S
Horvath et al36	RCT N=108 dyads of patients with progressive dementia of Alzheimer's type/caregiver n=48(control) n=60 (intervention) Country: USA	Intervention components: Home Safety Toolkit Control components: 1 page standard patient education sheet Toolkit target: caregivers of patients with Alzheimer's Toolkit contents: ‘Keep the Home Safe for a Person with Memory Loss’ booklet; low-cost sample items to reduce risky behaviours and accidents	Principles of health literacy, patient-centred care and self-efficacy	Caregiver self-efficacy Caregiver strain Home safety Risky behaviours and accidents Fidelity Costs	Significantly higher caregiver self-efficacy (p=0.002), significantly lower caregiver strain (p≤0.001), significant improvement in home safety (p≤0.001), significantly fewer risky behaviours and accidents (p≤0.001) in intervention versus control Implementation outcomes: fidelity to protocol achieved Cost of toolkit included but not a cost/benefit analysis Toolkit effectiveness: mostly effective	S

DLNET, Diabetes Literacy Numeracy Education Toolkit; FPTK, Fall Prevention Toolkit; LN, lymph node; M, moderate; NS, non significant; PCP, primary care physician; PHQ-9, Patient Health Questionnaire-9; RCT, randomised controlled trial; S, strong.