Objectives To compare the effectiveness of pharmacist medication review, with or without pharmacist prescribing, with standard care, for patients with chronic pain.
Design An exploratory randomised controlled trial.
Setting Six general practices with prescribing pharmacists in Grampian (3) and East Anglia (3).
Participants Patients on repeat prescribed pain medication (4815) were screened by general practitioners (GPs), and mailed invitations (1397). 196 were randomised and 180 (92%) completed. Exclusion criteria included: severe mental illness, terminally ill, cancer related pain, history of addiction.
Randomisation and intervention Patients were randomised using a remote telephone service to: (1) pharmacist medication review with face-to-face pharmacist prescribing; (2) pharmacist medication review with feedback to GP and no planned patient contact or (3) treatment as usual (TAU). Blinding was not possible.
Outcome measures Outcomes were the SF-12v2, the Chronic Pain Grade (CPG), the Health Utilities Index 3 and the Hospital Anxiety and Depression Scale (HADS). Outcomes were collected at 0, 3 and 6 months.
Results In the prescribing arm (n=70) two patients were excluded/nine withdrew. In the review arm (n=63) one was excluded/three withdrew. In the TAU arm (n=63) four withdrew. Compared with baseline, patients had an improved CPG in the prescribing arm, 47.7% (21/44; p=0.003) and in the review arm, 38.6% (17/44; p=0.001), but not the TAU group, 31.3% (15/48; ns). The SF-12 Physical Component Score showed no effect in the prescribing or review arms but improvement in TAU (p=0.02). The SF-12 Mental Component Score showed no effect for the prescribing or review arms and deterioration in the TAU arm (p=0.002). HADS scores improved within the prescribing arm for depression (p=0.022) and anxiety (p=0.007), between groups (p=0.022 and p=0.045, respectively).
Conclusions This is the first randomised controlled trial of pharmacist prescribing in the UK, and suggests that there may be a benefit for patients with chronic pain. A larger trial is required.
Trial registration: www.isrctn.org/ISRCTN06131530. Medical Research Council funding.
Statistics from Altmetric.com
Chronic pain (lasting >3 months) affects up to half the adult population, most of whom are primarily managed in primary care but prescribing is often suboptimal.
Pharmacists now have prescribing rights but no published research has compared the effectiveness of their prescribing with that of GPs.
The hypothesis was that pharmacist advice (with or without pharmacist prescribing) would lead to better outcomes than usual care.
The findings suggest that there may be improved pain-related outcomes for patients receiving pain-related care from a pharmacist prescriber.
A larger trial is called for.
Strengths and limitations of this study
This the first randomised controlled trial of pharmacist prescribing in the UK looking at patient reported clinical outcomes.
The study was designed as an exploratory trial so no power calculation was done.
Chronic pain (pain lasting more than 3 months) affects up to half the adult UK population, and is considered severely limiting in about 15% of cases.1 Recovery is uncommon with nearly 80% of those identified with chronic pain at baseline still reporting chronic pain 4 years later.2 It adversely affects many aspects of a person's physical and psychological health, and social and economic well-being.3–6
In the UK, most patients with chronic pain present, and are managed, in primary care.7 Although non-pharmacological treatments are available, these are accessed by few patients, with mixed success.8–10 Analgesics prescribed in primary care remain the mainstay of treatment,4 representing substantial workload and cost. Suboptimal prescribing may lead to poor pain control and other adverse patient outcomes. One study found that the most common medications involved in adverse drug reaction-related emergency admissions involved non-steroidal anti-inflammatory drugs (NSAIDs)11 which are commonly used to manage pain. Improved prescribing could result in better outcomes and remove the need for more costly, scarce, alternatives.
Pharmacists working in UK general practices are well-placed to improve pain pharmacotherapy because of their expertise in therapeutics, understanding of the polypharmacy regimens12 frequently used in chronic pain management and established relationships with other primary care colleagues. In the UK National Health Service (NHS), recent regulatory changes now allow accredited pharmacists (as well as some other healthcare professionals such as nurses) to prescribe prescription-only medicines.13 Pharmacists can either be qualified as supplementary prescribers, in which case they operate within an agreed clinical management plan (CMP) in partnership with the doctor and patient, or as an independent prescriber, in which case they can either prescribe completely independently or within a CMP.
However, despite the increasing number of non-medical prescribers, including pharmacists, there has been no rigourous comparisons of the outcomes of non-medical versus general practitioner (GP) prescribing. This information is needed to assess the clinical effectiveness of different care models.
This paper reports findings from an exploratory randomised controlled trial (RCT) comparing pharmacist medication review, with or without pharmacist prescribing, with standard care for patients with chronic pain. Development of the trial was informed by earlier feasibility work.14 ,15
The a priori hypothesis was that, in patients with chronic pain, pharmacist advice (with or without pharmacist prescribing) would lead to better patient functioning and/or better pain control at 6 months, than treatment as usual (TAU).
Ethical approval was granted by the National Research Ethics Service Committee—North of Scotland (reference number 09/S0801/107). NHS Research and Development approval was granted by NHS Grampian and East Norfolk & Waveney Research Governance Committees. Patients gave informed consent before taking part.
An open, exploratory RCT in which patients were randomised to one of the three study arms. Participants were not blind to allocated treatment arm due to the nature of the intervention.
Recruitment of practices and independent prescribing pharmacists
Practices in the Grampian Health Board area, Scotland (n=18) and East Anglia region of England (n=4) known to have an attached Royal Pharmaceutical Society of Great Britain registered independent pharmacist prescriber, were eligible to take part. From those indicating a willingness to participate, convenience sampling was used to identify six general practices: three in Grampian and three in East Anglia.
Patient inclusion and exclusion criteria
Patients registered with the recruited practices were eligible for inclusion if they were over 18 years of age, living in their own houses and receiving regular prescribed medication for pain. Patients were identified by a computerised search14 of the drug records of all individuals registered with the practice, to identify those who had received either two or more acute prescriptions, and/or one repeat prescription within the last 120 days, for an analgesic (British National Formulary (BNF section 4.7) and/or NSAID (BNF section 10.1.1). Medications that can be used for analgesia but whose primary indication is not chronic pain (eg triptans, antiepileptics or antidepressants) were excluded as these drugs identify few additional eligible patients.16 In accordance with trial criteria, GPs excluded and recorded reasons for patients who had: a concomitant severe mental health problem or terminal illness; had suffered recent bereavement; had a known alcohol or drug addiction; suffered pain caused by cancer or other malignancy; were unable to give informed consent; other (unspecified) reasons.
Eligible patients were sent an invitation pack (letter, information sheet, consent form) by practice staff between March and June 2010. Consent forms were returned directly to the researchers, who sent out a baseline questionnaire. Patients returning completed questionnaires were randomised by the researcher using a telephone randomisation service with a random number allocation which ensured allocation concealment. The allocation sequence was 1 : 1 : 1.
All participating pharmacists took part in a 2-day course updating them about pain management. As part of the training, participants defined and agreed the treatment algorithm they would all use.
Pharmacists invited patients to a face-to-face consultation. Prior to the consultation, pharmacists completed a paper-based medication review of each patient's medical record and patients were asked to complete a pain diary to inform the consultation. A pharmaceutical care plan was agreed between the pharmacist and the patient. The plan assessed and documented relevant medical history and current conditions; known allergies and adverse drug reactions; relevant laboratory results; pain-related medications prescribed in the previous 10 years; current pain-related prescription medications; current symptoms; lifestyle issues, including units of alcohol consumed per week; recommendations for changes to medication (if any); whether non-pharmaceutical treatments had been considered; and any other relevant issues. Copies of the pain diary and pharmaceutical care plan are available from the authors on request. At the end of the consultation any required prescriptions for medicines were issued by the pharmacist. Owing to Controlled Drug (CD) regulations in place at the time, prescribing for CDs was performed using a supplementary prescribing CMP,17 rather than independent prescribing. Patients were followed up either by phone or face-to-face, at each pharmacist's discretion.
‘Review arm’: The pharmacists conducted a paper-based medication review focussed on pain-related prescription medications, before creating a pharmaceutical care plan which detailed any recommendations for medication changes. The plan was passed to the patient's GP for implementation. The GPs were asked subsequently about actions taken as a result of the recommendations.
Treatment as usual: Patients received standard general practice care.
A core aim of this exploratory RCT was to finalise the selection of outcome measures for a subsequent multicentred RCT. In the Current Controlled Trials Registration (ISRCTNO6131530) we specified both primary and secondary outcome measures (primary: SF12, Health Utilities Index (HUI); secondary: chronic pain grade (CPG), HADS) based on our judgement following the earlier feasibility study.15 However in practice, all outcomes were considered equal and no single measure was defined as the primary outcome, for example, for the purpose of a sample size calculation (see below). These four outcome measures are described below. Inclusion of both pain-specific and generic outcome measures was based on Initiative on Methods, Measurement and Pain Assessment in Clinical Trials (IMMPACT) recommendations18 and an earlier feasibility study.15
The SF-12v2 is a generic health and functioning scale,19 previously used in population-based studies of pain.20 ,21 A Physical (PCS) and Mental Component Score (MCS) were calculated, ranging from 0 to 100; a higher score indicates better functioning.
The HUI3 is a preference-based system for measuring comprehensive health status and health-related quality of life (HRQL).22 It provides descriptive evidence and a score for each dimension of health (vision, hearing, speech, ambulation/mobility, pain, dexterity, self-care, emotion and cognition) and an HRQL score for overall health. Each dimension has 3–6 levels. Owing to the cost of the additional license fee to score data from this measure, this instrument was not subsequently analysed.
The CPG23 is a seven-item scale which assesses pain severity on two dimensions: disability and intensity. The scale classifies pain according to the level of intensity and disability (I (low disability–low intensity) to IV (high disability–severely limiting)).
The Hospital Anxiety and Depression Scale (HADS)24 is a 14-item screening instrument which identifies the possible and probable caseness of anxiety (7 items (HADS-A)) and depression (7 items (HADS-D)); each item scored from 0 (not present) to 3 (highly present). Standard thresholds and previously used labels25 were applied: no depression/anxiety (0–7), mild (8–10), moderate (11–15) or severe (>15)).
Questionnaires were posted to participants at baseline (pre-randomisation) and 3 and 6 months post-randomisation (follow-up was conducted between July 2010 and January 2011). Up to two reminders were sent. Questionnaire content included the outcome measures described earlier together with items on: demographic status (baseline only); screening items to confirm eligibility (baseline only); duration of pain condition (baseline only); location of pain; Morisky Medication Adherence Scale 4 (MMAS-4);26 participant satisfaction (11 statements derived from the feasibility study for the prescribing arm (3 months only)) and additional comments by participants. The MMAS-4 provides a score of self-reported adherence to medication regimen. Scores range from 0 (low adherence) to 4 (high adherence).
Follow-up interviews with staff
Post-intervention, all pharmacists and all GPs in participating practices were invited to take part in semistructured interviews, carried out face-to-face when possible, otherwise by telephone. Interviews were taped, transcribed verbatim and content analysis was carried out.
As this was an exploratory trial to estimate the effect size for a larger trial, no formal sample size calculation was possible.27 We aimed to recruit 30 participants per practice (n=180; with an additional six per practice for training purposes, that is, 216 in total). This was deemed sufficient to give reliable effect size estimates for the outcome measures of health status or CPG.
Data management and analysis
Data were entered into identical SPSS databases at each site and accuracy checks carried out on 10% before databases were merged. Descriptive statistics included means and SD for normally distributed continuous data, medians (IQR) for skewed continuous data and percentages (n) for categorical data. Analysis was conducted on an intention-to-treat basis for participants with complete data on relevant measures using SPSS V.18.
Exploratory analyses for parametric data included the paired t-test for within-arm comparisons of mean difference between baseline and 6 months and one-way analysis of varience for between arm comparisons of mean difference. For non-parametric data it included the Wilcoxon signed rank test for within-arm comparisons of median difference and the Kruskal-Wallis test for between arm comparisons of median difference. Categorical data were analysed using the marginal homogeneity test for within-arm comparisons (with null hypotheis that the distribution of CPG grade or HADS group does not change between baseline and 6 month follow-up) and the χ² test for between-arm comparisons; analyses reported here are based on 6 month follow-up data (other than for participant experiences). Within-arm effect sizes, expressed in terms of a Pearson correlation coefficient (r) have been calculated using the formulas from Rosenthal.28 Effect sizes can be directly compared using Cohen's29 criteria of r=0.1 (small effect); r=0.3 (medium effect) and r=0.5 (large effect).
Response rates and demography
Six of the seven practices approached participated. GPs excluded 12% (392/3281) of patients, mostly those with dementia. There was no statistically significant difference between participants and non-participants in terms of age, gender and index of multiple deprivation. Figure 1 shows the flow of participants through the study. Overall, the consent rate was 25% (356/1397) and the recruitment rate was 14% (196/1397).
Eighty-six per cent of participants (251/289) returned baseline questionnaires, of whom 232 were randomised (36 participants were randomised to one of the two intervention arms for training purposes and were not included in any further analysis and 19 were not included as the recruitment target had been met). The overall follow-up rate at 3 months was 86% (161/187) and at 6 months 84% (152/180).
As shown in table 1, groups were similar at baseline for demographic and socioeconomic variables and pain data. Most participants were married, Caucasian and female, older (mean (SD) age 65 (12.6) years), had an annual income of <£25 000 and had suffered from pain for at least 5 years. Most (57%; 103/181) reported being fully adherent to their medication regimen (MMAS-4, median 4.0 (IQR 3.0–4.0); 15 missing MMAS scores).
In the prescribing arm, 78% (53/68) attended an initial prescribing consultation, 31 had at least one planned follow-up (of which 34/37 were conducted by phone) and 130 recommendations were made for 92% (49/53) of participants seen. Examples are shown in box 1. The median time taken for the note-based record review was 35 min (IQR 20.0, 45.0), the consultation was 30 min (IQR 20.0, 40.0), careplan preparation 10 min (IQR 10.0, 20.0) and median duration of follow-ups was 10 min (IQR 5.0–15.0).
Examples of pharmacist interventions in the prescribing arm
Changes to pain management: ‘use paracetamol regularly’, ‘take tramadol if needed’ ‘add piroxicam gel PRN’, ‘given web links to self help groups’
Compliance aid: ‘gave written times that this drug could be taken’
Addressing side effects/safety: ‘take paracetamol after initial NSAID’, ‘take senna’, ‘ordered blood monitoring’, ‘stop use of two NSAIDS’
General health: ‘discussed weight loss’, ‘invited to practice nurse for BP’, ‘glucose, lipids and lifestyle update’
Cost minimisation: ‘change aspirin EC to plain’.
In the review arm 97% (60/62) of participants’ records were reviewed (note there was one postrandomisation exclusion) for whom 197 recommendations were made. Where GP feedback was provided (n=48), they generally agreed with pharmacists’ recommendations, which were fully implemented for 20 participants (2 by the pharmacist following request by GP), partially for 19 participants and not at all for nine participants. The median time taken for the note-based record review was 30 min (IQR 24.3, 45.0), and careplan preparation was 10 min (IQR 5.0, 20.0).
Clinical outcome measures
In the prescribing arm, there was a statistically significant within arm improvement for the CPG intensity (p=0.002, effect size (r)=0.45) and disability (p=0.003, effect size (r)=0.43) subscales and between arms on the intensity subscale (p=0.02), but not the disability subscale (p=0.55; table 2). There was a significant within-arm improvement in overall CPG grade in the prescribing (p=0.003) and review arm (p=0.001), but not in the TAU arm. The SF-12 PCS showed a statistically significant within arm improvement in the TAU arm (p=0.02, effect size (r)=0.35; table 3), but not between trial arms. The SF-12 MCS showed a statistically significant deterioration in the TAU arm (p=0.002, effect size (r)=0.45; (table 3), as did the HADS-D (p=0.03, table 4). Analysis was also carried out on the non-categorised HADS scores which showed a statistically significant improvement within the prescribing arm for Depression (p=0.022) and Anxiety (p=0.007). These were both significant between groups (p=0.022 and p=0.045, respectively; table 5).
Acceptability of the pharmacist prescribing intervention
All six pharmacists and 56% of the GPs (23/41) were interviewed. All pharmacists and most GPs were positive about the intervention, although some GPs suggested that the pharmacists’ recommendations had been minor and questioned the cost-effectiveness of the service. Patient participants were generally positive about the pharmacist prescribing service although some concerns were identified, as illustrated by the quotes shown in box 2.
Examples of quotes from pharmacists (n=6), general practitioners (GPs) (n=23) and patient participants (n=40) on the prescribing intervention
Pharmacists (from interviews):
Satisfying (n=6):‘contact with patients’, ‘being able to help patients’, ‘being able to make a difference to long-standing pain’...'even in small ways’
Interesting (n=6):‘learning about pain’
Challenging (n=6):‘complex, chronically ill patients’
GPs (from interviews):
Support for the service (n=17): it's been a very positive thing’
Agreement with pharmacists’ recommendations (n=23): ‘oh very reasonable suggestions’, ‘tinkering round the edges’, ‘had been tried already’.
Trust in the practice pharmacist (n=23):‘I respect his professional judgement’
Cost effectiveness (n=6): ‘if there's limited resources do we want to spend the money on a pharmacist’.
Patients (from 3-month questionnaire):
Proportion agreeing that:
The pharmacist was interested in them (89%; 39/44)
They were totally satisfied (85%; 39/46)
They were told about their treatment (82%; 38/46)
Their consultation was thorough (79%; 34/44)
They would have liked more time (9%;4/44)
They would have preferred to see their GP (9%; 4/44)
Too many people were now involved in their treatment (11%; 5/44).
Open text questions:
Positive (n=39): ‘She was professional, relaxed, pleasant and interested. Excellent!’
Negative (n=1): ‘A waste of time, altered my tablets which made my pain worse’.
This exploratory RCT of pharmacist-led management of patients with chronic pain suggests that pharmacist prescribing (and possibly pharmacist review alone) may be effective in improving pain-related outcomes and be acceptable to both patients and most professionals. There was an indication of a positive effect on emotional health, but no measurable effect on general health.
Strengths and weaknesses
This was the first RCT to assess clinical and humanistic outcomes after pharmacist prescribing for any clinical condition compared with usual GP care, and the first RCT to specifically assess pharmacist-led management of chronic pain, compared with usual GP care. It was based on extensive development and feasibility work14 ,15 in line with MRC framework for development and evaluation of complex interventions.30 A range of validated outcome measures was included, as well as a parallel qualitative process evaluation which assessed satisfaction and acceptability. The inclusion of six practices and their associated pharmacists from both Scotland and England increased the generalisability of the findings. Pharmacists received formal training and agreed and used a common treatment algorithm which should have increased standardisation of treatment. The preponderance of women (overall 62%) and average age of 65 years reflects the wider chronic pain population1 as does the distribution of pain site.31 ,32
There were, however limitations. Although high follow-up response rates were achieved at both 3 (86%) and 6 months (85%) only 25% of eligible patients entered the trial. This low initial consent rate is in line with other studies,33 ,34 but may cause unknown biases including problems of generalisability, as does the solely Caucasian ethnicity. Concerns identified by participants during the formal feedback, for example, having too many people involved in one's care may have contributed to poor response rates and rewording of participant recruitment documentation to reassure participants of the role of the pharmacist could address this. More participants withdrew in the prescribing arm compared with the other two arms, which might be attributed to the need for an additional practice visit. The study was an exploratory trial so no formal power calculation was undertaken. However, because there were no published MIDs available to estimate effect size for the outcomes in this population, it was important to present the actual clinical magnitude of change in outcome at 6 months alongside a statistical assessment of this change (p value). This allows an assessment of both clinical and statistical significance simultaneously with the caveat that this is an exploratory study. With around 50 patients per arm, this was deemed sufficient numbers to examine the change in outcome measures with appropriate within-group and between-group univariate statistical tests. Owing to the nature of the intervention, no participants were blind to their group allocation, and so some outcomes, especially the qualitative components, may have been affected by social desirability bias.
Our outcome measures were self-reported, but this is the norm in pain studies as pain is a subjective experience.18 Furthermore we do not know how important the observed differences were to participants. Following precedents set in previous research,25 and because there is no consensus on an alternative measure35 we used the HADS as a tool to classify people by severity of depression and anxiety. However it is strictly a screening tool, and the four levels of severity have not been formally validated. We therefore also compared outcomes using it as a continuous scale.
Relationship with other studies
This study is important because no other RCT has evaluated pharmacist prescribing and few studies, and importantly no RCTs, have evaluated pharmacist interventions for pain. In pharmacist prescribing most research has focused on reported experiences of professionals and patients, and not used validated outcome measures. Yet pharmacist prescribing is now widely practised. For pain, there have been a few small studies. Briggs et al36 conducted a small before-and-after evaluation (involving 65 patients) of a nurse and pharmacist-led chronic pain clinic in primary care. Pain intensity visual analogue scale scores reduced significantly over 6 months. Another evaluation of 26 patients using a medication review service provided jointly by a physiotherapist and pharmacist in the UK, reported improvement in pain control for 88% of patients.37
The CPG was found to show a graded effect across the three arms, showing discrimination with both direction and strength of improvement, suggesting maximum benefit for those in the pharmacist prescribing arm. However, the reduction in overall score appears to be mediated by a change in the intensity of pain subscale rather than in pain-related disability. The effect size of 0.45 suggests this could be an important difference. In contrast, the SF-12, a measure of general health and functionality showed no significant difference between intervention arms, reflecting either no effect or or lack of power to detect an effect.
While most participants in this study were already within the normal range on the HADS scale, and therefore had minimal chance of improvement, there were nonetheless suggestions of better ourcomes in participants in the prescribing arm. Including a range of instruments is in line with IMMPACT recommendations,38 which state that focus should be on the whole person, not just about pain. However, this needs to be balanced with minimising participant burden.
Explanations, implications and future research
The number of pharmacist’s recommendations per participant was higher in the review arm than in the prescribing arm. This might seem contradictory to the possible greater benefit found in the prescribing arm. However, in the prescribing arm pharmacists met the participant and may have more readily identified and dismissed suggestions previously tried. The interview feedback highlighted that some recommendations for change, while sensible, had been tried already. This might also be the reason why there were only 60% of pharmacist recommendations with which the GP fully agreed. Self-reported adherence to medication at baseline was good. Despite this, the pharmacists still improved pain outcomes in the prescribing arm. This could have been due to changes in medications and/or participant education about optimal timing for administration of analgesic medicines. Further research is needed to confirm the beneficial effect of pharmacist prescribing and its sustainability.
Our results suggest that pharmacist prescribing (and possibly pharmacist review alone) for patients with chronic pain is feasible, acceptable and may lead to improvements in pain and other measures. A larger fully powered trial is now needed to confirm these findings.
We thank the participating patients, practices and pharmacists. We would also like to thank Annie Blyth who was the lead researcher in East Anglia, Kirsten Harrild (Medical Statistics, UOA) for statistical support. Rick Adams (School of Pharmacy, UEA) helped design and deliver the pharmacist training and Lesley Thomson (NHS Grampian) helped design the pharmacist data collection forms. The patient postal questionnaire was based on work by Nicola Cooper and the Norfolk Arthritis Register (NOAR) research team. The Pharmacy-Led Management of Chronic Pain Study Team acknowledges the financial support of NHS Research Scotland (NRS), through Scottish Primary Care Research Network Northeast. The work was conducted as part of the Aberdeen Pain Research Collaboration. All authors had access to all of the study data and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Contributors CMB was the chief investigator, conceived the study and led the writing of proposal for funding, provided day to supervision of study conduct, analysis and interpretation of data and co-led the writing of the paper, including revisions and final draft. AME, PCH, RH, AJL, PM, BHS, DW and MW were coinvestigators, contributed to writing of the proposal, oversight of the study conduct, data analysis and interpretation, drafting of the paper and approved the final version. PM had particular responsibility for the health economic components and AJL for the statistical analyses. BHS and AME provided specific advice on chronic pain and it measurement and DW expertise on the design and delivery of the pharmacist training. HB was the research fellow, coordinated the two study sites and was lead researcher for Grampian. She was responsible for daily study conduct and coordination, acquisition of the data, analysis, producing tables and figures and interpretation of data. She co-led the drafting of the paper with CMB and approved the final version.
Funding The project was funded by the Medical Research Council (grant ID: 85356). They had no further involvement in any aspect of study conduct; all researchers were independent of the funding body.
Competing interests None.
Ethics approval North of Scotland.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Consent was not obtained from participants for data sharing and so cannot be released to third parties. We welcome, however, opportunities to work collaboratively with other investigators to use the data for further analyses.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.