Introduction Adherence to oral anticoagulation (OAC) treatment, vitamin K antagonists or new oral anticoagulants, is an essential element for effectiveness. Information on adherence to OAC in atrial fibrillation (AF) and the impact of adherence on clinical outcomes using real-world data barely exists. We aim to describe the patterns of adherence to OAC over time in patients with AF, estimate the associated factors and their impact on clinical events, and assess the same issues with conventional measures of primary and secondary adherence—proportion of days covered (PDC) and persistence—in routine clinical practice.
Methods and analysis This is a population-based retrospective cohort study including all patients with AF treated with OAC from 2010 to date in Valencia, Spain; data will be obtained from diverse electronic records of the Valencia Health Agency. Primary outcome measure: adherence trajectories. Secondary outcomes: (1) primary non-adherence; (2) secondary adherence: (a) PDC, (b) persistence. Clinical outcomes: hospitalisation for haemorrhagic or thromboembolic events and death during follow-up. Analysis: (1) description of baseline characteristics, adherence patterns (trajectory models or latent class growth analysis models) and conventional adherence measures; (2) logistic or Cox multivariate regression models, to assess the associations between adherence measures and the covariates, and logistic multinomial regression models, to identify characteristics associated with each trajectory; (3) Cox proportional hazard models, to assess the relationship between adherence and clinical outcomes, with propensity score adjustment applied to further control for potential confounders; (4) to estimate the importance of different healthcare levels in the variations of adherence, logistic or Cox multilevel regression models.
Ethics and dissemination This study has been approved by the corresponding Clinical Research Ethics Committee. We plan to disseminate the project's findings through peer-reviewed publications and presentations at relevant health conferences. Policy reports will also be prepared in order to promote the translation of our findings into policy and clinical practice.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This is a population-based study using real-world data to assess adherence to oral anticoagulation (OACs) and its association with effectiveness and safety outcomes.
The study considers information on what physicians prescribe, and also on what patients fill from the pharmacy.
The study considers multiple indicators of adherence, including group-based trajectory patterns, taking into account the dynamic nature of adherence.
The observational nature of the study might lead to selection bias and confounding. Propensity scores will be used to address this issue.
Results on use and adherence to OACs might not be generalisable to other settings.
Atrial fibrillation (AF), the most common sustained arrhythmia, favours embolic stroke, which is one of the leading causes of cerebrovascular morbidity, neurological disability, quality of life loss and death.1 ,2 Prevalence in population-based studies in industrialised countries is 6.6 men and 3.9 women for every 1000 people of the respective gender, with a strong age gradient.3
Several randomised clinical trials (RCTs) have shown that preventive treatment with vitamin K antagonists (VKA) such as warfarin is highly effective, decreasing the incidence of cardioembolic stroke in patients with AF by approximately two-thirds, and thus reducing deaths and improving quality of life.4–8 Based on this evidence, low-intensity VKA therapy to maintain the international normalised ratio (INR) between 2.0 and 3.0 has, for many years, been the standard treatment for patients with AF at the highest risk of a stroke.9 ,10
In recent years, however, new (non-VKA) oral anticoagulants (NOACs), such as dabigatran, rivaroxaban or apixaban, have been marketed. Their respective pivotal phase 3 trials reported similar or better thromboembolic event rates than warfarin and rates of haemorrhagic events similar to or less than those of warfarin.11–13 Several systematic reviews and meta-analyses confirm these results.14–19 Although some characteristics may favour one NOAC over another, direct comparisons are not available, and comparative effectiveness and safety remain unsatisfactorily tested.20 Several indirect comparisons have been performed showing conflicting results,21–32 and the validity of the conclusions from these studies is hindered by multiple methodological problems.33
The use of VKAs is associated with an increased risk of bleeding, regular blood monitoring, and drug–drug and drug–food interactions, and often imposes lifestyle changes. These factors lead to non-adherence, discontinuation of treatment and difficulties maintaining an optimal INR.34 Non-adherence and discontinuation of anticoagulant therapy leads to increased ischaemic stroke risk and contributes to suboptimal outcomes of the anticoagulant treatment.35–37 Owing to the scarcity of interactions, predictable effects with fixed dosages and the lack of need for INR monitoring, NOACs have some advantages over VKAs regarding comfort and convenience. However, they also have significant limitations, such as the unavailability of tests for monitoring their anticoagulant effectiveness, the absence of antidotes to reverse their effect, their renal clearance, some potential adverse effects and their higher cost (the cost per day of treatment with NOACs far outweighs that of VKAs, including monitoring costs). Furthermore, whether or not the advantages of NOACs may also translate into increased adherence and persistence is unclear. In fact, there is rising concern regarding poor adherence with these newer anticoagulant agents.38–40 In RCT studies, NOACs exhibit a persistence similar to that of warfarin, with only marginal differences.41 In the NOAC pivotal trials, permanent discontinuation stood between 21% and 25%,11–13 figures that were not significantly different from their controls with warfarin (varying between 17% and 27%), with median or mean follow-up ranging between 1.8 and 2 years. In real-world studies, and as expected since the conditions in trials are not generalisable to routine clinical practice, the warfarin persistence rates are lower than in clinical trials.42–51 Regarding NOACs, real-world data on persistence and adherence are sparse and heterogeneous,50 ,52–54 and their association with effectiveness and safety outcomes has not been comprehensively examined in clinical practice; this association has been examined even less in settings such as Spain, where data may diverge considerably from those in the countries that participated in the trials. This is a relevant issue because patient characteristics, health system organisation and sociocultural contexts are significant determinants of adherence, and generalising drug adherence or persistence figures from one country to another may be an unreliable extrapolation. In the context of the Spanish National Health Service, information on adherence to anticoagulation therapy in AF and the impact of adherence on clinical outcomes barely exists.
At present, and although many factors have been shown to be strongly associated with discontinuation or non-adherence,55 our ability to distinguish or characterise non-adherent patients is poor. Additionally, adherence has been assessed in diverse ways, many of which are not standardised or replicable (eg, some studies confuse physician non-adherence to prescription guidelines with patient non-adherence to prescribed treatments, or use very different criteria for defining non-adherent patients, or mix naïve-drug cohorts with experienced-drug cohorts). Furthermore, not all studies use the same definitions regarding clinical outcomes (eg, some studies consider that the patient has discontinued treatment when switching from one anticoagulant to another).
Currently, the information available in electronic medical records enables the construction of observational cohorts for measuring adherence and persistence and its predictors, and for the assessment of their impact on clinical outcomes.56 ,57 However, those studies based on electronic data sets usually share three important limitations: (1) they are based on limited groups—such as those affiliated to a particular insurance company or a pharmaceutical benefit scheme—which are not necessarily representative of the general population, (2) they use pharmacy claims data with no information about physician prescription and, therefore, they combine physicians’ failure to prescribe with patients’ failure to refill the medication, and (3) they classify patients into groups of adherence using single indicators, overlooking the dynamic nature of non-adherence over time.
The use of advanced electronic prescription systems may be useful for addressing some of these drawbacks. The Valencia Health Agency (VHA), the public service responsible for healthcare in the Autonomous Community of Valencia (Spain), works under a scheme of universal coverage and tax-based funding. An extensive network of hospitals and primary healthcare centres are operated by the VHA, sharing an electronic medical record with an advanced electronic prescription system that includes the traceability of the prescription (from prescription by the doctor to the dispensing of the drug in pharmacies and subsequent claims to the VHA). These characteristics improve the representativeness of the population studied, and help differentiate between gaps in non-adherence owing to physicians’ failure to prescribe from those related to the patients’ failure to refill the medication.
Regarding the fact that adherence changes over time and that this aspect has been neglected in most existing studies, new methodological approaches such as group-based trajectory models (GBTMs), a type of latent class analysis that incorporates information on the dynamic nature of adherence for its assessment, may also add some insights into the study of medication non-adherence.58 These alternative models estimate the change over time in an endpoint that is measured repeatedly, facilitating the qualitative identification of different sequences that define underlying subpopulations. GBTM is a person-centred approach (as is cluster analysis) focused on the relationships among individuals.59 The most important outputs of the GBTM are the classification of patients into different trajectories over time (eg, nearly-always adherent patients, early gap in adherence after discharge with later recovery, brief gaps in medication use or occasional users, slow decline adherence, fast decline) and the description of such trajectories through easily interpretable graphics.
Although GBTMs have been widely used in some areas of medical and sociological research, and have an important potential to classify patients according to their long-term adherence,57 their use in studies of medication adherence is scarce.57 ,60–64 Therefore, the availability of the aforementioned population-based electronic data sets and the use of new methodological approaches offer a great opportunity for assessing adherence to oral anticoagulation in a more comprehensive manner.
In summary, in spite of great efforts made in the scientific community and professional societies through clear guideline recommendations and quality of care policies based on solid evidence, underprescribing and low adherence to anticoagulation therapies remain a significant challenge in the management of patients suffering from AF, either because physicians fail to prescribe these medications and/or because patients fail to obtain and/or use them. There are relatively few studies reporting the conventional measures of adherence (primary, persistence, percentage of days covered, etc) to VKA, and even fewer to NOAC, and none of them classify patients according to their adherence behaviour over time. Identifying adherence patterns, their associated factors and their relationship with clinical outcomes, would improve the ability of medical organisations to advance and focus more precisely on policies and interventions to improve their management.
The main objective of the study is to describe the adherence trajectories (trajectory models or latent class growth analysis models) for oral anticoagulants, VKAs and NOACs, in patients with AF, and to estimate the associated factors and their impact on clinical events. We further aim to assess the same issues with the conventional measures of primary and secondary adherence (PDC and persistence) in routine clinical practice in a population-based setting.
The specific study objectives are: (1) to describe the trajectories or patterns of adherence oral anticoagulants in patients with AF in the Valencia region who have been treated with these drugs during the period 2010–present; (2) to estimate the likelihood of patients belonging to different trajectories and to identify the characteristics of the patients associated with each trajectory of adherence; (3) to estimate primary and secondary (proportion of days covered (PDC) and persistence) adherence to oral anticoagulants; (4) to identify factors associated with the conventional measures of non-adherence to oral anticoagulants, with reference to patient characteristics as well as those of the drugs themselves, the professionals and health organisations; (5) to estimate the impact on clinical outcomes of non-adherence (defined by descending patterns or trajectories of adherence, and by conventional measures of adherence).
Patients and methods
This is a population-based retrospective cohort study of all patients with AF on oral anticoagulant therapy from 2010 to date in the Valencia region. Some analyses will be carried out differentiating between naïve patients (starting anticoagulant treatment for the first time in the period) and experienced patients (previously treated with anticoagulants).
Population and setting
The study will take place in the Valencia region and, specifically, in the population covered by the VHA, the public health system covering about 97% of the population of the region's population (≈5 million inhabitants). All patients with AF (diagnosis code of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) 427.31) under treatment with oral anticoagulants (warfarin, acenocoumarol, dabigatran, rivaroxaban, apixaban) since 2010 will be included. The time window will extend from 2009 to date to define the baseline characteristics of the population. We will define the naïve population as those patients without anticoagulant treatment in the 12 months preceding the first prescription. People without pharmaceutical/health coverage by VHA, mainly some government employees whose prescriptions are reimbursed by civil service insurance mutualities, and thus not included in the pharmacy databases of the VHA, and patients not registered in the municipal census (non-residents or temporary residents), or who left the region or who were disenrolled from VHA coverage for other causes, were excluded because of limitations on follow-up.
All necessary information will be obtained from the electronic information systems of the VHA (see figure 1): (1) The Population Information System (SIP) provides an identification number for each person under VHA coverage, and registers some demographic characteristics, and dates and causes of VHA discharge, including death. (2) The Minimum Basic Dataset (MBDS) at hospital discharge is a synopsis of clinical and administrative information on all hospital discharges, including diagnoses and procedures (all electronic health systems in the VHA use the ICD-9-CM). (3) The electronic medical record for ambulatory care, ABUCASIS, is available in all primary healthcare centres and other ambulatory settings. It has all the information on patients regarding diagnoses, their personal and family medical history, laboratory results, lifestyle, etc. (4) The pharmaceutical module, called GAIA (prescription information system), part of ABUCASIS, includes information about both physician prescriptions and dispensations from pharmacy claims. (5) The Corporate Resource Catalogue (CRC) provides information about the geographical and functional organisation of VHA, its health centres, health services provided and professionals in healthcare. All the information in these systems can be linked at an individual level through the SIP number.
Outcome measures (main and secondary endpoints)
Main outcome measure
The primary outcome measure will be the patterns or trajectories of adherence to the different anticoagulants over the time of follow-up. By calculating the number of days covered by dispensed medication, monthly adherence will be ascertained from the time of the first prescription (defining being adherent as having ≥24 days covered out of 30 (≥80% of the time covered)), obtaining a repeated dichotomous measure for each month of follow-up and patient. Patterns or adherence trajectories will be identified using group-based trajectory modelling, patients being classified according to their most likely pattern of adherence (ie, the trajectory that the patient has the highest probability of belonging to).
Secondary outcome measures
The following will be used as secondary measures (see figure 2): (1) Primary non-adherence, defined as not filling the first prescription of a specific anticoagulant from the pharmacy.65 ,66 The first prescription is defined as the first prescription identified in the electronic clinical record in the study period. (2) Secondary adherence, as assessed by: (a) availability, measured by the proportion of days covered (PDC) by medication. PDC is a widely used measure of adherence calculated by dividing the number of days of medication dispensed by the number of days of the observed follow-up period. This measure is very similar to the medication possession ratio, with the difference that the PDC is capped at 100% in case the days of medication dispensed exceed those of the follow-up period.67 ,68 Days with available medication during the follow-up period will be estimated through the dose regimen defined by the physician and the number of pills per package (eg, for a regimen of one pill every 12 h and packages of 30 tablets, each dispensation will entail 15 days of medication available). We will allow a maximum of accumulated days supply (stockpiling) of 180 days. To classify patients as adherent, the widely accepted cut-off point of 80% will be used (alternative cut-off points will be tested in sensitivity analyses); and (b) persistence, defined as the duration of continuous use of the medication after the index prescription. The treatment will be considered discontinued if a patient does not fill the medication from the pharmacy after grace periods of 30 and 60 days, once the period with available medication from the previous dispensation is over.67 ,69 In the sensitivity analysis, the impact of using shorter and longer grace periods will be estimated.
We seek to include the following clinical outcomes (see figure 2): hospitalisation for thromboembolic events, bleeding events and death from any cause during the follow-up period (2010–to date). Such admissions will be identified through the corresponding ICD-9-CM codes in the MBDS, while the information about mortality (from any cause) will be obtained from the SIP.
Covariates related to prescription, patients and level of healthcare system will be selected from the different electronic databases (see figure 1): (1) prescription-related variables: generic name, price, strength, dosing schedule/regimen, electronic dispensing, reduced co-payment (10% of the cost with a ceiling of €4.13 (≈£3.27) per package for some long-term treatments); (2) patient-related variables: date of birth, sex, co-payment level, comorbidity (eg, stroke, heart failure, ischaemic heart disease, valvular heart disease, arrhythmias, thyroid disease, diabetes, sleep apnoea, chronic obstructive pulmonary disease, chronic renal failure, dementia, among others); lifestyle and risk factors (eg, obesity, hypertension, smoking, alcohol intake, sedentarism); previous treatments (time window: 12 months prior to taking the first anticoagulant) and concomitant treatments (time window: follow-up period). Additionally, information on health services utilisation such as outpatient visits, including medical specialty, hospital admissions and visits to emergency services during the follow-up period and in the previous 12 months, will be collected; (3) healthcare-system-related variables: primary healthcare centre, basic healthcare zone, health department and hospital.
Data analysis plan
First, the baseline characteristics of the cohort, stratified by VKA and NOACs, will be described. Adherence measures (primary adherence—filling the first prescription—and secondary adherence—PDC and persistence at 3, 6, 12 and 24 months) will be estimated using the appropriate parameters (means or proportions) for each variable with its respective confidence intervals at 95% (CI 95%). The monthly PDC of each patient will be used to describe trajectories or patterns of adherence (trajectory models or latent class growth analysis models) for each drug. The selection of the number of trajectory groups or latent classes that best represents the heterogeneity in adherence will be based on: (1) model fit indices: Bayesian information criteria (BIC),70 Akaike information criterion (AIC)71 where a lower index value indicates a better model fit, and the Lo-Mendell-Rubin likelihood ratio test (LMR-LRT),72 which assesses the improvement in fit between neighbouring class models (comparing k−1 and the k class models). The LMR-LRT test provides a p value that can be used to determine if there is a statistical improvement in fit for the inclusion of one more class; (2) a minimum proportion of the study sample in a class: 5%; and (3) entropy, which indicates uncertainty in the classification of the model and is a measure of how well—or how precisely—study participants are classified into their most likely class. The cut-off value will be a posterior probability of ≥0.7.
Later, we will analyse bivariate associations between covariates—patients’ characteristics (sociodemographic and clinical), those of the drugs, professionals and healthcare system—and the predefined endpoints (primary and secondary adherence). Multivariate logistic regression analysis will be used to evaluate possible independent associations between covariates of interest (depending on their clinical significance and the results of the previous bivariate analysis), and dichotomous adherence measures (primary adherence and PDC) and multinomial logistic regression models to identify patients’ characteristics associated with each trajectory of adherence. The corresponding odds ratios with their respective CIs 95% will be estimated using backward-forward stepwise methods. The association between the covariates and persistence will be studied with a similar scheme but using Kaplan-Meier models (bivariate analysis) and the Cox proportional hazards regression model (multivariate analysis). The corresponding hazard ratio (HR) will be estimated with their CIs 95%. The predictions from these models will also be used to construct the propensity scores that will be incorporated as explanatory variables with respect to clinical outcomes, in order to minimise confounding.
To analyse the association between different measures of adherence and clinical outcomes, Cox proportional hazard models will be used again, adjusting for the corresponding propensity scores. Cases will be censored at the date of VHA discharge if the main event (hospitalisation for thromboembolic or bleeding events or death) occurs, or at the end of study follow-up. Additionally, in sensitivity analyses, to further assess the robustness of our findings, we will: (1) assess the association between persistence and clinical events by an extension of the Cox models, which allows the use of multiple correlated events, (2) evaluate such association using persistence as a time-varying covariate, (3) evaluate such association using PDC as a time-varying covariate, by calculating PDC in intervals, throughout the length of follow-up. When an event occurs, the PDC for the period preceding such event will be applied. Details on this approach can be seen in (Ho, 2008, AHJ).73 We will (4) repeat our analysis after excluding days patients spent in the hospital from the denominators of the adherence measures, and (5) when using the PDC as adherence measure, we will also run the models only considering outcomes that occurred at least 12 months after the index date (being the adherence assessment period the first 12 months after the first prescription).
Finally, to estimate the role of different levels of care in the variations of adherence, multilevel logistic or Cox regression models will be used as appropriate, with random effects for the following variables: primary healthcare centre, basic healthcare zone (n=240) and health department (n=24). We will start with the empty model before introducing explanatory variables, including as random effects only the hierarchical levels: individual/ primary healthcare centre/basic healthcare zone/health department. Following the criterion of improvement in the model goodness of fit and parsimony (Deviance Information Criteria) the relevant levels will be chosen, contextual covariates will be incorporated and the interaction between levels will be analysed. Simulation-based Markov Chain Monte Carlo (MCMC) methods will be used to estimate the error terms of the hierarchical structure. Analyses will be performed using STATA and R statistical packages.
Ethics and dissemination
Interventions derived from the study
This is an observational, retrospective study developed on databases, and does not include any intervention or randomisation, thus it does not imply any additional risk to patients and cannot result in the prescription of any drug. Compliance with the standards of good research practices: This study will be conducted in accordance with the international standards for epidemiological studies, as established in the International Guideline for Ethical Review of Epidemiological Studies (Council for the International Organizations of Medical Sciences-CIOMS-Geneva, 2009). Clinical Research Ethics Committee: The study protocol was approved by the Clinical Research Ethics Committee of the ‘Dirección General de Salud Pública y Centro Superior de Investigación en Salud Pública’ (CEIC DGSP-CSISP, ruling of March 5, 2014). Classification and authorisations: The study protocol was submitted to the Spanish Agency of Drugs and Health Products (the competent authority) and classified as a ‘postauthorisation study with design other than prospective follow-up’ requiring no further authorisations (SPM-ACO-2014–01; 12 March 2014). The study protocol was also authorised by the General Directorate of Pharmacy and Medical Products (23 June 2014) and the cession of anonymised data was approved by the Regulatory Commission of Access to Ambulatory Care Information Access of the VHA. Confidentiality: In accordance with the Spanish laws on right to privacy and patients’ protection, and the 2009 and 2012 Resolutions of the Regional Health Secretary regarding information requests from the VHA information system, the database linkages will be carried out in the corresponding Department of the VHA by the people authorised to do these tasks. To protect patient privacy, data will be sent to the researchers with dissociate non-traceable codes that would not allow the identification of individual patients, or their linkage with other databases.
We plan to disseminate the project's findings through peer-reviewed publications and presentations, at relevant national and international health conferences. Policy reports will also be prepared and discussed with stakeholders, in order to promote the translation of our findings into policy and clinical practice. Certain types of measures could be incorporated into the electronic information systems and be incorporated into the sets of performance indicators of the VHA.
First, the assessment of adherence based on data from the pharmacy electronic records allows a reliable assessment of adherence in terms of prescriptions dispensed. However, there will be no certainty that the patient actually consumes the medication filled from the pharmacy. Nevertheless, studies evaluating the correlation between adherence measures obtained by pharmacy claims and pill counts show a high degree of agreement,74 ,75 so high consistency between dispensing and consumption is expected. Second, the misclassification of patients (treated/non-treated or adherent/non-adherent) due to non-registered dispensations, either because they were purchased without a prescription (exceptional for the therapeutic group studied), or because they were acquired in another region (eg, during holiday periods). Third, information biases due to absent registration or differing data recording practices in the electronic medical record. This problem, however, is always present when data from routine clinical practice are used. Fourth, currently, almost all prescriptions in the VHA are issued electronically (less than 1% of prescriptions are made manually by physicians). However, the study includes data since 2010, when this percentage was slightly higher, although still small (around 5%). This limitation particularly affects the estimate of primary adherence, because we cannot retrieve those prescriptions issued manually, and in the case of a first prescription not being filled, we would not be able to identify it as non-adherence, thus overestimating real primary adherence. Therefore, for the estimation of primary non-adherence, we will exclude patients not receiving the first prescription through the electronic prescribing module of the electronic medical record due to the inability to establish a prescription–dispensation relationship. Fifth, given the characteristics of the study (observational, based on retrospective real-world data), we can expect the presence of selection bias and confounding, especially in drug selection, because NOAC prescriptions are subject to prior authorisation (requiring a previous failure to keep INR in range using VKA). To address this problem, we have planned the use of matching methods (propensity score).
Funding This study was partially funded by the 2013 Collaboration Agreement between the Fundación para el Fomento de la Investigación Sanitaria y Biomédica (FISABIO) from the Valencia Ministry of Health and Boehringer Ingelheim, a non-conditioned programme to conduct independent research in chronic healthcare, pharmacoepidemiology and medical practice variation. CLR-B was funded by grant RD12/0001/0005 from the Instituto de Salud Carlos III, Spanish Ministry of Health (co-financed by the European Regional Development Fund).
Competing interests SP has received research grants from various pharmaceutical companies and fees for participation in scientific meetings sponsored by pharmaceutical companies. GS-G participated in 2014 in an advisory meeting of Boehringer Ingelheim.
Ethics approval The study protocol was approved by the Clinical Research Ethics Committee of the “Dirección General de Salud Pública y Centro Superior de Investigación en Salud Pública” (CEIC DGSP-CSISP, ruling of March 5, 2014). Classification and authorisations: The study protocol was submitted to the Spanish Agency of Drugs and Health Products (the competent authority) and classified as a ‘post-authorisation study with design other than prospective follow-up’, requiring no further authorisations (SPM-ACO-2014-01; March 12, 2014). The study protocol was also authorised by the General Directorate of Pharmacy and Medical Products (June 23, 2014) and the cession of anonymised data was approved by the Regulatory Commission of Access to Ambulatory Care Information Access of the VHA.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.