Objectives To externally validate the accuracy of the Global Anticoagulant Registry in the FIELD-Atrial Fibrillation (GARFIELD-AF) model against existing risk scores for stroke and major bleeding risk in patients with non-valvular AF in a population-based cohort.
Design Retrospective cohort study.
Setting Danish nationwide registries.
Participants 90 693 patients with newly diagnosed non-valvular AF were included between 2010 and 2016, with follow-up censored at 1 year.
Primary and secondary outcome measures External validation was performed using discrimination and calibration plots. C-statistics were compared with CHA2DS2VASc score for ischaemic stroke/systemic embolism (SE) and HAS-BLED score for major bleeding/haemorrhagic stroke outcomes.
Results Of the 90 693 included, 51 180 patients received oral anticoagulants (OAC). Overall median age (Q1, Q3) were 75 (66–83) years and 48 486 (53.5%) were male. At 1-year follow-up, a total of 2094 (2.3%) strokes/SE, 2642 (2.9%) major bleedings and 10 915 (12.0%) deaths occurred. The GARFIELD-AF model was well calibrated with the predicted risk for stroke/SE and major bleeding. The discriminatory value of GARFIELD-AF risk model was superior to CHA2DS2VASc for predicting stroke in the overall cohort (C-index: 0.71, 95% CI: 0.70 to 0.72 vs C-index: 0.67, 95% CI: 0.66 to 0.68, p<0.001) as well as in low-risk patients (C-index: 0.64, 95% CI: 0.59 to 0.69 vs C-index: 0.57, 95% CI: 0.53 to 0.61, p=0.007). The GARFIELD-AF model was comparable to HAS-BLED in predicting the risk of major bleeding in patients on OAC therapy (C-index: 0.64, 95% CI: 0.63 to 0.66 vs C-index: 0.64, 95% CI: 0.63 to 0.65, p=0.60).
Conclusion In a nationwide Danish cohort with non-valvular AF, the GARFIELD-AF model adequately predicted the risk of ischaemic stroke/SE and major bleeding. Our external validation confirms that the GARFIELD-AF model was superior to CHA2DS2VASc in predicting stroke/SE and comparable with HAS-BLED for predicting major bleeding.
- pacing & electrophysiology
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This validation study was able to compare prediction performance Global Anticoagulant Registry in the FIELD-Atrial Fibrillation model versus CHA2DS2VASc for stroke and HAS-BLED for major bleeding in patients with atrial fibrillation.
This study used a large contemporary population-based cohort with atrial fibrillation with many events and very limited loss to follow-up.
The validation was based on International Classification of Diseases, Tenth Revision coding from the Danish registries which is prone to misclassification bias and lacked clinical measurements.
Atrial fibrillation (AF) is a common cardiac arrhythmia with a lifetime prevalence of 20%–30% and is the cause of one in four strokes.1 AF is associated with an increased risk of several cardiovascular conditions, most notably a nearly fivefold increased stroke risk.2 3 The risk of stroke can be substantially diminished by thrombotic prophylaxis.4 5 However, 20%–40% of potentially eligible patients do not receiving oral anticoagulant (OAC) therapy.6–8 The most important and modifiable contributing factor is inappropriate risk assessment, with underutilisation of existing risk scores, resulting in overestimation of bleeding risks and underestimation of potential stroke risk.9 10 Recently, the Global Anticoagulant Registry in the FIELD-Atrial Fibrillation (GARFIELD-AF) model was developed that allowed for simultaneous calculation of death, stroke and bleeding risks in an international prospective registry of patients with newly diagnosed AF.11 In the GARFIELD-AF and ORBIT-AF registries, the GARFIELD-AF model was found to improve discrimination of the existing risk scores for stroke (CHA2DS2-VASc) and bleeding (HAS-BLED).11–13 These registries may not cover the full spectrum of patients with AF, which warrants external validation of these risk scores in other population-based cohorts. We aimed to (1) externally validate the GARFIELD-AF model of ischaemic stroke and major bleeding outcomes among patients with newly diagnosed AF in a large contemporary Danish cohort and (2) perform a head-to-head comparison of the predictive properties of GARFIELD-AF model with CHA2DS2-VASc for thromboembolic events and HAS-BLED for major bleeding. We did not externally validate the GARFIELD-AF model for risk of death, as we did not have blood pressure and heart rate measurements; covariates that the GARFIELD-AF model for death requires.
Materials and methods
We reported our findings according to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis criteria.14
We used the Danish nationwide registers cross-linking The Civil Registration System, The Danish National Patient Register (DNPR) and The Danish Drug Statistical Registry. The Civil Registration System holds data on age, sex and vital status. DNPR contains all hospital admissions according to International Classification of Diseases, Tenth Revision (ICD-10) and procedures. The Danish Drug Statistical Registry was used to characterise pharmacotherapy in which all claimed drug prescriptions are registered. To compare characteristics (baseline and outcomes) of the Danish registry, we used data from the GARFIELD-AF registry which, in brief, is an observational, multicentre, international study of newly diagnosed AF with ≥1 risk factor for stroke.15
From the DNPR, patients aged ≥18 years with a primary or secondary diagnosis of AF (ICD‐10: I48), hospitalisation or outpatient visit were included from 1 January 2010 until 1 August 2016 with follow-up to 1 August 2017. The diagnosis of AF in the DNPR has a positive predictive value of 94.0%.16 Patients with rheumatic valvular heart disease or valve interventions were excluded. To allow patients time to fill their prescriptions after discharge, a 10-day wash-out period was used. Due to no data on race/ethnicity in the The Civil Registration System, we excluded immigrants and those with missing information on immigration and presumed Caucasian/European-white for non-immigrants. Given complete nationwide coverage of DNPR, missing data are not present. For baseline characteristic and outcome comparison, we included the worldwide enrolled GARFIELD-AF patients and the patients enrolled from the Scandinavian sites; Denmark, Sweden, Norway and Finland (GARFIELD-AF Scandinavia).
Covariates, GARFIELD-AF model, CHA2DS2-VASc and HAS-BLED
For the Danish AF cohort, all baseline variables were defined from ICD-10 codes, as any primary or secondary diagnosis, inpatient or outpatient, registered up to 10 years prior to the inclusion date. Pharmacotherapy at baseline was identified by Anatomical Therapeutic Chemical codes of prescription drugs claimed up to 180 days prior to the inclusion date. For OAC, within the 180 days, the latest prescription filled of either vitamin K antagonist (VKA) or non-vitamin K oral anticoagulants (NOAC) was used. Prescriptions claimed for antidiabetic drugs were used as proxy for diabetes mellitus. Hypertension was defined as claimed prescription for a combination of at least two of the seven different antihypertensive drug classes as previously reported.17 The algorithm for GARFIELD-AF 1-year risk of ischaemic stroke/systemic embolism (SE) relies on the following variables: age (in years), history of ischaemic stroke, prior bleeding (any recorded in medical records), heart failure (medical history of heart failure or ejection fraction of <40%), chronic kidney disease (CKD) (stages III–V), race/ethnicity and use of OAC (VKA or NOAC). The algorithm for GARFIELD-AF 1-year risk of major bleeding/haemorrhagic stroke was developed in patients taking OAC and involves age, CKD and vascular disease, in which the latter is defined as history of myocardial infarction (MI) or unstable angina, aortic or peripheral artery disease. The CHA2DS2-VASc score is composed of age, sex, a history of heart failure, hypertension, stroke, transient ischaemic attack, thromboembolism and/or diabetes. HAS-BLED is composed of age, uncontrolled hypertension renal disease, liver disease, labile international normalised ratio (INR), medication use predisposing to bleeding and a history of stroke, major bleeding and/or predisposition to bleeding. All covariates were based on ICD-10 codes. The equations for these respective scores and ICD-10 codes can be found in online supplementary table S1.
Definitions of endpoints
The primary efficacy endpoint for this study involved a 1-year composite of ischaemic stroke or SE. The primary safety endpoint involved a composite of haemorrhagic stroke or major bleeding. Major bleeding was defined as an organ-specific bleeding requiring hospitalisation.18 A list of ICD-10 codes used to compute these definitions can be found in Table S2. In the GARFIELD-AF cohorts, the occurrence of ‘major bleeding’ was defined according to the International Society on Thrombosis and Haemostasis (ISTH) criteria.11 15 19
Patient and public involvement
Patients or the public were not involved in the design of the study.
For the Danish AF cohort, we stratified baseline characteristics by CHA2DS2-VASc score (≤2 for women, 0–1 for men and >2 for women and >1 for men). The Danish AF cohorts were followed for a maximum of 1-year from the discharge date and until the event of interest (stroke/SE or major bleeding), death, emigration or end of follow-up (August 2017). For all three cohorts (Danish AF cohort, GARFIELD Scandinavia and GARFIELD Global), 1-year absolute risks of stroke/SE and major bleeding were estimated non-parametrically using the Aalen-Johansen estimator with competing risk of death. For the Danish AF cohort, using a logistic regression model, we used the original coefficients from the GARFIELD-AF model development study. The discriminative performance of GARFIELD-AF for predicting ischaemic stroke/SE and major bleeding hospitalisations was assessed using receiver operating characteristics curves and reported as area under the curve (AUC) values. The C-index with 95% CI was reported as a measure of discrimination. As there was no censoring issue because the data did not contain any loss to follow-up within the first year of the study period, the significance test of C-index differences was tested for significance with DeLongs method.20 Calibration was assessed by calculating deciles of predicted probabilities and plotting the average predicting with the observed Kaplan-Meier rate and 95% CI within each decile. A subgroup analysis for stroke prediction was undertaken for low risk patients. As a sensitivity analysis, we estimated the logistic regression coefficients for each individual covariate used in the GARFIELD-AF models. Statistical analyses were performed using R software (Team RC. R: A Language and Environment for Statistical Computing, 2019).
From the Danish registries, a total of 110 276 patients were diagnosed with AF between 2010 and 2016. Of those patients, 91 836 met the inclusion criteria for the study (Figure S1). Of these, 70 020 were identified as high risk and 20 673 was identified as low risk of stroke/SE. A total of 51 180 were on OACs. From the GARFIELD-AF registries, 52 080 patients with AF were included globally, and 2396 patients were included from the Scandinavian sites.
Characteristics of study participants
Baseline characteristics for all three cohorts (Danish AF cohort, GARFIELD Scandinavia and GARFIELD Global) are shown in table 1. Compared with GARFIELD Scandinavia, the Danish AF cohort had a more equal representation of men and women (54% vs 58%) and was older (median age 75 vs 73). There were also notable differences in comorbidities, in which diabetes, hypertension and CKD were less prevalent in the Danish AF cohort, whereas a history of bleeding (11.6% vs 1.9% vs 2.5%) was much more prevalent compared with the GARFIELD Scandinavia or global cohorts. Despite these differences, the median CHA2DS2Vasc scores were comparable among all cohorts but the HAS-BLED median was higher in the Danish AF cohort. For the GARFIELD model, the median GARFIELD scores were higher in the Danish AF cohort than both GARFIELD cohorts. The use of OAC therapy was 56.4% in the Danish AF cohort, which was lower than the reported percentages in GARFIELD Scandinavia (68.0%) and GARFIELD Global (60.8%). The use of aspirin, ADP-inhibitors and non-steroidal anti-inflammatory drugs was higher in the Danish AF cohort. The characteristics of patients at low risk (CHA2DS2Vasc of 0–1 in men and 1–2 in women; n=20 673) and high risk (n=70 020) for the Danish AF population are displayed in Table S3. The median (IQR) for CHA2DS2-VASc and HAS-BLED scores in the high-risk group was 43–5 and 2.2 3
Clinical outcomes at 1-year follow-up
Table 2 displays the number of stroke/SE, major bleeding and all-cause mortality for the Danish AF cohort and GARFIELD-AF cohort. For the Danish AF cohort, over a 1-year follow-up period, a total of 2094 (2.3%) stroke/SE, 2642 (2.9%) major bleedings were reported. Annual mortality rates were high with 10 915 deaths (12.0 per 100 person years). For the OAC-treated patients, there were 4521 deaths (8.8 per 100 person years) and for low risk patients, there were 623 deaths (3.0 per 100 person years). For the GARFIELD-AF Global, the rates of events were lower for stroke/SE (1.2%), major bleeding (1.1%) and deaths (4.7%). For the GARFIELD-AF Scandinavia, the rates of events were 1.0% for stroke/SE, 1.7% for major bleeding (1.7%), and 3.7% for deaths. The cumulative incidence for ischaemic stroke to 1 year for low-risk and high-risk patients are presented in Figure S2 for the Danish AF cohort. The cumulative incidence of stroke/SE and ischaemic stroke for the Danish AF cohort, GARFIELD Scandinavia and GARFIELD Global can be found in figure 1.
External validation of GARFIELD-AF model
The C-index for the GARFIELD-AF model for 1-year stroke was 0.71 (95% CI: 0.70 to 0.72) in the overall Danish AF cohort of 90 693 patients and 0.64 (95% CI: 0.59 to 0.69) in low-risk patients not requiring OAC therapy (n=20 673). The C-index for the GARFIELD-AF model for 1-year major bleeding risk was 0.64 (95% CI 0.63 to 0.66) in patients using OAC therapy. The GARFIELD-AF model for stroke/SE and major bleeding scores were both well calibrated in the Danish AF cohort (figures 2 and 3). The individual covariates in the GARFIELD-AF model expressed a similar regression coefficient for most covariates in the Danish cohort when compared with the original derivation GARFIELD-AF cohort (online supplementary table S4).
GARFIELD-AF model versus CHA2DS2-VASc score for predicting stroke/SE
The AUC curves for predicting stroke using the GARFIELD model and CHA2DS2-VASc scores are displayed in figure 4A for the Danish AF cohort and figure 4B for the low risk patients. The AUC and corresponding C-index for GARFIELD was significantly higher when compared with CHA2DS2Vasc for predicting stroke outcomes, in the overall cohort (C-index: 0.71, 95% CI: 0.70 to 0.72 vs 0.67, 95% CI: 0.66 to 0.68) as well as in low-risk patients (C-index: 0.62, 95% CI: 0.59 to 0.69 vs 0.57, 95% CI: 0.53 to 0.61).
GARFIELD-AF model versus HAS-BLED score for predicting major bleeding/haemorrhagic stroke
Figure 4C illustrates the AUC curves for the prediction of major bleeding based on the GARFIELD and HAS-BLED scores in patients taking OAC therapy. The discriminatory value of GARFIELD was comparable with HAS-BLED: C-index: 0.64 (95% CI: 0.63 to 0.66) versus C-index: 0.64 (95% CI: 0.63 to 0.65), respectively.
Table 3 displays available evidence comparing the discriminatory properties of various models for stroke/SE and major bleeding.
In a large unselected contemporary Danish AF cohort, our study demonstrates that the GARFIELD-AF model serves as a reliable risk stratification tool. We found that the GARFIELD-AF model surpasses the widely used CHA2DS2-VASc score in predicting the risk of stroke, both in high-risk and low-risk patients. For predicting major bleeding, the three-item GARFIELD-AF model is on par with HAS-BLED among anticoagulated patients with AF.
Risk-stratification of patients with AF is essential to mitigate the risk of stroke/SE when initiating anticoagulation therapy. As such, the easy-to-calculate CHA2DS2-VASc score serves as the risk-stratification tool recommended by international guidelines to commence therapy when the risk of stroke reaches a threshold of >2% per annum.21 Prior studies have shown that guideline-adherent (risk-stratified) anticoagulation therapy is associated with a 60%–70% reduction in thromboembolic associated complications and mortality.22 Despite these reductions in stroke risk, there is still room for improvement in risk stratification of patients with AF and anticoagulation therapy. In this regard, the GARFIELD-AF model, which calculates the risk of stroke, death and major bleeding, is promising, particularly as it performed better than the CHA2DS2-VASc and HAS-BLED.11 The GARFIELD-AF model is not a risk score that is categorised into risk groups but instead it provides risk prediction on a continuous scale. As the model provides risk estimates of stroke/SE and bleeding (and mortality when blood pressure and heart rate data are available) in one single calculation based on routinely collected data, it could have potentially wide clinical applications.
Underuse of anticoagulation therapy is a well-known problem in particular in high-risk patients.23 Our study found that 73.1% of patients with increased stroke risk (n=51 180, CHA2DS2-VASc>2 in women and >1 in men) received anticoagulation therapy. Studies suggest that treatment barriers are based on overestimation of bleeding risks (eg, recurrent falls and prior peptic ulcers) and lack of reliable risk stratification tools and readily available information that does not rely on specific laboratory tests such as HAS-BLED.9 10 23
To provide predictions for bleeding risk, several bleeding scores have been developed, of which HAS-BLED has been most rigorously tested.24 25 While the HAS-BLED risk score is clinically useful to identify patients at high bleeding risk, it requires information that may not always be available in outpatient and primary care settings, such as information on liver function tests, INR status and the presence of anaemia. This may be a limitation for the implementation of this score in low-resource settings and is of relevance as the majority (60%) of community-dwelling patients with AF are seen in primary care.26 27 Moreover, primary care physicians achieve lower anticoagulation rates compared with hospital-based physicians.28 29 While there are multiple reasons for these differences, which include differences in populations, with high percentage of (relative) contraindications (40%–65%) and care-related factors, the lack of an integral decision tool impedes informed decisions on initiating or continuing antithrombotic therapy.10 30
In the present study, the external validation show that GARFIELD-AF is well calibrated with the predicted risks, and the good calibration aligns well with the original GARFIELD-AF derivation cohort.11 GARFIELD-AF score has improved discriminatory abilities compared with CHA2DS2-VASc score in stroke prediction and on par with HAS-BLED for bleeding prediction. To our surprise, our external validation of GARFIELD-AF discriminatory c-index for stroke prediction was slightly higher (0.71) than the c-index in the original GARFIELD cohort (0.69). Furthermore, the Danish AF cohorts were older with more stroke events than both GARFIELD cohorts and higher predicted risk than the GARFIELD cohorts. The opposite was the case for major bleeding prediction where the original GARFIELD cohort had better discriminatory abilities than HAS-BLED. The significant differences in the cohort characteristics, for example, higher HAS-BLED and more use of combined OACs and antiplatelet therapy in the Danish AF cohort, and definitions of covariates and outcomes are likely to play a key role in both instances.
The value of CHA2DS2-VASc score in defining patients with a truly low risk of stroke is uncertain, partly because the number of patients to accurately assess lower risk patients in the original studies was insufficient.12 31 The evaluation of the performance of risk scores in this low-risk category is important, as it determines the threshold when initiating antithrombotic therapy outweighs the risk of bleeding. In our study although both models had modest predictive discrimination for low-risk patients, we found that in these patients, the GARFIELD-AF model was well calibrated and provided a better prediction than CHA2DS2-VASc score in indicating which patients are truly at low risk for subsequent stroke.
Limitations and future directions
The primary limitation of this study is that definitions are based on administrative ICD-10 codes which are prone to non-systematic misclassification bias. Discrepancies exist between the GARFIELD-AF registry and the Danish registries in the definitions of comorbidities and clinical outcome data. This was most notable for major bleeding and stroke/SE events and CKD. In the Danish AF cohort, selected ICD-10 codes for bleeding hospitalisations were applied, whereas GARFIELD-AF applied the ISTH criteria for major bleeding, which are more restrictive. Apart from differences in disease definitions, the relative high number of bleeding and stroke/SE events in the Danish AF cohort could also be explained by site of enrolment (only hospital/out-patient). Similar data limitations apply to the construction of CHA2DS2-VASc score and HAS-BLED scores which was calculated from ICD-10 code usage although this study followed the standards set by other researchers using the same Danish registries.17 Specifically, we were unable to account for labile INR component of the HAS-BLED score. Another limitation was the inability to ascertain ethnicity status which is an integrated covariate of the GARFIELD-AF model. Therefore, we excluded immigrants to strengthen the assumption of European/Caucasian ethnicity in the cohort. The GARFIELD Scandinavian cohort consisted of 99.3% Caucasians. Population-based studies in other more ethnically diverse cohorts are warranted. We did not asses net reclassification improvement as this statistic is not appropriate when using it for point-based scores, such as CHA2DS2-VASc.32 For the implementation of the GARFIELD-AF model, an online risk calculator already exists. A next step would be to provide an electronic health record integrated solution, in which stroke/SE and bleeding risks are automatically calculated when a patient is identified with AF. Doing so would promote balanced and evidence-based decision-making on anticoagulation therapy. Integrating the GARFIELD-AF model into electronic health records would also provide a way for anonymous monitoring of outcomes of patients in which the GARFIELD-AF model was applied, which can be used to further optimise risk prediction.
The GARFIELD-AF model adequately predicted the risk of ischaemic stroke/SE and major bleeding/haemorrhagic stroke in a nationwide Danish cohort of contemporary patients with non-valvular AF. Our external validation confirms that the performance of the GARFIELD-AF model was superior to CHA2DS2VASc in predicting stroke/SE both in high-risk and in low-risk patients and comparable with HAS-BLED for predicting major bleeding. The GARFIELD-AF model holds an advantage over the existing risk scores as it permits for simultaneous evaluation of death, stroke and bleeding risks and uses readily available clinical parameters. As such, the tool may lead to more informed treatment decisions, improve monitoring for bleeding complications and improve outcomes for patients treated for AF in the community.
We would like to thank the investigators and participants involved in the GARFIELD-AF registry, Madhusudana Rao for programming support of GARFIELD-AF tables and Saverio Virdone for statistical support of GARFIELD-AF survival curves.
Contributors FD, RH, MR, FV, KF, AKK, HVW and KP contributed to the study concept and design; FD and KP conducted the data analysis; REH and FD drafted the manuscript; FD, REH, MR, FV, KF, AKK, HVW, JC, JP, PVR, TBL, CTP and GHG critically reviewed all drafts of the manuscript.
Funding REH reports salary support from The Netherlands Organization for Scientific Research (NWO) through a RUBICON-grant to study bleeding risks associated with NOAC use. FD is funded by Danish Heart Foundation (Grant no: 17-R115-A7443-22062) and Gangstedfonden (Grant no. A35136) and reports travel grants from Knud Højgaard Fonden, Torben og Alice Frimodts Fond. KP is funded by the Duke Clinical Research Institute.
Competing interests KF has received research grants Bayer/Janssen and AstraZeneca and Consulting/Fees Bayer, Sanofi/Regeneron and Verseon. FV has received honoraria for consulting and presentations from Bayer HealthCare, Boehringer Ibgelheimy, BMSPfizer and Daiichi-Sankyo. GHG has ownership of stocks in Novo Nordisk Pharmaceuticals and reports research grants from Pfizer, Bristol Myers Squibb, Boehringer Ingelheim and Bayer. CTP declares grants for studies from Bayer. Other authors have no conflicts of interest to declare.
Patient consent for publication Obtained.
Ethics approval For the GARFIELD-AF registry, an independent ethics committee and hospital-based institutional review board approvals were obtained for the registry protocol. In Denmark, retrospective register studies do not require approval from the ethics committees. The Danish Data Protection Agency approved this study (Ref no: 2007-58-0015, I-Suite no: 02732, GEH-2014-014). Data were made available in an anonymised format such that specific individuals could not be identified.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Due to restrictions related to Danish law and protecting patient privacy, the combined set of data as used in this study can only be made available through a trusted third party, Statistics Denmark. This state organisation holds the data used for this study. Any request on data access should be addressed to Statistics Denmark https://www.dst.dk/en.aspx%23
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.