You are here

Article Text

Article menu
Review
Prediction models for the risk of cardiovascular disease in patients with type 2 diabetes: a systematic review
  1. S van Dieren1,
  2. J W J Beulens1,
  3. A P Kengne1,2,3,
  4. L M Peelen1,
  5. G E H M Rutten1,
  6. M Woodward3,
  7. Y T van der Schouw1,
  8. K G M Moons1
  1. 1Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
  2. 2South African Medical Research Council & University of Cape Town, Cape town, South Africa
  3. 3The George Institute for Global Health, University of Sydney, Sydney, Australia
  1. Correspondence to Dr Susan van Dieren, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Str 6.131, PO Box 85500, 3508 GA Utrecht, The Netherlands; s.vandieren{at}umcutrecht.nl

Abstract

Context A recent overview of all CVD models applicable to diabetes patients is not available.

Objective To review the primary prevention studies that focused on the development, validation and impact assessment of a cardiovascular risk model, scores or rules that can be applied to patients with type 2 diabetes.

Design Systematic review.

Data sources Medline was searched from 1966 to 1 April 2011.

Study selection A study was eligible when it described the development, validation or impact assessment of a model that was constructed to predict the occurrence of cardiovascular disease in people with type 2 diabetes, or when the model was designed for use in the general population but included diabetes as a predictor.

Data extraction A standardized form was sued to extract all data of the CVD models.

Results 45 prediction models were identified, of which 12 were specifically developed for patients with type 2 diabetes. Only 31% of the risk scores has been externally validated in a diabetes population, with an area under the curve ranging from 0.61 to 0.86 and 0.59 to 0.80 for models developed in a diabetes population and in the general population, respectively. Only one risk score has been studied for its effect on patient management and outcomes. 10% of the risk scores are advocated in national diabetes guidelines.

Conclusion Many cardiovascular risk scores are available that can be applied to patients with type 2 diabetes. A minority of these risk scores has been validated and tested for its predictive accuracy, with only a few showing a discriminative value of ≥0.80. The impact of applying these risk scores in clinical practice is almost completely unknown, but their use is recommended in various national guidelines.

  • Cardiovascular disease
  • diabetes mellitus
  • prediction
  • validation
  • impact assessment
  • diabetic heart disease
  • macrovascular disease
  • microvascular disease
  • epidemiology
  • coronary artery disease
  • obesity
  • clinical hypertension
  • prevention
  • diabetes
  • general practice

https://doi.org/10.1136/heartjnl-2011-300734

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    People with type 2 diabetes have a twofold increased risk of cardiovascular disease (CVD).1 2 Guidelines for the management of type 2 diabetes advocate calculating CVD risk to guide the initiation of appropriate treatment.3–5 Over the past decades many prediction models (or risk scores) have been developed to predict CVD, of which only a small number have been specifically developed for people with type 2 diabetes.6 However, most prediction models developed for the general population take diabetes into account and can therefore be applied to the diabetes population. The performance of several prediction models has been examined in different populations for their discrimination (ability to discriminate between patients who will get the disease and those who will not) and calibration (ability to correctly quantify the absolute risk), but the outcomes have varied widely.7–9

    A systematic review by Chamnan et al6 provides an overview of CVD prediction models that have been developed in diabetes populations, and prediction models for the general population that have been validated in a diabetes population. However, new prediction models for the diabetes population have been developed since this review, and many more prediction models exist that can be applied to people with diabetes. Moreover, it is unknown whether applying a certain prediction model in clinical practice affects the treatment of patients with diabetes and subsequently improves cardiovascular outcome.

    Clinical prediction models need to provide accurate and validated estimates of the risk of the targeted outcome, in order to be useful for clinical decision-making. Three stages of prediction modelling can be identified: (1) model development, which includes among other aspects determining the clinically relevant predictors, assigning the relative weights to these predictors and estimating the model's predictive performance, ideally after adjustment for overfitting or optimism with internal validation techniques; (2) assessment of the model's predictive performance in new patients (external validation studies); (3) quantifying whether the use of a prediction model in daily practice indeed improves decision-making and subsequently patient outcome as compared with not using the model (model impact studies).10–15

    The aim of this study was to identify all CVD prediction models (or scores or rules) that can be applied to patients with type 2 diabetes, and subsequently to assess their status and compliance in relation to the three stages of prediction modelling.

    Methods

    We started by systematically searching the literature for all CVD prediction models that can be applied to, or have been developed in, people with type 2 diabetes. Subsequently, we searched specifically for all studies that validated these models in patients with type 2 diabetes, followed by a search for all impact studies of these models and guidelines that incorporated these prediction models.

    Search strategy

    Existing CVD prediction models for patients with diabetes

    At 1 April 2011, Medline was searched using a mixture of MeSH terms and truncated words to identify all papers presenting a CVD prediction model developed in patients with diabetes or that can be applied to individuals with type 2 diabetes. The precise search query is available in the appendix 1 (search term 1).

    To identify additional studies we searched our own literature files, previous reviews of cardiovascular prediction models and checked citations. A study was eligible when (1) the prediction model was either developed in people with diabetes or included diabetes as a predictor, (2) the outcome of the prediction model was CVD or a cardiovascular component (ie, coronary heart disease (CHD), heart failure or stroke) and (3) it presented a specific prediction rule/model with sufficient information on all variables to calculate the CVD risk in a different population (β coefficients of the model or otherwise a scoring system/graph/score card/nomogram was provided). Exclusion criteria were non-human studies, articles in languages other than English or Dutch, studies presenting a prediction model developed in patients with previous CVD or other vascular condition (eg, hypertensive patients) or studies focusing on the added predictive value of new risk factors to an existing prediction model. Furthermore, we checked the reference list of all retrieved development studies, to determine whether any studies presenting a risk score had been missed.

    External validation studies

    After retrieval of all developed prediction models, we performed a second search to identify all studies that evaluated these models in a new, independent patient sample (search term 2, appendix 1). The resulting studies were added to the validation studies obtained with search term 1. Again, to identify potentially missed studies we searched our own literature files, previous reviews of cardiovascular prediction models and checked citations.

    Impact studies

    To find any impact studies of the retrieved cardiovascular prediction models, we first searched among the publications that were identified through the first and second search strategy. Furthermore, we developed a specific search query for impact studies for each prediction model separately. This strategy was based on search strategy for impact studies developed by Reilly and Evans16 (search term 3, appendix 1) combined with the specific prediction model's acronym, or if not available the name of the cohort in which the score was developed or first author of the paper.

    Data extraction

    Two authors (SVD and JWJB) independently reviewed all titles, followed by the abstract and full text of the studies. Any disagreements were resolved by a third (APK) and fourth (KGMM) reviewer. A standardised form was used for the data extraction, including outcome of the prediction model, number of events, specifics of the development population, presentation of the prediction model, type of model, results from an internal validation, selection procedure of, and variables included in, the prediction model.

    For the data extraction of the external validation studies, a different data extraction sheet was used, including specifics of the validation population, number of events, type of outcome, statistical tests and measures of discrimination and calibration of the prediction model. From the retrieved impact studies additional data were extracted—namely, number of subjects, type of intervention, prediction model used in the intervention, type of outcome, number of outcomes and statistical tests.

    Implementation of prediction models in guidelines

    Finally, we searched various guidelines for clinical practice written in English to examine the implementation of the prediction models in countries in which the prediction models have been developed. As there are no strategies available for these types of internet searches, we specifically searched among the guidelines developed by the following organisations that are known to deal with patients with diabetes: fourth joint task force of the European Society of Cardiology and other societies on cardiovascular disease prevention in clinical practice,17 the American Diabetes Association18 National Institute for Health and Clinical Excellence (NICE),4 Joint British Society (JBS2),5 Canadian Diabetes Association,19 the Australian National Vascular Disease Prevention Alliance20 and International Diabetes Federation (IDF).21

    Results

    Figure 1 describes the systematic selection process of studies presenting a CVD prediction model applicable to type 2 diabetes patients. After screening 6803 titles, 218 studies remained for abstract screening; main reasons for exclusion were ‘aetiological or therapeutic research’ and ‘outcome other than CVD’. After examining the full text, 45 studies remained, of which 12 studies presented a CVD prediction model specifically developed for the diabetes population, and 33 included diabetes as a factor in the model. The majority of these models were developed in a predominantly Caucasian population and 12 were developed in Asian countries (India, China and Japan). The development study samples ranged from 698 to 1.5 million participants.

    Figure 1
    Figure 1

    Flow chart of systematic review of studies presenting a prediction model for cardiovascular disease (CVD) which can be applied to individuals with type 2 diabetes. *Studies did not present a new risk score and were therefore excluded during the search for prediction models, but were included during the systematic search for validation studies or impact studies.

    Cardiovascular prediction models developed in diabetes patient populations

    Only two prediction models were developed in patients with newly diagnosed diabetes (both from the UK Prospective Diabetes Study (UKPDS study)),22 23 the other development populations were a mixture of people with varying duration of diagnosed diabetes (table 1). Six of these prediction models estimate CHD risk, while four estimate total CVD risk. The majority of the prediction models predicted 5-year risk with an average of eight predictors. The most commonly used predictors in these prediction models were age, sex, duration of diagnosed diabetes, HbA1c concentration and smoking (online appendix table 1).

    View this table:
    Table 1

    Cardiovascular prediction models specifically developed for patients with type 2 diabetes

    Nine out of 12 studies reported the discrimination of their prediction model, with an area under receiver operating characteristic curve (AUC) ranging from 0.68 to 0.85. Eight studies also reported the calibration: all Hosmer–Lemeshow p values were >0.05, indicating no significant lack of calibration. Only half of these models examined the internal validation (performance of the model in the development population when the prediction model is corrected for overoptimism) using split samples or bootstrapping techniques. All studies reported the original model with the β coefficents, and two studies additionally presented a risk chart.

    Cardiovascular prediction models developed in general populations including diabetes as predictor

    Prediction models derived from general populations which included diabetes as a predictor used a variety of cardiovascular outcomes: 14 had CHD as the endpoint, 12 had CVD, eight had stroke and three had myocardial infarction (table 2). Most of these prediction models predicted 10-year risk using an average of eight predictors. The most commonly used predictors, in addition to diabetes, were age, sex, systolic blood pressure, smoking and cholesterol measurements (online appendix table 2).

    View this table:
    Table 2

    Cardiovascular risk models developed in general populations with diabetes as risk factor

    Twenty of 33 studies gave a measure of validation, generally the AUC, which ranged from 0.65 to 0.86. Fourteen studies assessed calibration, with all Hosmer–Lemeshow p-values >0.05. Twelve of these prediction models were internally validated using either split sample, cross validation or bootstrapping techniques. There was a great variety in the way in which the prediction model was presented. The majority presented the original model with the β coefficients, others presented a scoring chart, nomogram or produced a computerised software program that was available through a website.

    Validation studies of the CVD prediction models

    Thirty studies externally validated 14 different prediction models in patients with type 2 diabetes, nine studies validated two versions of the UKPDS risk engine,22 23 10 studies validated three versions of the Framingham Prediction model41 62 65 66 and nine other prediction models were externally validated only once (table 3). The UKPDS risk engine for stroke22 was validated in two studies, which obtained very different results. One study observed an AUC of 0.61 and a poor calibration, while the other observed an AUC of 0.86 and good calibration. The UKPDS risk engine for CHD23 was validated in eight studies. Discrimination ranged from 0.65 to 0.76, and most of these studies observed poor calibration with an overestimation of the risk.

    View this table:
    Table 3

    Overview of the cardiovascular risk scores that have been externally validated in a diabetes population

    The Framingham prediction models by D'Agostino et al41 and by Anderson et al65 66 were validated three times in diabetes populations and the prediction model developed by Wilson et al62 four times. The AUC varied widely, ranging between 0.56 and 0.80 and the calibration was poor (p<0.001). Of the prediction models that were externally validated once, the Fremantle prediction model had the best discrimination with an AUC of 0.84 and a good calibration (p=0.85).

    Impact studies of the CVD prediction models and incorporation in clinical guidelines

    Only three studies examined the impact of applying a prediction model in clinical practice. All three studies used a randomised controlled trial design and examined the impact of the Framingham prediction model78–80 on treatment and prevention of CVD (table 4). Two of these studies were restricted to patients with a history of hypertension and the other study consisted of patients with type 2 diabetes (n=323). Half of them were allocated to an intervention group for which the cardiovascular prediction model was noted on the patient's records, while in the control group it was not. No difference was observed in prescription of glucose control, blood pressure-lowering and lipid-modifying treatments. However, restricting analysis to the high-risk group, patients in the intervention group were more likely to receive lipid-modifying or blood pressure-lowering prescriptions.78

    View this table:
    Table 4

    Overview of the cardiovascular risk scores, that have been quantified for their impact on treatment and cardiovascular complications, and that are incorporated in national/clinical guidelines

    Five out of seven guidelines recommended calculating cardiovascular risk in patients with type 2 diabetes with a prediction models that can be applied to the diabetes population (table 3). The IDF21 and NICE guidelines4 recommended using the UKPDS risk engine,23 as did the Canadian Diabetes Association,19 although this guideline mentioned PROCAM43 and the Strong Heart prediction model47 as well.

    The EASD3 recommends using Framingham41 and DECODE52 as the preferred prediction models for calculating CVD risk. The Australian National Vascular Disease Prevention Alliance20 recommends using the Framingham prediction model41 as well, but they also mention the UKPDS risk engine.23 The JBS25 has developed risk charts based on the Framingham prediction model.41

    Discussion

    This systematic review provides an overview of all CVD prediction models that are specifically developed for, or can be applied to, patients diagnosed with type 2 diabetes to calculate future cardiovascular risk. Twelve prediction models are specifically designed for patients with type 2 diabetes and 33 have been developed in the general population accounting for diabetes as a predictor in the model. Only 31% of these prediction models have been examined for their external validity, with varying results. Interestingly, the impact on treatment and prevention of CVD by applying a prediction model has only been examined for the most commonly known CVD prediction model—that is, the Framingham prediction model.65 Nonetheless, several prediction models are incorporated in guidelines for the management of type 2 diabetes.

    There are an extensive number of CVD prediction models with great variety in their quality and the methodology used to develop them. However, some of the older prediction models were developed when the statistical analysis methods for constructing such models were not yet established. The discriminative ability of both diabetes-specific CVD prediction models and prediction models that account for diabetes in the datasets from which they were developed was mostly good, with a c statistic often >0.70. However, the discriminative ability in new patients (validation studies) varied widely, as did the methods used to assess the performance in new patients. The discrimination of the prediction models designed for the general population was moderate (0.59 to 0.80) and the calibration mostly poor. This could be explained by a difference in incidence of CVD between the general and diabetes populations or that CVD prediction models developed in the general population do not account for diabetes-specific risk factors, as was suggested by Chamnan et al.6 They argued that this problem could be overcome by using only diabetes-specific prediction models for patients with diabetes. However, when diabetes-specific CVD prediction models where tested in other diabetes patient samples, the calibration was also poor with a moderate to good discrimination.

    The more contemporary models, like the DCS,26 Fremantle25 or DARTS31 seem to have the best external validity, but these were validated in other patient populations only once. Therefore, more validation studies on the performance of these prediction models in different diabetes populations are needed.

    The moderate performance of most prediction models suggests that it is difficult to predict CVD in patients with type 2 diabetes. Many biomarkers for CVD are emerging and being tested for their added prognostic value by extending the prediction models with these biomarkers.14 81

    Few studies have assessed the impact of applying a CVD prediction model in clinical practice. We found only one study that assessed the impact of a prediction model (ie, the Framingham risk model). Consequently, it is unknown if the use of prediction models will indeed change treatment of patients with diabetes and thus reduce the number of cardiovascular complications. Even though the effect of applying a prediction model in clinical practice has not been established, guidelines recommend using CVD prediction models to determine appropriate treatment of diabetes. Interestingly, these guidelines refer both to the Framingham prediction model, which was assessed for its impact, but also to other prediction models, many of which have not been comprehensively validated. For instance, the EASD refers to the DECODE prediction model, which has only been validated once, with moderate performance. Other guidelines refer to the UKPDS risk engine, which indeed has been validated by several studies, but in most studies it showed poor calibration and severely overestimated the risk. Considering that the validation and impact of most prediction models has not been assessed, there is a great need for such studies to support their incorporation in clinical guidelines.

    Prediction models which have been developed in the general population and do not correct for diabetes as a factor were excluded from this review. Therefore we did not include, for example, the Systematic COronary Risk Evaluation (SCORE).82 This model predicts fatal CVD over 10 years based on five predictors—namely, sex, age, smoking, systolic blood pressure and either total cholesterol or ratio total/high-density lipoprotein-cholesterol. The predictive ability of SCORE in patients with type 2 diabetes, however, has been assessed by three studies and was similar to other CVD prediction models included in our review.72 74 83

    In conclusion, three stages of constructing a good prediction model can be identified: (1) development and internal validation of a model; (2) external validation and (3) assessment of the impact of using the model in practice. Many studies exist that present a CVD prediction model that can be applied to the diabetes population. A minority of studies have externally validated this large number of clinical prediction models in a diabetes population. Assessment of the impact on diabetes treatment and complications has been made for only one prediction model. Nevertheless, several prediction models are incorporated in guidelines for the management of type 2 diabetes and prevention of cardiovascular complications. New studies investigating prediction of CVD among patients with type 2 diabetes should, in our view, focus on further validating the performance of existing prediction models in new patients,10 improving the predictive performance of these existing models by adjusting them to local circumstances or adding new predictors14 81 84 and assessing their impact on treatment and prevention of cardiovascular events instead of developing new prediction models.11 16

    Acknowledgments

    This research was performed within the framework of CTMM, the Center for Translational Molecular Medicine (http://www.ctmm.nl), project PREDICCt (grant 01C-104), and supported by the Netherlands Heart Foundation, Dutch Diabetes Research Foundation and Dutch Kidney Foundation.

    Appendix

    Search terms used in this study

    Search term 1

    The search term used for identification of prediction models that are devolved in, or can be applied to, diabetes individuals: ((Validat$ OR Predict$.ti. OR Rule$) OR (Predict$ AND (Outcome$ OR Risk$ OR Model$)) OR (Decision$ AND (Model$ OR Clinical$ OR Logistic Models/)) OR (Prognostic AND (History OR Variable$ OR Criteria OR Scor$ OR Characteristic$ OR Finding$ OR Factor$ OR Model$)) OR (“risk score”[All fields] OR “prediction model”[All fields] OR “prediction rule”[All fields] OR “risk assessment” [All fields] OR “algorithm”[All fields])) AND (cardiovascular OR coronary OR cerebrovascular OR heart OR stroke) AND (diabetes OR “diabetes mellitus” OR “type 2 diabetes”) NOT (Animals[MeSH] NOT Humans[MeSH]).

    Search term 2

    Additional search term used to identify all validation studies of the prediction models: ((“diabetes”[All fields] OR “diabetes mellitus”[All fields] OR “type 2 diabetes”[All fields]) AND (“risk score”[All fields] OR “prediction model”[All fields] OR “risk model”[All fields] OR “predicting”[All fields] OR “predictive model”[All fields] OR “prediction tool”[All fields] OR “prediction rule”[All fields] OR “risk assessment”[All fields] OR “algorithm”[All fields] OR “validation”[All fields] or “discrimination”[All fields] OR “calibration”[All fields]) AND (“cardiovascular”[All fields] OR “coronary heart”[All fields] OR “heart”[All fields] OR “stroke”[All fields] OR “cerebrovascular”[All fields]) AND (Humans[Mesh])).

    Search term 3

    Search term used to identify all impact studies, which is combined with each specific risk scores acronym, or if not applicable the name of the cohort in which the score was developed or first author:

    (Effectiveness [tiab] OR Comparing [tiab] OR Compared [tiab]OR Evaluate [tiab]) AND (Algorithm [tiab] OR Strategy [tiab] OR Managed [tiab] OR Management [tiab] OR Decision [tiab]) AND (cardiovascular OR coronary OR cerebrovascular OR heart OR stroke) AND (diabetes OR “diabetes mellitus” OR “type 2 diabetes”) NOT (Animals[MeSH] NOT Humans[MeSH]).

    References

    1. ↵
      1. Sarwar N,
      2. Gao P,
      3. Seshasai SR,
      4. et al
      ; Emerging Risk Factors Collaboration. Diabetes mellitus, fasting blood glucose concentration, and risk of vascular disease: a collaborative meta-analysis of 102 prospective studies. Lancet 2010;375:2215–22.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Woodward M,
      2. Zhang X,
      3. Barzi F,
      4. et al
      ; Asia Pacific Cohort Studies Collaboration. The effects of diabetes on the risks of major cardiovascular diseases and death in the Asia-Pacific region. Diabetes Care 2003;26:360–6.
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. Ryden L,
      2. Standl E,
      3. Bartnik M,
      4. et al
      ; Task Force on Diabetes and Cardiovascular Diseases of the European Society of Cardiology (ESC); European Association for the Study of Diabetes (EASD). Guidelines on diabetes, pre-diabetes, and cardiovascular diseases: executive summary. The Task Force on Diabetes and Cardiovascular Diseases of the European Society of Cardiology (ESC) and of the European Association for the Study of Diabetes (EASD). Eur Heart J 2007;28:88–136.
      OpenUrlFREE Full Text
    4. ↵
      National Collaborating Centre for Chronic Conditions. Type 2 Diabetes National Clinical Guideline for Management in Primary and Secondary Care (update). http://www.nice.org.uk/nicemedia/live/11983/40803/40803.pdf (accessed Jul 2011).
    5. ↵
      British Cardiac SocietyBritish Hypertension Society, Diabetes UK, HEART UK, Primary Care Cardiovascular Society, Stroke Association. JBS 2: Joint British Societies' guidelines on prevention of cardiovascular disease in clinical practice. Heart 2005;91(Suppl 5):v1–52.
      OpenUrlFREE Full Text
    6. ↵
      1. Chamnan P,
      2. Simmons RK,
      3. Sharp SJ,
      4. et al
      . Cardiovascular risk assessment scores for people with diabetes: a systematic review. Diabetologia 2009;52:2001–14.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Stephens JW,
      2. Ambler G,
      3. Vallance P,
      4. et al
      . Cardiovascular risk and diabetes. Are the methods of risk prediction satisfactory? Eur J Cardiovasc Prev Rehabil 2004;11:521–8.
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Song M,
      2. Alexander CM,
      3. Mavros P,
      4. et al
      . Use of the UKPDS outcomes model to predict all-cause mortality in US adults with type 2 diabetes mellitus: comparison of predicted versus observed mortality. Diabetes Res Clin Pract 2011;91:121–6.
      OpenUrlCrossRefPubMed
    9. ↵
      1. Pellegrini E,
      2. Maurantonio M,
      3. Giannico IM,
      4. et al
      . Risk for cardiovascular events in an Italian population of patients with type 2 diabetes. Nutr Metab Cardiovasc Dis 2011;21:885–92.
      OpenUrlCrossRefPubMed
    10. ↵
      1. Atlman DG,
      2. Vergouwe Y,
      3. Royston P,
      4. et al
      . Prognosis and prognostic research: validating a prognostic model. BMJ 2009;338:b605.
      OpenUrlFREE Full Text
    11. ↵
      1. Moons KG,
      2. Altman DG,
      3. Vergouwe Y,
      4. et al
      . Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009;338:b606.
      OpenUrlFREE Full Text
    12. ↵
      1. Moons KG,
      2. Royston P,
      3. Vergouwe Y,
      4. et al
      . Prognosis and prognostic research: what, why, and how? BMJ 2009;338:b375.
      OpenUrlFREE Full Text
    13. ↵
      1. Royston P,
      2. Moons KG,
      3. Altman DG,
      4. et al
      . Prognosis and prognostic research: developing a prognostic model. BMJ 2009;338:b604.
      OpenUrlFREE Full Text
    14. ↵
      1. Hlatky MA,
      2. Greenland P,
      3. Arnett DK,
      4. et al
      ; American Heart Association Expert Panel on Subclinical Atherosclerotic Diseases and Emerging Risk Factors and the Stroke Council. Criteria for evaluation of novel markers of cardiovascular risk: A scientific statement from the American heart association. Circulation 2009;119:2408–16.
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Justice AC,
      2. Covinsky KE,
      3. Berlin JA
      . Assessing the generalizability of prognostic information. Ann Intern Med 1999;130:515–24.
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Reilly BM,
      2. Evans AT
      . Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med 2006;144:201–9.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Graham I,
      2. Atar D,
      3. Borch-Johnsen K,
      4. et al
      ; European Society of Cardiology (ESC); European Association for Cardiovascular Prevention and Rehabilitation (EACPR); Council on Cardiovascular Nursing; European Association for Study of Diabetes (EASD); International Diabetes Federation Europe (IDF-Europe); European Stroke Initiative (EUSI); Society of Behavioural Medicine (ISBM); European Society of Hypertension (ESH); WONCA Europe (European Society of General Practice/Family Medicine); European Heart Network (EHN); European Atherosclerosis Society (EAS). European guidelines on cardiovascular disease prevention in clinical practice: Full text: Fourth Joint Task Force of the European Society of Cardiology and other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of nine societies and by invited experts). Eur J Cardiovasc Prev Rehabil 2007;14(Suppl 2):S1–113.
      OpenUrlAbstract/FREE Full Text
    18. ↵
      American Diabetes Association. Standards of medical care in diabetes–2011. Diabetes Care Jan 2011;34(Suppl 1):S11–61.
      OpenUrlFREE Full Text
    19. ↵
      Canadian Diabetes Association. Clinical Practice Guidelines for the Prevention and Management of Diabetes in Canada. Canadian Journal of Diabetes 2008;32(Suppl 1):S1–201.
      OpenUrl
    20. ↵
      National Vascular Disease Prevention Alliance. Guidelines for the Assessment of Absolute Cardiovascular Disease Risk. http://www.heartfoundation.org.au/SiteCollectionDocuments/absolute-risk-assessement.pdf (accessed Jul 2011).
    21. ↵
      International Diabetes Federation. IDF Clinical Guidelines Task Force. Global Guideline for Type 2 Diabetes. http://www.idf.org/webdata/docs/IDF%20GGT2D.pdf (accessed Jul 2011).
    22. ↵
      1. Kothari V,
      2. Stevens RJ,
      3. Adler AI,
      4. et al
      . UKPDS 60: risk of stroke in type 2 diabetes estimated by the UK Prospective Diabetes Study risk engine. Stroke 2002;33:1776–81.
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Stevens RJ,
      2. Kothari V,
      3. Adler AI,
      4. et al
      ; United Kingdom Prospective Diabetes Study (UKPDS) Group. The UKPDS risk engine: a model for the risk of coronary heart disease in Type II diabetes (UKPDS 56). Clin Sci (Lond) 2001;101:671–9.
      OpenUrlCrossRefPubMed
      1. Kengne AP,
      2. Patel A,
      3. Marre M,
      4. et al
      ; (on behalf of the ADVANCE Collaborative Group). Contemporary model for cardiovascular risk prediction in people with type 2 diabetes. Eur J Cardiovasc Prev Rehabil 2011;18:393–8.
      OpenUrlAbstract/FREE Full Text
    25. ↵
      1. Davis WA,
      2. Knuiman MW,
      3. Davis TM
      . An Australian cardiovascular risk equation for type 2 diabetes: the Fremantle Diabetes Study. Intern Med J 2010;40:286–92.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Elley CR,
      2. Robinson E,
      3. Kenealy T,
      4. et al
      . Derivation and validation of a new cardiovascular risk score for people with type 2 diabetes: the new zealand diabetes cohort study. Diabetes Care 2010;33:1347–52.
      OpenUrlAbstract/FREE Full Text
      1. Cederholm J,
      2. Eeg-Olofsson K,
      3. Eliasson B,
      4. et al
      ; Swedish National Diabetes Register. Source Department of Public Health and Carin. Risk prediction of cardiovascular disease in type 2 diabetes: a risk equation from the Swedish National Diabetes Register. Diabetes Care 2008;31:2038–43.
      OpenUrlAbstract/FREE Full Text
      1. Yang X,
      2. So WY,
      3. Kong AP,
      4. et al
      . Development and validation of a total coronary heart disease risk score in type 2 diabetes mellitus. Am J Cardiol 2008;101:596–601.
      OpenUrlCrossRefPubMedWeb of Science
      1. Yang X,
      2. Ma RC,
      3. So WY,
      4. et al
      . Development and validation of a risk score for hospitalization for heart failure in patients with Type 2 diabetes mellitus. Cardiovasc Diabetol 2008;7:9.
      OpenUrlCrossRefPubMed
      1. Yang X,
      2. So WY,
      3. Kong AP,
      4. et al
      . Development and validation of stroke risk equation for Hong Kong Chinese patients with type 2 diabetes: the Hong Kong Diabetes Registry. Diabetes Care 2007;30:65–70.
      OpenUrlAbstract/FREE Full Text
    31. ↵
      1. Donnan PT,
      2. Donnelly L,
      3. New JP,
      4. et al
      . Derivation and validation of a prediction score for major coronary heart disease events in a UK type 2 diabetic population. Diabetes Care 2006;29:1231–6.
      OpenUrlAbstract/FREE Full Text
      1. Folsom AR,
      2. Chambless LE,
      3. Duncan BB,
      4. et al
      ; Atherosclerosis Risk in Communities Study Investigators. Prediction of coronary heart disease in middle-aged adults with diabetes. Diabetes Care 2003;26:2777–84.
      OpenUrlAbstract/FREE Full Text
      1. Yudkin JS,
      2. Chaturvedi N
      . Developing risk stratification charts for diabetic and nondiabetic subjects. Diabet Med 1999;16:219–27.
      OpenUrlCrossRefPubMedWeb of Science
      1. Chien KL,
      2. Su TC,
      3. Hsu HC,
      4. et al
      . Constructing the prediction model for the risk of stroke in a Chinese population: report from a cohort study in Taiwan. Stroke 2010;41:1858–64.
      OpenUrlAbstract/FREE Full Text
      1. Hippisley-Cox J,
      2. Coupland C,
      3. Robson J,
      4. et al
      . Derivation, validation, and evaluation of a new QRISK model to estimate lifetime risk of cardiovascular disease: cohort study using QResearch database. BMJ 2010;341:c6624.
      OpenUrlAbstract/FREE Full Text
      1. McGorrian C,
      2. Yusuf S,
      3. Islam S,
      4. et al
      ; INTERHEART Investigators. Estimating modifiable coronary heart disease risk in multiple regions of the world: the INTERHEART Modifiable Risk Score. Eur Heart J 2011;32:581–9.
      OpenUrlAbstract/FREE Full Text
      1. Arima H,
      2. Yonemoto K,
      3. Doi Y,
      4. et al
      . Development and validation of a cardiovascular risk prediction model for Japanese: the Hisayama study. Hypertens Res 2009;32:1119–22.
      OpenUrlCrossRefPubMed
      1. Ishikawa S,
      2. Matsumoto M,
      3. Kayaba K,
      4. et al
      ; Jichi Medical School (JMS) Cohort Study Group. Risk charts illustrating the 10-year risk of stroke among residents of Japanese rural communities: the JMS Cohort Study. J Epidemiol 2009;19:101–6.
      OpenUrlCrossRefPubMed
      1. Matsumoto M,
      2. Ishikawa S,
      3. Kayaba K,
      4. et al
      ; Jichi Medical School (JMS) Cohort Study Group. Risk charts illustrating the 10-year risk of myocardial infarction among residents of Japanese rural communities: the JMS Cohort Study. J Epidemiol 2009;19:94–100.
      OpenUrlCrossRefPubMed
      1. Pencina MJ,
      2. D'Agostino RB Sr.,
      3. Larson MG,
      4. et al
      . Predicting the 30-year risk of cardiovascular disease: the framingham heart study. Circulation 2009;119:3078–84.
      OpenUrlAbstract/FREE Full Text
    41. ↵
      1. D'Agostino RB Sr.,
      2. Vasan RS,
      3. Pencina MJ,
      4. et al
      . General cardiovascular risk profile for use in primary care: The Framingham heart study. Circulation 2008;117:743–53.
      OpenUrlAbstract/FREE Full Text
      1. Hippisley-Cox J,
      2. Coupland C,
      3. Vinogradova Y,
      4. et al
      . Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ 2008;336:1475–82.
      OpenUrlAbstract/FREE Full Text
    43. ↵
      1. Assmann G,
      2. Schulte H,
      3. Cullen P,
      4. et al
      . Assessing risk of myocardial infarction and stroke: new data from the Prospective Cardiovascular Munster (PROCAM) study. Eur J Clin Invest 2007;37:925–32.
      OpenUrlCrossRefPubMedWeb of Science
      1. Ridker PM,
      2. Buring JE,
      3. Rifai N,
      4. et al
      . Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA 2007;297:611–19.
      OpenUrlCrossRefPubMedWeb of Science
      1. Woodward M,
      2. Brindle P,
      3. Tunstall-Pedoe H
      ; SIGN group on risk estimation. Adding social deprivation and family history to cardiovascular risk assessment: the ASSIGN score from the Scottish Heart Health Extended Cohort (SHHEC). Heart 2007;93:172–6.
      OpenUrlAbstract/FREE Full Text
    46. Asia Pacific Cohort Studies Collaboration. Coronary risk prediction for those with and without diabetes. Eur J Cardiovasc Prev Rehabil 2006;13:30–6.
      OpenUrlAbstract/FREE Full Text
    47. ↵
      1. Lee ET,
      2. Howard BV,
      3. Wang W,
      4. et al
      . Prediction of coronary heart disease in a population with high prevalence of diabetes and albuminuria: the Strong Heart Study. Circulation 2006;113:2897–905.
      OpenUrlAbstract/FREE Full Text
      1. Mainous AG 3rd.,
      2. Koopman RJ,
      3. Diaz VA,
      4. et al
      . A coronary heart disease risk score based on patient-reported information. Am J Cardiol 2007;99:1236–41.
      OpenUrlCrossRefPubMedWeb of Science
      1. Wu Y,
      2. Liu X,
      3. Li X,
      4. et al
      ; USA-PRC Collaborative Study of Cardiovascular and Cardiopulmonary Epidemiology Research Group; China Multicenter Collaborative Study of Cardiovascular Epidemiology Research Group. Estimation of 10-year risk of fatal and nonfatal ischemic cardiovascular diseases in Chinese adults. Circulation 2006;114:2217–25.
      OpenUrlAbstract/FREE Full Text
      1. Ferrario M,
      2. Chiodini P,
      3. Chambless LE,
      4. et al
      ; CUORE Project Research Group. Prediction of coronary events in a low incidence population. Assessing accuracy of the CUORE Cohort Study prediction equation. Int J Epidemiol 2005;34:413–21.
      OpenUrlAbstract/FREE Full Text
      1. Menotti A,
      2. Lanti M,
      3. gabiti-Rosei E,
      4. et al
      . Riskard 2005. New tools for prediction of cardiovascular disease risk derived from Italian population studies. Nutr Metab Cardiovasc Dis 2005;15:426–40.
      OpenUrlCrossRefPubMedWeb of Science
    52. ↵
      1. Balkau B,
      2. Hu G,
      3. Qiao Q,
      4. et al
      . Prediction of the risk of cardiovascular mortality using a score that includes glucose as a risk factor. The DECODE Study. Diabetologia 2004;47:2118–28.
      OpenUrlCrossRefPubMedWeb of Science
      1. Liu J,
      2. Hong Y,
      3. D'Agostino RB Sr.,
      4. et al
      . Predictive value for the Chinese population of the Framingham CHD risk assessment tool compared with the Chinese Multi-Provincial Cohort Study. JAMA 2004;291:2591–9.
      OpenUrlCrossRefPubMedWeb of Science
      1. Pignone M,
      2. Sheridan SL,
      3. Lee YZ,
      4. et al
      . Heart to Heart: a computerized decision aid for assessment of coronary heart disease risk and the impact of risk-reduction interventions for primary prevention. Prev Cardiol 2004;7:26–33.
      OpenUrlPubMed
      1. Schau B,
      2. Boysen G,
      3. Truelsen T,
      4. et al
      . Development and validation of a model to estimate stroke incidence in a population. J Stroke Cerebrovasc Dis 2003;12:22–8.
      OpenUrlCrossRefPubMed
      1. Assmann G,
      2. Cullen P,
      3. Schulte H
      . Simple scoring scheme for calculating the risk of acute coronary events based on the 10-year follow-up of the prospective cardiovascular Munster (PROCAM) study. Circulation 2002;105:310–15.
      OpenUrlAbstract/FREE Full Text
      1. Lumley T,
      2. Kronmal RA,
      3. Cushman M,
      4. et al
      . A stroke prediction score in the elderly: validation and Web-based application. J Clin Epidemiol 2002;55:129–36.
      OpenUrlCrossRefPubMedWeb of Science
      1. Menotti A,
      2. Lanti M,
      3. Puddu PE,
      4. et al
      . The risk functions incorporated in Riscard 2002: a software for the prediction of cardiovascular risk in the general population based on Italian data. Ital Heart J 2002;3:114–21.
      OpenUrlPubMed
      1. Moons KG,
      2. Bots ML,
      3. Salonen JT,
      4. et al
      . Prediction of stroke in the general population in Europe (EUROSTROKE): Is there a role for fibrinogen and electrocardiography? J Epidemiol Community Health 2002;56(Suppl 1):i30–6.
      OpenUrlAbstract/FREE Full Text
      1. Thomsen TF,
      2. Davidsen M,
      3. Ibsen H,
      4. et al
      . A new method for CHD prediction and prevention based on regional risk scores and randomized clinical trials; PRECARD and the Copenhagen Risk Score. J Cardiovasc Risk 2001;8:291–7.
      OpenUrlCrossRefPubMedWeb of Science
      1. Knuiman MW,
      2. Vu HT,
      3. Bartholomew HC
      . Multivariate risk estimation for coronary heart disease: the Busselton Health Study. Aust N Z J Public Health 1998;22:747–53.
      OpenUrlPubMedWeb of Science
    62. ↵
      1. Wilson PW,
      2. D'Agostino RB,
      3. Levy D,
      4. et al
      . Prediction of coronary heart disease using risk factor categories. Circulation 1998;97:1837–47.
      OpenUrlAbstract/FREE Full Text
      1. Wood D,
      2. De Backer G,
      3. Faergeman O,
      4. et al
      . Prevention of coronary heart disease in clinical practice. Recommendations of the Second Joint Task Force of European and other Societies on Coronary Prevention. Eur Heart J 1998;19:1434–503.
      OpenUrlFREE Full Text
      1. Zodpey SP,
      2. Kulkarni HR,
      3. Vasudeo ND,
      4. et al
      . A risk scoring system for prediction of coronary heart disease based on multivariate analysis: development and validation. Indian Heart J 1994;46:77–83.
      OpenUrlPubMed
    65. ↵
      1. Anderson KM,
      2. Odell PM,
      3. Wilson PW,
      4. et al
      . Cardiovascular disease risk profiles. Am Heart J 1991;121:293–8.
      OpenUrlCrossRefPubMedWeb of Science
    66. ↵
      1. Anderson KM,
      2. Wilson PW,
      3. Odell PM,
      4. et al
      . An updated coronary risk profile. A statement for health professionals. Circulation 1991;83:356–62.
      OpenUrlFREE Full Text
      1. Kengne AP,
      2. Patel A,
      3. Colagiuri S,
      4. et al
      . The Framingham and UK Prospective Diabetes Study (UKPDS) risk equations do not reliably estimate the probability of cardiovascular events in a large ethnically diverse sample of patients with diabetes: the Action in Diabetes and Vascular Disease: Preterax and Diamicron-MR Controlled Evaluation (ADVANCE) Study. Diabetologia 2010;53:821–31.
      OpenUrlCrossRefPubMedWeb of Science
      1. Davis WA,
      2. Colagiuri S,
      3. Davis TM
      . Comparison of the Framingham and United Kingdom Prospective Diabetes Study cardiovascular risk equations in Australian patients with type 2 diabetes from the Fremantle Diabetes Study. Med J Aust 2009;190:180–4.
      OpenUrlPubMedWeb of Science
      1. van Dieren S,
      2. Peelen LM,
      3. Nothlings U,
      4. et al
      . External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes. Diabetologia 2011;54:264–70.
      OpenUrlCrossRefPubMedWeb of Science
      1. Song SH,
      2. Brown PM
      . Coronary heart disease risk assessment in diabetes mellitus: comparison of UKPDS risk engine with Framingham risk assessment function and its clinical implications. Diabet Med 2004;21:238–45.
      OpenUrlCrossRefPubMedWeb of Science
      1. Guzder RN,
      2. Gatling W,
      3. Mullee MA,
      4. et al
      . Prognostic value of the Framingham cardiovascular risk equation and the UKPDS risk engine for coronary heart disease in newly diagnosed Type 2 diabetes: results from a United Kingdom study. Diabet Med 2005;22:554–62.
      OpenUrlCrossRefPubMedWeb of Science
    72. ↵
      1. van der Heijden AA,
      2. Ortegon MM,
      3. Niessen LW,
      4. et al
      . Prediction of coronary heart disease risk in a general, pre-diabetic, and diabetic population during 10 years of follow-up: accuracy of the Framingham, SCORE, and UKPDS risk functions: The Hoorn Study. Diabetes Care 2009;32:2094–8.
      OpenUrlAbstract/FREE Full Text
      1. Simmons RK,
      2. Coleman RL,
      3. Price HC,
      4. et al
      . Performance of the UK Prospective Diabetes Study Risk Engine and the Framingham Risk Equations in Estimating Cardiovascular Disease in the EPIC- Norfolk Cohort. Diabetes Care 2009;32:708–13.
      OpenUrlAbstract/FREE Full Text
    74. ↵
      1. Coleman RL,
      2. Stevens RJ,
      3. Retnakaran R,
      4. et al
      . Framingham, SCORE, and DECODE risk equations do not provide reliable cardiovascular risk estimates in type 2 diabetes. Diabetes Care 2007;30:1292–3.
      OpenUrlFREE Full Text
      1. Game FL,
      2. Jones AF
      . Coronary heart disease risk assessment in diabetes mellitus–a comparison of PROCAM and Framingham risk assessment functions. Diabet Med 2001;18:355–9.
      OpenUrlCrossRefPubMedWeb of Science
      1. Game FL,
      2. Bartlett WA,
      3. Bayly GR,
      4. et al
      . Comparative accuracy of cardiovascular risk prediction methods in patients with diabetes mellitus. Diabetes Obes Metab 2001;3:279–86.
      OpenUrlCrossRefPubMedWeb of Science
      1. McEwan P,
      2. Williams JE,
      3. Griffiths JD,
      4. et al
      . Evaluating the performance of the Framingham risk equations in a population with diabetes. Diabet Med 2004;21:318–23.
      OpenUrlCrossRefPubMedWeb of Science
    78. ↵
      1. Hall LM,
      2. Jung RT,
      3. Leese GP
      . Controlled trial of effect of documented cardiovascular risk scores on prescribing. Br Med J 2003;326:251–2.
      OpenUrlFREE Full Text
    79. ↵
      1. Hetlevik I,
      2. Holrnen J,
      3. Krüger Ø
      . Implementing clinical guidelines in the treatment of hypertension in general practice: Evaluation of patient outcome related to implementation of a computer-based clinical decision support system. Scand J Prim Health Care 1999;17:35–40.
      OpenUrlCrossRefPubMedWeb of Science
    80. ↵
      1. Montgomery AA,
      2. Fahey T,
      3. Peters TJ,
      4. et al
      . Evaluation of computer based clinical decision support system and risk chart for management of hypertension in primary care: randomised controlled trial. BMJ 2000;320:686–90.
      OpenUrlAbstract/FREE Full Text
    81. ↵
      1. Moons KG
      . Criteria for scientific evaluation of novel markers: a perspective. Clin Chem 2010;56:537–41.
      OpenUrlFREE Full Text
    82. ↵
      1. Conroy RM,
      2. Pyörälä K,
      3. Fitzgerald AP,
      4. et al
      ; SCORE project group. Estimation of ten-year risk of fatal cardiovascular disease in Europe: The SCORE project. Eur Heart J 2003;24:987–1003.
      OpenUrlAbstract/FREE Full Text
    83. ↵
      1. Chen L,
      2. Tonkin AM,
      3. Moon L,
      4. et al
      . Recalibration and validation of the SCORE risk chart in the Australian population: the AusSCORE chart. Eur J Cardiovasc Prev Rehabil 2009;16:562–70.
      OpenUrlAbstract/FREE Full Text
    84. ↵
      1. Steyerberg EW,
      2. Borsboom GJ,
      3. van Houwelingen HC,
      4. et al
      . Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 2004;23:2567–86.
      OpenUrlCrossRefPubMedWeb of Science

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Funding KGMM receives funding from the Netherlands Organisation for Scientific Research (project 9120.8004 and 918.10.615). APK was in receipt of a fellowship from the International Society of Hypertension.

    • Competing interests None.

    • Provenance and peer review Commissioned; externally peer reviewed.