Objective This study aims to examine the validity of the Framingham general cardiovascular disease (CVD) risk chart in a primary care setting.
Design This is a 10-year retrospective cohort study.
Setting A primary care clinic in a teaching hospital in Malaysia.
Participants 967 patients’ records were randomly selected from patients who were attending follow-up in the clinic.
Main outcome measures Baseline demographic data, history of diabetes and smoking, blood pressure (BP), and serum lipids were captured from patient records in 1998. Each patient's Framingham CVD score was computed from these parameters. All atherosclerotic CVD events occurring between 1998 and 2007 were counted.
Results In 1998, mean age was 57 years with 33.8% men, 6.1% smokers, 43.3% diabetics and 59.7% hypertensive. Median BP was 140/80 mm Hg and total cholesterol 6.0 mmol/L (1.3). The predicted median Framingham general CVD risk score for the study population was 21.5% (IQR 1.2–30.0) while the actual CVD events that occurred in the 10 years was 13.1% (127/967). The median CVD points for men was 30.0, giving them a CVD risk of more than 30%; for women it is 18.5, a CVD risk of 21.5%. Our study found that the Framingham general CVD risk score to have moderate discrimination with an area under the receiver operating characteristic curve (AUC) of 0.63. It also discriminates well for Malay (AUC 0.65, p=0.01), Chinese (AUC 0.60, p=0.03), and Indians (AUC 0.65, p=0.001). There was good calibration with Hosmer-Lemeshow test χ2=3.25, p=0.78.
Conclusions Taking into account that this cohort of patients were already on treatment, the Framingham General CVD Risk Prediction Score predicts fairly accurately for men and overestimates somewhat for women. In the absence of local risk prediction charts, the Framingham general CVD risk prediction chart is a reasonable alternative for use in a multiethnic group in a primary care setting.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
The strength of our study is that it is done in a primary care setting where the cardiovascular disease (CVD) risk profiles are different from that in secondary care.
Another strength is that it examines the validity and applicability of the Framingham general CVD risk tool in a multiethnic primary care population.
Our sample size is large enough to give us a better picture of CVD risk of a multiethnic population in Asia where data about the use and applicability of any CVD risk prediction tool has not extensively been validated to particularly include non-fatal events.
Our study found the Framingham General CVD risk score to have a good calibration and hence, it can be used by all the three different ethnic groups.
However, most of our patients were already on treatment and this resulted in a seemingly overestimation of cardiovascular risk.
As this is a retrospective study, there will be missing data to some extent. However, we were able to compare the characteristics of those with missing data to those included in the analysis and found no clinical differences between them.
Guidelines on management of cardiovascular (CV) diseases (CVDs) recommend assessing overall CV risk to determine the necessity of pharmacotherapy.1–9 To date, many CVD risk assessment tools have devised.10 The most well studied is the Framingham Coronary Heart Disease (CHD) risk prediction Chart.11 ,12 This tool has been validated in different populations and was found to overestimate risk particularly in women, the older age group, British men and European population and certain ethnicities with lower background risk of CVD, for example, the Chinese population.1 ,13–17
Furthermore, it has been criticised that the Framingham CHD risk score only predicts risk for fatal and non-fatal coronary events and does not take into account strokes as well as patients already receiving antihypertensive therapy, which is the case with many individuals.
Hence, the Framingham general CVD risk score tool for primary care was developed in 2008.18 This newer tool now includes strokes, heart failure and peripheral artery disease, and adjusts for patients already on antihypertensive therapy. However, this risk score chart tool has not been widely validated particularly in the communities with different races. Malaysia is a multiethnic country with a population of 30 million. The major ethnic group is the Malay (50.3%) followed by Chinese (21.7%), Indians (6.5%) and others (21.5%).19
Hence, our aim was to examine the validity of the Framingham general CVD in a multiethnic primary care population.
Materials and methods
This is a retrospective cohort study. We computed the Framingham general CVD risk score retrospectively in a randomly selected cohort of patients registered with an urban primary care clinic. This selected sample consisted of adult patients aged 30 years and older, who were free of any cardiovascular events at baseline and who had complete data needed to compute their overall CV risk. The study was conducted in an outpatient clinic of an academic teaching hospital.
There were 1536 patients in our original cohort. We excluded 563 patients as they did not have all the variables needed to calculate the Framingham general CVD risk score. Another six patients were excluded as we could not ascertain their CVD status by the end of 2007. Thus, a total of 967 patients (62.9%) were eligible for analysis (figure 1). Of these, 912 (94.3%) patients completed 10 years follow-up and only 55 (5.7%) patients defaulted follow-up. Of these, 26 (2.7%) had died with 19 (73%) deaths from non-CVD causes.
All adult patients 30 years and older, without any cardiovascular events and with documented blood pressure (BP) readings whether they were on or not on treatment, total cholesterol, high-density lipoprotein (HDL) cholesterol, smoking status, and presence or absence of diabetes mellitus (DM) were eligible for this study. These parameters were needed to compute each individual's general CVD risk level.
This sample was randomly selected using a computer-generated number based on the patient's unique registration number with the clinic. All patient records were in paper form.
We extracted the patients’ sociodemographic data and clinical information based on the first entry in 1998 from their records manually, according to a predetermined proforma (clinical report form). Use of hypoglycaemic and lipid-lowering agents were also captured. Using age, SBP (systolic BP; treated or not treated) total cholesterol, HDL cholesterol levels, smoking and diabetes status, we computed each individual's general CVD risk score.
CVD events, namely, fatal and non-fatal CHD, fatal and non-fatal strokes, heart failure and peripheral artery disease were also captured from the records. For those who did not complete their subsequent 10-year follow-up at our clinic (n=24), we traced and examined the case records from the main hospital to determine their CVD outcome. For those patients who did not complete their follow-up in our clinic, we contacted them individually (n=31) to ascertain their CVD status and for any events fatal or non-fatal. Each CVD event was counted as one event in individuals who had more than one event during the 10-year follow-up. We further categorised the 10-year CVD risk as low (<10% 10-year CVD risk), medium (10–20% CVD risk), or high >20% CVD risk.20
Blood pressure (BP) was measured by our attending doctor using a mercury sphygmomanometer as part of the daily routine clinical practice. Diagnosis of hypertension was made in accordance with standard recommendations.7 Target control BP is defined as BP<140/90 mm Hg for patients without DM and <130/80 mm Hg for patients with DM. DM is defined as documented by the attending physician or the use of hypoglycaemic agents or both. Smokers were defined if they were still smoking. Non-smokers were those who never smoked or currently not smoking regardless of when they had stopped smoking. Total cholesterol, low-density lipoprotein (LDL) and HDL cholesterol were also collected. All blood tests were performed in our teaching hospital's laboratory, which is certified by the Royal College of Pathologists of Australasia standards.
All statistical analysis was carried out using the Statistical Package for Social Sciences (SPSS V.21). Continuous data are described with mean and SD if the distribution is normal. When the data is a skewed distribution, median and IQR (25–75th centile) are used. Categorical data are reported as proportions (percentage). The χ2 test (χ2) or Fisher exact tests were used for the categorical or dichotomous variables.
Discrimination is defined as the ability of the risk prediction model in distinguishing patients who suffered from a CVD event from those who do not use an overall c index. The c index is analogous to the area under the receiver operating characteristic curve (AUC). Discrimination is defined as good when c index is closer to 1, whereas a value of 0.5 implies that the risk score tool is no better than chance.21 A value of 0.75 is considered as good discrimination. Values between 0.51 and 0.74 are considered moderate and ≤0.5 as poor.22
Calibration was used to assess if the observed 10-year CVD events differed significantly from the predicted CVD events.8 The calibration of the Framingham general CVD risk score was determined using Hosmer-Lemeshow test.23 A χ2 value of greater than 20 or a p value of less than 0.05 indicates poor calibration.
A total of 967 of 1536 patient records from 1998 (baseline) were entered into our analysis.
Table 1 shows the demographic data of our cohort. Our study population was made up of three main ethnic groups, namely, the Chinese (45.4%), Malays (23.3%) and Indians (30.2%). The mean age was 56.9 (9.9) years and two-third (66.2%) were females. At baseline in 1998, over half (59.7%) had hypertension, less than half (43.3%) had diabetes and less than 10% (6.1%) were current smokers, 80.9% (n=339) of the patients with DM were receiving treatment with either oral hypoglycaemic agents and/or insulin. The mean SBP was 140.7±18.6 mm Hg and more than half (59.7%) of them were on antihypertensive agents. However, only 9.8% of the population (n=95) received statin therapy in 1998. We also compared the baseline characteristics for those who were entered into our analysis to the patients who were excluded. Basically, there were no significant differences between their baseline characteristics except for the smoking status and statin use.
Table 1 also demonstrates the change in CVD risk factors between baseline and at the end of 10 years. In spite of the increase in the prevalence of diabetes from 43.3% to 46.3%, glycated haemoglobin (HbA1c) control actually improved from a mean of 7.8% to 7.5%. The mean SBP was reduced by 5.8 mm Hg, from 140.7 to 134.9 mm Hg. The control rate for BP improved from 41.3% to 55.6%. There was an increase in the use of renin-angiotensin system (RAS) blockers from 7.3% to 48.1%. The number of patients receiving statin increased significantly from 9.8% to 62.9% in 2007. There was also improvement in the lipid profile of patients at the end of 10 years compared with baseline. Mean total cholesterol was reduced from 6.1 to 4.9 mmol/L, while the mean HDL increased from 1.2 to 1.3 mmol/L.
Table 2 compares CVD risk factors at baseline and events rate at the end of 10 years in males and females. Generally, males had higher CV risk profiles than females. They were older and more of them were smokers; they had poorer control of BP and lipid profiles, but had a slighter better glucose profile. The CHD events were twice more common in males (13.5%) than in females (6.4%). Similarly, strokes were higher in males (6.1%) than in females (3.1%). In total, six CVD events were fatal in males and only one fatal in females.
Table 3 shows the risk categories of Framingham general CVD risk score and the CVD events that occurred. The median Framingham general CVD risk score for the study population was 21.1% (IQR 1.2–30.0%) with a 95% CI of 19.9% to 21.1%. The actual number of CVD events that occurred in the 10 years was 127 (13.1%), while the predicted number was 204 (21.1%). In males, their median risk score was 30.0% (95% CI 24.9% to 26.4%), while the events that occurred were 62 (19%). In females, their median risk score was 18.5% (95% CI 24.9% to 26.4%) while the events that occurred were 65 (10.2%).
The number of events occurring in the low-risk group, in men and women, were consistent with the number predicted. In the medium risk group, the risk score tool overpredicted risk in men and women, but more so for men. In the high-risk group of men, the prediction of CVD events is consistent with the actual events that occurred. However, in females with high risk, the risk score tool overpredicted the CVD risk.
Table 4 shows the risk categories of Framingham general CVD risk score and medications used in a 10-year interval (1998–2007). Overall, the use of medications for CVDs was highest in the high-risk group in men and women at baseline. Nearly three-quarters of men and women in the high-risk group were already receiving antihypertensive at baseline. Medication use increased across all risk categories in men and women over 10 years; the greatest increase in use was the use of antihypertensive in females with medium risk (increment of 34%), and use of antilipid agents by females in high-risk groups (increment of 70%).
The AUC for Framingham general CVD risk score was 0.63, showing moderate discrimination as seen in figure 2. The calibration for Framingham general CVD risk score was good as the Hosmer-Lemeshow test result was χ2=3.25, p=0.78.
Table 5 shows the comparison between Framingham general CVD risk score and CVD events according to ethnicity. The Framingham general CVD risk score discriminates moderately well for Malay race (AUC 0.65, p=0.01), Chinese (AUC 0.60, p=0.03) and Indians (AUC 0.65, p=0.001). Males (AUC 0.63, p=0.002) seem to have a better discrimination index than females (AUC 0.58, p=0.04).
Our study shows the Framingham general cardiovascular risk score tool to have moderate discrimination and good calibration in an Asian population. These findings are not surprising as our study cohort are actually patients from a clinic and not subjects in the community. Hence, not unexpectedly, the overall CVD risk profile of our cohort is already high before CVD events have occurred, with the majority of them having underlying diabetes and/or hypertension. Therefore, a clustering of patients with higher risk and fewer patients with lower risk explain our finding of only moderate discrimination. Perhaps this risk prediction model may give a better discrimination in a general population of subjects, with a wider range of CV risk. Similarly, the overall CVD risk as predicted by the Framingham general CVD risk score tool for our cohort of primary care patients, was high in males and medium in females. This is again not a surprising finding as males in our cohort generally had higher CVD risk profiles (poorer BP and LDL<2.6%). Conseqeuently, they also had more CVD events and higher mortality rates. Thus, the Framingham CVD risk tool helps to identify appropriately patients at high risk where more needs to be done to reduce their risk . A recent study validated the Framingham cardiovascular risk score in Tehran, a Middle Eastern population, and found it also works very well.24
Inspite of the Framingham risk score seemingly overpredicting CVD risk to some extent, we feel that it is appropriate for use in our cohort. This is because the prediction is accurate for both men and women with low risk and for men with high risk as the predicted and the observed risk score belong to the same category of risk, that is, when the predicted risk is high, the observed risk is also high. In fact, one could argue that the tool underestimates risk as the majority of high-risk patients were already receiving one or another medication for the CVD; hence, their risk is expected to be reduced.
Our study found in general that this new risk tool overestimated risk in women. This is consistent with findings in the other studies which used other risk tools.1 ,13–17 While this appears as an overestimation, this may not be the case. It could be because of the increase in use of antihypertensive agents which would have lowered their risk. For example, a great increase in the use of anti-hypertensive agents was seen over the 10 years of follow-up.25 Similarly, while use of antihypertensive increased across all risk categories, the greatest increase was seen in both men and women with medium risk. Furthermore, at the end of 10 years, the achieved mean BP was lower and the control rate of BP higher in the medium risk-group compared to the high-risk group. The greater use of medication and better control of BP could have led to a greater reduction in CV events and hence, this lead to what appears to be like an overestimation of risk.
Our present study has several strengths and few limitations. First, the strength of our study is that it is done in the primary care setting where the CVD risk profiles is different from that in secondary care. Another strength of this study is that it examines the validity and applicability of the Framingham general CVD risk tool in a multiethnic primary care population. Second, our sample size is large enough to give us a better picture of CVD risk of a multiethnic population in Asia, where data about the use and applicability of any CVD risk prediction tool is lacking. Our study found that it did have a good fit and good calibration to be used by all the three different ethnic groups. Although it seems to be apparently overestimating risk, it must be remembered that this could be due to the treatment effect. Use of such a tool is of particular importance in the light of limited resources and the rapidly increasing CVD burden in the region as it can help identify those most at risk; these need to be treated in order to reduce their risk.
A limitation of our study is that the use of CV risk prediction tools are meant to be used to estimate overall CV risk in order to decide on treatment. Most of our patients were already on treatment and that would result in an overestimation of risk in most situations. However, based on current clinical practice, it would be unethical to do a study where patients at risk are not treated in order to derive a risk tool without treatment being on board. As this is a retrospective study, there will be missing data to some extent. However, we were able to compare the characteristics of those with missing data (and hence excluded from the analysis) to those included in the analysis and found no major clinical differences between them, suggesting there was no substantial bias.
The Framingham general CVD risk prediction tool is valid for use in primary care population. In the absence of local risk prediction tools, the Framingham CVD risk tool can be used as a surrogate to stratify risk and hence determine the indication for pharmacotherapy. Furthermore this Framingham general CVD risk prediction tool which is meant for use in primary care provides a good fit for use in a multiethnic population in Asia.
The author would like to acknowledge Department of Primary Care at the University of Malaya for providing the support during the data collection.
Contributors YCC and SYWG contributed to the conceptualisation of the paper, data entry and writing of the manuscript. SMC, KC and HML contributed to the data analysis and writing of the manuscript. All the authors read and approved the final manuscript.
Funding University of Malaya funded the research (UMRG 116/09HTM).
Competing interests None declared.
Ethics approval Ethics Committee of University of Malaya.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The data is deposited under the website of https://mynotebook.labarchives.com/, an open access repository under name of manuscript and the DOI link is 10.6070/H48W3B96.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.