Data sources and searches

We searched PubMed, Embase, CINAHL and Google Scholar from database inception through to the search date 17 October 2018. We searched for studies written in English, Dutch or German. We used keywords: chest pain, coronary artery disease, acute coronary syndrome, general practice, primary care practice, prediction rule, decision model or decision aid. Online supplement A of the supplemental data document displays the full search strategy.

Study selection

Two investigators (REH, SCL) identified potentially eligible studies, with a third (WAML) to resolve any disagreements. We used an online systematic review platform (Covidence, Veritas Health Innovation, Melbourne, Australia) for this purpose. In addition to the language (English, Dutch, German) and human research restrictions, the following inclusion criteria for eligible studies were applied: (1) original studies in adults (≥18 years of age) with enrolment in a primary care setting; (2) chest pain either acute or intermittent-type; (3) ascertainment of the diagnosis of coronary artery disease or acute coronary syndrome at follow-up; (4) predictive tool based on multivariable analysis; (5) predictive tool derived from findings that are applicable in primary care setting. These findings may include: medical history, physical examination, ECG or previously documented laboratory findings (such as lipid levels). We excluded studies with a retrospective study design and studies that used a prediction rule that was based on serial biomarker testing (ie, sequential troponin testing at 2–3-hour time interval), required advanced computer algorithms or advanced diagnostic testing (cardiac imaging, coronary angiography).

Clinical decision rules and outcomes of interest

The clinical decision aids may include items from history taking, physical examination, laboratory and electrocardiographic data. The outcomes of interest are diagnostic test characteristics of included rules, including: sensitivity, specificity, negative and positive prediction values.

Reference diagnosis

The clinical outcomes that we used as reference diagnosis were (1) any form of coronary artery disease (CAD) or coronary heart disease; or (2) a more restricted form including unstable angina or myocardial infarction (referred to as acute coronary syndrome) in patients with acute chest pain. We applied no restrictions on minimum or maximum time of follow-up. The assessment of applicability of the reference standard for each study is assessed by the QUADAS-2 tool, which can be found as supplemental data in online supplement B.

Study population

We included studies with adult populations that present at a GP office or out-of-office setting (ie, patient visits when making house calls). In-hospital, emergency department (ED) and/or preselected outpatient populations were not eligible.

Data extraction and quality assessment

Two investigators (REH, SCL) extracted data elements from each study, with a third investigator (WAML) independently reviewing these data for accuracy. The quality of the studies was assessed by three investigators (REH, SCL, WAML) using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool for assessing risk of bias in diagnostic accuracy studies. This tool comprises four key domains, namely: patient selection, index test, reference standard and flow and timing.9 We assessed whether a clinical decision rule (CDR) was ready for application in clinical practice based on the level of evidence for each rule using the definitions of the Mount Sinai Evidence-Based Medicine Working group.10

Data synthesis and analysis

The extracted data on study and patient characteristics, outcome measures and follow-up information of the included studies will be displayed in tables. Subsequently, we extracted data on the discriminatory properties (C-statistic) of the decision rule from each studies, as well as data on sensitivity, specificity, positive (PPV) and negative predictive values (NPV) and true and false positives and negatives. We constructed a summary receiver operating characteristic curve based on 2×2 tables from the individual study data using Review Manager (RevMan V.5.3. The Cochrane Collaboration, Copenhagen, Denmark).

Patient and public involvement

This study did not involve direct patient involvement. For the current analysis, we did not a priori consult with representatives of patient organisations. After peer review and acceptance of publication, we will share the findings of our research with the Dutch Heart Foundation, relevant patient organisations, as well as general practitioners within our academic network.

Search results

Our search resulted in 3105 unique studies of which we assessed 94 in full text. Of those, eight studies met the inclusion criteria of our study, in which five different CDRs were evaluated. All studies were written in English. The flowchart of our search strategy and reasons for exclusions can be found in figure 1.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Flow chart of systematic search of the literature. ACS, acute coronary syndrome; CAD, coronary artery disease; ED, emergency department.

Quality assessment

The overall quality of the studies was moderate as graphically displayed in figure 2. In six out of eight studies, we found a high risk of bias in the reference standard, as the assessors who determined the final diagnosis (delayed type) were not blinded to the index test results. In three studies, we found a high risk of bias in patient selection as a significant proportion of patients were excluded prior to enrolment.11–13 Also, in four studies, a high risk of bias was found in flow and timing, due to relatively high dropout rates of patients.7 12–14 In one study, >15% of participating GPs stopped recruiting prematurely.7 Quality concerns of the pooled individual data study (INTERCHEST) included possible bias due to missing data in >20% of the study population and unverifiable risks of bias regarding patient selection.14 Details of the quality assessment can be found as online supplement B in the supplemental data file.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Quality assessment by QUADAS-2.

Study and patient characteristics

As shown in table 1, a total of seven single cohort studies were included involving 6959 patients and one pooled individual data study from five cohorts (INTERCHEST) involving 3099 patients. The sample size of the individual cohort studies ranged from 289 to 1249 patients. Studies were conducted in Europe and the USA and were published 1995 and 2017. All studies were conducted in general practice, with two studies mandating immediate work-up of all patients at the ED.12 13 The prevalence of CAD, with a variable diagnostic follow-up period of up to 1 year, ranged from 8.0% to 15.0%. In three studies concentrating on acute-onset chest pain, the prevalence of ACS ranged from 22.0% to 47.8%.11–13 The reported mean age of patients ranged from 41 to 67 years, with women comprising 44% to 58% of the population. In studies that reported the prevalence of comorbidities, hypertension (45%–50%) and dyslipidaemia (31%–41%) were common, and diabetes was present in approximately 13%. The inclusion and exclusion criteria as well as the definitions that were used for the reference diagnoses for each of the studies can be found as online supplement C and D in the supplemental file.

Clinical decision rules

We identified a total of five CDRs, namely the Gencer rule,7 the Marburg Heart Score,15–17 INTERCHEST,14 Grijseels rule12 13 and Bruins Slot rule.11 As shown in table 2, the CDRs have been developed based on readily available clinical information, such as patient characteristics, medical history and physical examination. The Grijseels rule also requires an ECG. The former three scores (Gencer, Marburg Heart Score and INTERCHEST) were developed for rule out of CAD, whereas the Grijseels and Bruins Slot rules were constructed for rule out of ACS.

Decision rules for stable coronary artery disease in patients with intermittent chest pain

As shown in table 3, the decision aid that was most extensively tested is the Marburg Heart Score. This study has good overall discrimination (C-statistic of 0.84–0.90), with a sensitivity of 86%–89%, specificity of 64%–81%, with a PPV of 23%–40% and a NPV of 97%–98%. The diagnostic properties of the Marburg Heart Score are visualised in figure 3, illustrating its consistent diagnostic performance in terms of sensitivity and specificity. As shown in table 4, The Marburg Heart Score was found to outperform unaided clinical judgement. When used as a decision aid, both the sensitivity (+8.0%) and specificity were higher (+5.8%). Moreover, when the Marburg Heart Score was used for an initial triage tool, it led to higher specificity (+11.6%) with similar sensitivity (−1.5%) compared with unaided clinical judgement. Based on the combined body of evidence, the level of evidence is 2 for the Marburg Heart Score, which implicates that this rule can be used in a general practice setting of low-risk patients with intermittent chest pain with confidence in its accuracy.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Summary receiver operating characteristic curve of specificity and sensitivity of the Marburg Heart Score across the individual studies.

The other two CDRs for rule out of stable CAD were the INTERCHEST rule and the Gencer rule. The INTERCHEST rule which was derived from a pooled data analysis also shows promise (C-statistic of 0.84, sensitivity 82%–88%, specificity 74%–82%, PPV of 35%–43% and NPV of 96%–98%), but has a number of quality concerns, and has not been compared with unaided clinical judgement. As such, the INTERCHEST rule should not be considered ready for clinical application (level of evidence is 4). The Gencer rule was developed and externally validated in only one study (C-statistic: 0.75–0.95, sensitivity 87%–98%, specificity 42%–71%). Given the limited evidence, the Gencer rule can only be used with caution (level of evidence for its use is 3).

Decision rules designed for acute coronary syndrome

Grijseels et al developed a decision rule for ruling out ACS in general practice in the late 1990s that was later updated by Bruins Slot et al. These studies show that the discrimination of these decision rules was mediocre (C-statistic of 0.66 and 0.72). Unaided clinical judgement provided a better overall fit (C-statistic of 0.75) with a 51% agreement in risk estimation. Other diagnostic properties are listed in table 4. Although the study by Bruins Slot is limited by sample size, it appears that the CDR was safer than clinical judgement alone, as four patients that were considered low risk by the GP (8.2%) were correctly identified as high risk by the decision aid. The INTERCHEST score was also assessed among 169 patients with acute chest pain; the authors found a reasonable overall performance (C-statistic of 0.79). However, data on its test characteristics were lacking, and as such, we are unable to assess its safety and accuracy. Overall, neither the Grijseels, Bruins Slot or INTERCHEST rules ought to be recommended for rule out of ACS in a general practice setting.

Strengths and limitations

We performed a rigorous systematic search and quality assessment of the included articles involving chest pain rules in primary care. We avoided bias in the selection of studies by two reviewers individually identifying eligible studies, with a third to resolve any disagreements. While not being the first systematic review on this topic, this review is to our knowledge the first that examines the results of the CDRs while taking into account the results of the derivation, validation, and compared the performance of the CDR with the unaided clinical judgement of the GP.

Our study also has a number of limitations. First, we accepted a final diagnosis of coronary artery disease based on a delayed-type reference diagnosis based on consensus of a panel of experts using available symptom-related data and work-up. Such a strategy is valid, as mandating the use of coronary angiography as the reference standard would not be feasible in primary care.28 A second limitation is the substantial heterogeneity in the prevalence of ACS among studies of CDRs for acute chest pain (range of 22%–47.8%), which could indicate that GPs may have preselected patients.

Furthermore, we should acknowledge that while we searched for clinical prediction rules for chest pain to rule out CAD or specifically ACS, a minority of patients may present with non-chest pain symptoms (ie, dyspnoea, jaw pain) but do have myocardial ischaemia, these patients (which are more frequently elderly, women and diabetics) may not be properly represented in the included studies.29–31 A third limitation is that not all included studies reported sufficient data to allow construction of two-by-two contingency tables. Therefore, we cannot accurately assess the performance data of these CDRs. Finally, the CDRs were derived over a span of 22 years. Since the criteria for CAD, the prevalence of risk factors and prevalence of CAD may have changed over the years, some CDRs might be outdated.

Chest pain rules outside primary care

Our aim was to research the availability of chest pain rules that are applicable and have been validated in low-resource primary care settings. We, therefore, purposefully restricted the scope of this systematic review and excluded CDRs that rely on advanced laboratory, computer or diagnostic testing for their respective scoring systems. We therefore did not include studies on CDRs that are commonly used in EDs, such as the History, ECG, Age, Risk factors and initial Troponin (HEART),32 Global Registry of Acute Coronary Events33 and Thrombolysis in Myocardial Infarction34 scores as well as the more recent Manchester Acute Coronary Syndromes rule.35 For a comprehensive overview of chest pain rules recently validated in ED patients, we refer to the systematic review by Liu et al.36