Article Text
Abstract
Objective To determine the diagnostic accuracy of three tests—radial pulse palpation, an electronic blood pressure monitor and a handheld single-lead ECG device—for opportunistic screening for unknown atrial fibrillation (AF).
Design We performed a diagnostic accuracy study in the intention-to-screen arm of a cluster randomised controlled trial aimed at opportunistic screening for AF in general practice. We performed radial pulse palpation, followed by electronic blood pressure measurement (WatchBP Home A) and handheld ECG (MyDiagnostick) in random order. If one or more index tests were positive, we performed a 12-lead ECG at shortest notice. Similarly, to limit verification bias, a random sample of patients with three negative index tests received this reference test. Additionally, we analysed the dataset using multiple imputation. We present pooled diagnostic parameters.
Setting 47 general practices participated between September 2015 and August 2018.
Participants In the electronic medical record system of the participating general practices (n=47), we randomly marked 200 patients of ≥65 years without AF. When they visited the practice for any reason, we invited them to participate. Exclusion criteria were terminal illness, inability to give informed consent or visit the practice or having a pacemaker or an implantable cardioverter-defibrillator.
Outcomes Diagnostic accuracy of individual tests and test combinations to detect unknown AF.
Results We included 4339 patients; 0.8% showed new AF. Sensitivity and specificity were 62.8% (range 43.1%–69.7%) and 91.8% (91.7%–91.8%) for radial pulse palpation, 70.0% (49.0%–80.6%) and 96.5% (96.3%–96.7%) for electronic blood pressure measurement and 90.1% (60.8%–100%) and 97.9% (97.8%–97.9%) for handheld ECG, respectively. Positive predictive values were 5.8% (5.3%–6.1%), 13.8% (12.2%–14.8%) and 25.2% (24.2%–25.8%), respectively. All negative predictive values were ≥99.7%.
Conclusion In detecting AF, electronic blood pressure measurement (WatchBP Home A), but especially handheld ECG (MyDiagnostick) showed better diagnostic accuracy than radial pulse palpation.
Trial registration number Netherlands Trial Register No. NL4776 (old NTR4914).
- cardiology
- pacing & electrophysiology
- preventive medicine
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The index tests—radial pulse palpation, electronic blood pressure measurement (WatchBP Home A) and handheld ECG (MyDiagnostick)—and reference test were performed in quick succession, with on average only 25 min between the first index test and the ECG, minimising the risk of rhythm changes between measurements.
We minimised verification bias by performing a 12-lead ECG in a random sample of patients with three negative index tests and by performing multiple imputation.
We excluded patients with known atrial fibrillation (AF, thus increasing the validity of our results for the diagnostic purpose of case finding.
Participants were slightly younger and had less comorbidity than non-participants, which may have reduced the yield of AF in our study and decreased positive predictive values.
We cannot provide the numbers for the individual exclusion reasons, as this was not reported consistently enough to provide a reliable overview.
Introduction
Patients with atrial fibrillation (AF) often show non-specific or no symptoms, making it difficult to track them down.1 When left untreated, AF greatly increases the risk of stroke, heart failure and death.2 As anticoagulation prevents over 60% of AF-related strokes, timely diagnosis of AF is of utmost importance.3 General practice seems to be a suitable setting for case finding (‘opportunistic screening’) of AF, as prevention is an important task of primary care and various diagnostic methods seem feasible here.
Timely diagnosis of AF might be established with opportunistic screening, but community screening for AF is still controversial.4 5 In six randomised controlled trials, the effect of screening was studied; three favoured screening, three did not.6–11 Twelve-lead ECG is unsuitable for screening purposes in primary care since it requires extra effort and organisation from patients and staff. Palpation of the radial pulse is a simple and inexpensive method with a high reported sensitivity, but low specificity.12 Devices equipped with an AF detection algorithm, such as various handheld single-lead ECG devices and electronic blood pressure monitors, have shown promising sensitivity and specificity.13 14 However, these methods have not yet been compared head-to-head in an indicated population without AF.
In the ‘Detecting and Diagnosing Atrial Fibrillation’ (D2AF) study, we performed opportunistic screening for AF with three detection methods: radial pulse palpation and measurements with two devices with an AF detection algorithm—an electronic blood pressure monitor and a handheld single-lead ECG device.10 Here, we present a diagnostic accuracy study nested in the intention-to-screen arm of the D2AF study. We determine and compare the diagnostic performance of three tests—radial pulse palpation, electronic blood pressure measurement and handheld ECG—for the diagnosis of AF in primary care.
Methods
Design
We performed a diagnostic accuracy study, nested in the intention-to-screen arm of a cluster randomised controlled trial on opportunistic screening for AF in primary care, the D2AF study.10 15 Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.
Population
The intention-to-screen arm of the D2AF study included 47 general practices in the Netherlands. General practitioners, practice nurses and assistants performed the study procedures. They received an on-site 1.5-hour training on performing the study.
Patient inclusion ran from September 2015 through August 2018, for 1 year per practice. Before the start of the study, we preselected 200 patients in each practice, aged 65 years or over without the International Classification of Primary Care (ICPC) code for AF (K78) and marked their electronic medical record.15 When these patients visited their practice for any reason during the study period, they were invited to participate. At that moment, exclusion criteria were applied: suffering from a terminal illness, being legally incompetent or unable to give informed consent or having a pacemaker or implantable cardioverter-defibrillator. If AF had already been diagnosed the patient was excluded.
Index tests
Three index tests were performed: radial pulse palpation, and measurements with two devices with an AF detection algorithm, that is, an electronic blood pressure monitor (WatchBP Home A, Microlife, Widnau, Switzerland) and a handheld ECG device (MyDiagnostick, MyDiagnostick Medical, Maastricht, The Netherlands) (see figure 1).
We gave instructions to perform pulse palpation by feeling the radial artery in the wrist for at least 15 s, assessing regularity (regular, one to three extra beats, completely irregular), equality (yes/no) and frequency (beats per minute (bpm)). To maximise sensitivity, any irregularity during pulse palpation—including one to three extra beats and complete irregularity—was considered a positive result.
The upper arm cuff of the electronic blood pressure monitor automatically inflates and deflates three times in the ‘usual’ mode. The screen displays the average heart rate (bpm) and systolic and diastolic blood pressure (mm Hg). It displays an ‘AFIB’ icon if the built-in algorithm detects AF in two or three measurements. We considered this a positive result.
The handheld ECG is a bar of 24 cm with metallic electrodes at both ends. When holding it with both hands, it switches on and after 1 min a light indicates whether the built-in algorithm detects AF (‘red’) or not (‘green’). When connected to a computer, the associated software stores the rhythm strip and the algorithm-generated automatic interpretation of AF (red indicator light) or no AF (green indicator light). A red indicator light was considered a positive result.
Reference test
We equipped all practices with a 12-lead ECG device (Multichannel Holter ECG recorder model H2, Fysiologic, Amsterdam, The Netherlands), the gold standard for AF detection. The ECG results were transferred digitally. We defined AF as a completely irregular RR-interval without definable p-peaks.16 An experienced assessor supervised by a cardiologist checked the 12-lead ECG for AF. A second cardiologist independently assessed all 12-lead ECGs for AF. All evaluators were blinded for the index test results. In case of disagreement, a third cardiologist decided, blinded for the previous assessments and unaware of being the referee.
Study procedures
Written informed consent was followed by an inquiry of recently experienced symptoms possibly related to AF: palpitations, vertigo, syncope, dyspnoea, chest tightness and exercise intolerance. These questions were followed by radial pulse palpation, electronic blood pressure measurement and handheld ECG. Ethnic origin was registered as well. To curtail the risk of confirmation bias, the sequence of the last two tests differed per practice; 25 practices were randomly allocated to perform the electronic blood pressure measurement first, followed by the handheld ECG, and 22 practices vice versa. Measurements were not to be repeated, in order to minimise expectancy bias.
All patients with at least one positive index test received a 12-lead ECG at shortest notice. For logistic and financial reasons, a 12-lead ECG was not feasible in patients with three negative index tests, due to the expected large number.17 To limit verification bias, a 12-lead ECG was also performed at shortest notice in a 10% random sample of patients; after entering three negative index tests into the electronic case report form, the computer directly performed the randomisation and displayed the result.
Finally, in the D2AF screening trial, all patients in whom the 12-lead ECG did not show AF, were offered a 2-week Holter registration (Multichannel Holter ECG recorder model H2).
Data collection
Data were collected through an electronic case report form (MEMIC, centre for data and information management, Maastricht University, The Netherlands). We downloaded automatic algorithm results of the handheld ECG from the local software, compared them with the manually entered indicator light colours, and corrected them in case of disagreement. After the study period, we extracted ICPC codes from the electronic medical record system to determine baseline patient characteristics. We manually reviewed all medical records of patients with new AF, to ensure it had not been diagnosed before participation in the study.
Data analysis
We used IBM SPSS Statistics for Windows (V.25.0, IBM, Armonk, New York, USA). For descriptive statistics, we report numbers and percentages (n, %) for categorical variables and means and SDs or medians with IQRs for numerical variables. To check for selection bias, we compared characteristics of participants and non-participants, and characteristics of patients with three negative index tests within versus outside of the sample receiving a 12-lead ECG. We used a χ2 or Fisher’s exact test where appropriate for categorical variables and an independent samples T-test for continuous variables. We considered a two-sided p value ≤0.05 statistically significant.
We report our diagnostic accuracy study according to STARD.18 To limit verification bias, we performed a 12-lead ECG in a 10% random sample of patients with three negative index tests.15 To calculate the diagnostic parameters we applied multiple imputation (see text box), which is considered the best method to minimise verification bias.19 Multiple imputation was based on fully conditional specification, in particular predictive mean matching, creating 100 datasets with 10 iterations per set.20 Variables used for imputation were gender, age, symptoms, medical history, AF according to the electronic medical record and results of the three index tests, 12-lead ECG and Holter. In all 100 datasets, we computed sensitivity, specificity, predictive values and likelihood ratios of each index test (or combination of tests). We reported pooled diagnostic parameters as a mean plus range of the 100 datasets. With McNemar’s test for paired nominal variables, we investigated whether sensitivity and specificity differed significantly between the index tests.
Results
Study procedures
Study procedures were performed by a research or practice assistant in 42% (1829/4339) of patients, a practice nurse in 34% (1495/4339), a physician in 12% (520/4339) and by an unspecified practice worker in 11% (495/4339).
The median time between registration of the first index test and the 12-lead ECG was 25 min (IQR 18–44). The indicator light of the handheld ECG was registered for 4331 patients; for 3607 (83.3%) of them, we obtained the automatic interpretation from the local software. We corrected 17 manually entered handheld ECG results.
Participants
Out of the 9400 patients whose medical file was marked, 4339 patients participated (figure 2), with a mean (±SD) of 92±23 per practice. On average, participants were younger and had less comorbidity than non-participants (online supplemental appendix 1). Table 1 shows the participant characteristics and a comparison of patients with one or more positive index tests versus patients with three negative index tests. Within the group of patients with three negative tests, a comparison of the random sample who received a 12-lead ECG (n=308) vs patients outside the sample (n=3505) revealed that patient characteristics were not significantly different, except for hypertension (p=0.013; see online supplemental appendix 2).
Supplemental material
Supplemental material
Observed cases and multiple imputation
Out of the 4339 screened patients, 793 (18.3%) received a 12-lead ECG; 485 of them had at least one positive index test and 308 were triple-negative (figure 2). The cumulative incidence of AF in the observed cases was 0.7% (30/4339). Figure 3 shows the observed cases with at least one positive index test result (n=526) and their overlap.
Table 2 shows the pooled results after multiple imputation; complete cases (ie, patients with both an index and a reference test result) can be found in online supplemental appendix 3 and index test combinations in online supplemental appendix 4. The mean (±SD) pulse frequency was 71±11 bpm with pulse palpation. In patients with AF this was 76±13 (not shown in table).
Supplemental material
Supplemental material
Diagnostic accuracy
Table 3 displays the diagnostic test characteristics based on the pooled data. Both sensitivity and specificity of electronic blood pressure measurement (70.0% and 96.5%) and handheld ECG (90.1% and 97.9%) were higher than those of radial pulse palpation (62.8% and 91.8%). The sensitivity and specificity of the handheld ECG were significantly higher than those of the other two index tests in all 100 imputed datasets (all p values were ≤0.039). The negative predictive values of all index tests were ≥99.7%. The positive predictive value of the handheld ECG was the highest (25.2% vs 13.8% and 5.8% for electronic blood pressure measurement and radial pulse palpation, respectively). The positive likelihood ratios of electronic blood pressure measurement (19.9) and handheld ECG (42.0) were high; the negative likelihood ratio of handheld ECG was 0.1. Additional analysis of five index test combinations did not reveal a superior combination (see online supplemental appendix 5).
Supplemental material
Discussion
Main findings
Our diagnostic accuracy study—performed in 4339 patients of 65 years and older, visiting the general practice for any reason, of whom 0.8% had new AF—showed that all three AF detection methods could exclude AF (negative predictive value ≥99.7%). However, electronic blood pressure measurement and handheld ECG had a higher diagnostic accuracy than radial pulse palpation in detecting unknown AF (sensitivity and specificity 70.0% and 96.5%, 90.1% and 97.9%, 62.8% and 91.8%, respectively). The handheld ECG showed the highest sensitivity and specificity; its positive predictive value was 25.2% in this population. Combining index tests had no clear advantage.
Strengths and limitations
Our study had several strengths. First, the index and reference tests were performed in quick succession, with on average only 25 min between the first index test and the ECG. This short interval minimised the risk of rhythm changes between measurements.
Second, we minimised verification bias in the calculated diagnostic parameters. Rather than labelling patients with three negative index tests as ‘no AF’, we performed a 12-lead ECG in a random sample of these patients. A comparison of patient characteristics within versus outside the sample showed that our sample was representative. In addition, we applied multiple imputation to compute all diagnostic accuracy parameters in a valid way.19 Inverse probability weighting would have overestimated sensitivity and—to a lesser extent—the negative predictive value for the scenarios with the handheld ECG, due to zero false-negative results.21
Third, we excluded patients with known AF, which increased the validity of our results for the diagnostic purpose of case finding. Clinical features of patients with known AF may differ from those with newly diagnosed and untreated AF, affecting test characteristics.22 Moreover, including patients with known AF would artificially have raised AF frequency in the study population, affecting predictive values.23
A limitation of our study is that participants were slightly younger and had less comorbidity than non-participants. This may have reduced the yield of AF in our study and decreased positive predictive values. A second limitation is that we cannot provide the numbers for the individual exclusion reasons, as this was not reported consistently.
Incidence of atrial fibrillation and positive predictive values
The cumulative incidence of AF in our study (0.8%) is lower than in diagnostic studies that did not exclude known AF. Consequently, positive predictive values for all three methods are lower in our study than in previous studies.24–26 Nonetheless, the positive predictive values in our study better reflect real-life screening situations, with a low cumulative incidence of AF.
Radial pulse palpation
Despite defining ‘any’ irregularity as a positive result, the sensitivity of radial pulse palpation was lower in our study (62.8%) than in a previous meta-analysis (92%; 95% CI 85% to 96%); specificity (91.8%) was higher (82%; 95% CI 76% to 88%).27 The heart rate of patients with new AF in our study (76 bpm) was only slightly higher than the mean heart rate in our study population (71–72 bpm) and much lower than the typical AF frequency of 100–160 bpm.28 This makes it more challenging to discern AF from sinus rhythm and may explain our low sensitivity. The low cumulative incidence of AF in our study could explain the relatively high specificity.29
Electronic blood pressure measurement
In a study of Chan et al and in the meta-analysis of Verberk et al the sensitivity of the WatchBP Home A is markedly higher (80.6% and 98%) than in our study (70.0%).26 30 However, they did not always apply the reference test in case of a negative index test, nor apply a statistical computation to limit verification bias. Furthermore, they did not exclude patients with known AF. Test characteristics can also be influenced by variation in setting—not all studies were conducted in primary care—or country. In the Screen AF study, elderly hypertensive patients used the WatchBP Home A twice daily at home to screen for AF.11 All diagnostic parameters were lower than ours, possibly the quality of the measurements was lower in unsupervised performance at home than in performance by a healthcare worker.
Handheld ECG
The sensitivity and specificity of the handheld ECG in our study are comparable to those in previous studies.14 Predictive values in two other studies (56.3%, 45%) were higher than in ours (25.2%), probably because patients with known AF were not excluded.24 25 In our head-to-head comparison, we showed that diagnostic characteristics of electronic blood pressure measurement and handheld ECG exceed those of pulse palpation. This is in accordance with the results of the systematic review of Taggar et al.27
Implications for practice
This study showed that all three index tests could exclude AF in a case finding setting in primary care.31 Both devices outperformed radial pulse palpation. The diagnostic parameters of the handheld ECG device—in particular its sensitivity and positive predictive value—were the most favourable.
The use of ambulatory devices or technologies in healthcare—Mobile Health—rapidly increases, resulting in the development of many new devices.32 Results for WatchBP Home A and MyDiagnostick cannot simply be extended to other blood pressure monitors and handheld single-lead ECG devices with AF detection function. Other devices recording pulse irregularities or single-lead ECGs should be investigated in further research, preferably again in ‘indicated’ populations without known AF. Such studies should address the establishment or rejection of a new diagnosis of AF, either induced by physicians (case finding in high-risk patients) or by patients presenting with signs or symptoms suggestive of AF.
Conclusion
This study showed that radial pulse palpation, and measurements with two devices with AF detection algorithm—electronic blood pressure monitor (WatchBP Home A) and handheld ECG (MyDiagnostick)—are suitable for excluding AF in a case finding situation. Diagnostic accuracy of the electronic blood pressure monitor and especially the handheld ECG exceeded that of radial pulse palpation.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
The medical research ethics committee of the Amsterdam University Medical Center (Amsterdam UMC), Amsterdam, approved the D2AF study protocol (14 November 2014, No. NL48215.018.14). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We thank Fysiologic ECG Services (Esther Kuiper, Dave Hopman, Robert den Engelsman), Microlife/Retomed (Remon Winkel, Edwin Koekoek, Willem Verberk), MyDiagnostick Medical (Vincent Larik, Roger Hünen) and MEMIC (Daniel Huysmans and Dirk Veldman) for their collaboration. Harry Crijns, Trang Dinh, Joris de Groot, Justin Luermans and Maayke Sluman generously helped us in assessing the ECGs and Holters. Piet Portegijs was our independent expert. We are obliged to Alice Karsten, Ursula de Jonge Baas and Frederique van Nouhuys of the Academic GP network Academic Medical Centre. Ralf Harskamp, Jelle Himmelreich and Evert Karregat assisted the D2AF group. We also thank research assistant Mascha Twellaar for her help and guidance in data processing, and research assistants Sylvia de Graaf and Marion de Mooij, who were an essential part of the D2AF team. We are grateful for the work students performed for us (Karlijn Costongs, Sanne van de Moosdijk, Sander de Haas, Dominique van Mil, Arlette de Loo, Ulla van Sprang, Yvonne Griesen, Patrick Witvliet). The study would not have been possible without the help of all participating general practices. We also like to thank the participating patients for their effort.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
NV-vG and SBU contributed equally.
Contributors SBU and NV-vG contributed equally to this work. SBU, NV-vG, WAML, PMGE, AK, HCPMvW and HS conceived and designed the study. AK and HCPMvW supervised the study. WAML, PMGE, AK, HCPMvW and HS obtained funding. SBU, NV-vG, WAML, BW and HS acquired, analysed and interpreted the data. NV-vG wrote the first draft of the manuscript, and all authors revised the manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. HCPMvW is the guarantor.
Funding This work was supported by ZonMw, The Netherlands Organization for Health Research and Development (grant number 839110006) and funded internally by Amsterdam Universities Medical Centers. Microlife/Retomed provided equipment without charge, and MyDiagnostick Medical BV granted a discount. Boehringer Ingelheim loaned us additional MyDiagnostick devices during the study.
Disclaimer The funders did not have a role in the study design, in the collection, analysis and interpretation of data, in the writing of the report or in the decision to submit the article for publication.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.