Article Text
PDF
Abstract
Objective Clinicians often overestimate the probability of a disease given a positive test result (positive predictive value; PPV) and the probability of no disease given a negative test result (negative predictive value; NPV). The purpose of this study was to investigate whether experiencing simulated patient cases (ie, an ‘experience format’) would promote more accurate PPV and NPV estimates compared with a numerical format.
Design Participants were presented with information about three diagnostic tests for the same fictitious disease and were asked to estimate the PPV and NPV of each test. Tests varied with respect to sensitivity and specificity. Information about each test was presented once in the numerical format and once in the experience format. The study used a 2 (format: numerical vs experience) × 3 (diagnostic test: gold standard vs low sensitivity vs low specificity) within-subjects design.
Setting The study was completed online, via Qualtrics (Provo, Utah, USA).
Participants 50 physicians (12 clinicians and 38 residents) from the Department of Family and Community Medicine at St Michael’s Hospital in Toronto, Canada, completed the study. All participants had completed at least 1 year of residency.
Results Estimation accuracy was quantified by the mean absolute error (MAE; absolute difference between estimate and true predictive value). PPV estimation errors were larger in the numerical format (MAE=32.6%, 95% CI 26.8% to 38.4%) compared with the experience format (MAE=15.9%, 95% CI 11.8% to 20.0%, d=0.697, P<0.001). Likewise, NPV estimation errors were larger in the numerical format (MAE=24.4%, 95% CI 14.5% to 34.3%) than in the experience format (MAE=11.0%, 95% CI 6.5% to 15.5%, d=0.303, P=0.015).
Conclusions Exposure to simulated patient cases promotes accurate estimation of predictive values in clinicians. This finding carries implications for diagnostic training and practice.
- diagnostic inference
- experience-based learning
- ppv
- npv
- estimate accuracy
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Footnotes
Contributors BA (study guarantor): study design, study programming, participant recruitment, data collection, data analysis and manuscript writing. JS: study design and manuscript writing. NP: study design, study funder, participant recruitment and manuscript writing. All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. NP funded the study and is supported by the Department of Family and Community Medicine of St Michael’s Hospital, the Department of Family and Community Medicine of the University of Toronto, an Early Researcher Award from the Ministry of Research and Innovation and the Physicians Services Incorporated Graham Farquharson Knowledge Translation Fellowship.
Disclaimer The funders had no role in the study.
Competing interests None declared.
Patient consent Not required.
Ethics approval Ethics approval to conduct the current study was obtained from both St. Michael’s Hospital Research Ethics Board (REB number: 16-282) and the Ryerson Ethics Board (REB number: 2014-129).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data available.