You are here

Article Text

Article menu
Narrative review
The incidence of diagnostic error in medicine
  1. Mark L Graber
  1. Correspondence to Dr Mark L Graber, RTI International, SUNY Stony Brook School of Medicine, 1 Breezy Hollow, St James, NY 11780, USA; mgraber{at}rti.org

Abstract

A wide variety of research studies suggest that breakdowns in the diagnostic process result in a staggering toll of harm and patient deaths. These include autopsy studies, case reviews, surveys of patient and physicians, voluntary reporting systems, using standardised patients, second reviews, diagnostic testing audits and closed claims reviews. Although these different approaches provide important information and unique insights regarding diagnostic errors, each has limitations and none is well suited to establishing the incidence of diagnostic error in actual practice, or the aggregate rate of error and harm. We argue that being able to measure the incidence of diagnostic error is essential to enable research studies on diagnostic error, and to initiate quality improvement projects aimed at reducing the risk of error and harm. Three approaches appear most promising in this regard: (1) using ‘trigger tools’ to identify from electronic health records cases at high risk for diagnostic error; (2) using standardised patients (secret shoppers) to study the rate of error in practice; (3) encouraging both patients and physicians to voluntarily report errors they encounter, and facilitating this process.

  • Decision making
  • Medical error, measurement/epidemiology
  • Diagnostic errors
  • Patient safety

this is an open access article distributed in accordance with the creative commons attribution non commercial (cc by-nc 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. see: http://creativecommons.org/licenses/by-nc/3.0/

https://doi.org/10.1136/bmjqs-2012-001615

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    In God we trust, all others bring data1

    The patient safety movement in the USA has entered its second decade. A wide range of important safety concerns have been studied, and to this point, including medication errors, hospital-acquired infections, wrong-site surgery and a host of other issues. Strangely lacking, however, is a concerted effort to find, understand and address diagnostic errors.2–4 One factor that may contribute to its relative neglect is that the true incidence of diagnostic error is not widely appreciated. Measuring the rate of error and, in particular, error-related harm,5 would provide the necessary motivation to begin addressing this large and silent problem. How likely is a diagnosis to be wrong, missed, or egregiously delayed? How often do diagnostic errors cause harm? In this report, we briefly summarise the methods that have been used to estimate the rate of diagnostic error, and comment on their relative merits and limitations. A more comprehensive presentation of studies using each of these methodologies has been presented elsewhere.6

    The incidence of diagnostic error

    Arthur Elstein, a cognitive psychologist interested in ‘how doctors think’, studied clinical decision making for his entire career and concluded the diagnosis is wrong 10–15% of the time.7 A diverse range of research approaches that have focused on this issue over the past several decades suggest that this estimate is very much on target.6

    The incidence of diagnostic error has been estimated using eight different research approaches (table 1).

    View this table:
    Table 1

    Research approaches used to estimate the incidence of diagnostic error

    Autopsy studies identify major diagnostic discrepancies in 10–20% of cases. Most cases in autopsy series derive from inpatient settings, but they also include deaths from the emergency department which, for many reasons, is considered to be the natural laboratory for studying diagnostic error. Although autopsies have virtually disappeared in the USA, autopsies are still common in many other countries, and despite the availability of modern imaging, continue to show diagnoses being missed that might have been lifesaving, particularly infections and cardiovascular conditions.

    Although autopsy data is considered the ‘gold standard’ in terms of providing the most definitive data on the accuracy of diagnosis, only a subset of cases ever reach autopsy, and in many cases, the relationship between clinical diagnoses and autopsy findings remains unclear. Autopsies also discover a large number of incidental findings that were not suspected during life, but that were clinically irrelevant.

    Surveys have found that diagnostic errors are a major concern of both patients and physicians. A survey of over 2000 patients found that 55% listed a diagnostic error as their chief concern when seeing a physician in an outpatient setting.21 Similarly, physician surveys have consistently found that approximately half the respondents encounter diagnostic errors at least monthly.11 ,22 ,23 Moreover, compared with the many different safety concerns encountered in practice, physicians perceive diagnostic errors to be more likely to cause serious harm or death compared with other safety concerns.24

    Standardised patients studies using ‘secret shoppers’ have also been used to estimate the accuracy of diagnosis. In these studies, real or simulated patients with classical presentations of common diseases, like rheumatoid arthritis, asthma, or chronic obstructive pulmonary disease (COPD) are sent anonymously into real practice settings.

    The diagnostic error rates reported (13–15%) are very much in line with estimates from the other types of research approaches, and have substantial ‘face’ validity in that the studies are being carried out prospectively in real-world settings. In addition to providing an estimate of diagnostic error rates, this approach offers the unique ability to probe the various factors that promote or detract from optimal diagnosis.25 ,26

    The chief limitation of these studies in regard to estimating error rates is that they are performed under research-directed conditions, presenting a much smaller subset of conditions than would be seen in usual practice. Moreover, in studies seeking to also study the factors relevant to accurate diagnosis, these patients may present with comorbid conditions or contextual complexities that are not representative of typical patients, and case complexity is a major factor in determining diagnostic accuracy.27

    Second reviews refers to research protocols in the visual subspecialties (eg, radiology, pathology, dermatology) where a second radiologist examines the same films after a first radiologist, or a second pathologist reviews the same biopsy or cytology specimen as another pathologist. These second review studies may be performed under controlled conditions, involving the review of many or mostly abnormal cases. This approach has advantages from a research perspective, but substantially increases the possibility for diagnostic error, which can range from 10% to 50%.28–31 Interestingly, the studies also show that a diagnostician will also disagree with his or her own prior interpretation in a small fraction of cases.

    In ‘real world’ situations, the majority of examinations are normal. Under these conditions, a critical abnormality is detected by a second expert reviewer in the range of 2–5%.

    Diagnostic testing audits are used to estimate the incidence of error in the clinical laboratory. Thanks to impressive advances in quality control procedures, diagnostic errors in the modern age are rarely the result of an error in the analytical test itself. Most laboratory-related errors now originate from the preanalytical and postanalytical phases, namely issues related to the physician ordering and interpreting the test result. Overall, it is estimated that laboratory results are misleadingly wrong in 2–4% of cases, and roughly the same error rates are found in diagnostic radiology. The major limitation of diagnostic testing audits is that, although they excel at finding errors in test performance per se, they underestimate the true clinical impact because of the related preanalytical and postanalytical errors occurring outside the walls of the laboratory or radiology department which, typically, escape detection. Lapses in the reliable communication of abnormal test results, for example, is a universal problem, even in systems with advanced electronic medical records.32 ,33

    Malpractice claim databases are easily studied and have provided interesting data on diagnostic errors. Typical data (eg, from the Physicians Insurers Institute of America, shown below) reveals that problems related to diagnostic error are the leading cause of paid claims. Essentially identical data has been reported from the Department of Veterans Affairs, Kaiser-Permanente Healthcare System, and CRICO-RMF (CRICO Risk Management Foundation), the self-insurance entity covering the Harvard teaching hospitals, and from studies in the UK.34 A recent analysis of malpractice cases extracted from the National Practitioner Data Bank over a 25-year period identified 100 249 cases of diagnostic error.35 Diagnostic error was the most common reason for a claim (29%) and the most costly, averaging $386 849 per claim.

    Studying closed claims provides a convenient approach to analysing large numbers of cases, already preclassified as representing diagnostic errors. Limitations of this approach include the fact that only a small subset of errors results in claims, and these, typically, span just a narrow range of clinical conditions, predominated by cases involving cancer or cardiovascular disease in young or middle-aged patients. Moreover, these cases provide only limited opportunities to study the aetiology of diagnostic errors, as the proceedings by their nature are focused more on consequences than causes.

    Case Reviews. Many specific symptoms and disease states have been studied using retrospective case reviews, and in each of these the incidence of diagnostic error is unacceptably high. For example, a systematic review of more than 8000 ER patients found a delayed diagnosis of stroke in 9%.36 Kostopoulou and colleagues reported a systematic review of this literature and identified 21 studies meeting their quality criteria.37 In one such study, children with asthma experienced a median delay in making the correct diagnosis of more than 3 years, spanning 7 clinic/ER visits. Delayed or wrong diagnosis rates of 10–50% have been identified in studies of coronary artery disease, HIV-associated complications, tuberculosis and a wide range of malignancies. Very similar data has been reported by Gandhi and colleagues, who studied 181 cases of diagnostic error in ambulatory settings (figure 1).16 Delayed diagnosis of cancer and coronary artery disease were the most commonly identified problems in the study by Gandhi et al, and a growing number of studies have confirmed the frequency of missed opportunities for earlier cancer diagnosis.31 ,38 ,39 The point to be made is that whereas diagnostic errors may sometimes reflect encounters with extremely rare diseases or very unusual presentations of common diseases, in many cases, it is a relatively common disease that is mislabelled or missed entirely. A convenience sampling of studies that measured diagnostic error rates in 40 different symptom complexes or diseases is presented in online supplementary appendix table 1.

    Figure 1
    Figure 1

    Top alleged medical error named in claims where the patient expired (Physician Insurers Association of America (PIAA) Data Sharing Project Data 1985–2009, Physician Insurer, Vol 55, 2010).

    The chief limitation of case reviews is that they typically rely just on data contained in the medical record, and many diagnostic errors are missed as a result. As an example, Schwartz and colleagues used standardised patients to estimate costs resulting from diagnostic errors in ambulatory practices. Only their knowledge of the actual diagnosis allowed accurate estimates of these costs; virtually none could have been surmised from record reviews alone.26 An additional limitation is that the medical record, typically, is lacking documentation on what the clinician was thinking at the time the diagnosis was being considered. Finally, random chart reviews are not well suited to measuring incidence rates of diagnostic error because the rate of error is low, thus requiring a large number of reviews.

    Voluntary reports. Voluntary error-reporting systems are now in place in most healthcare organisations in the USA, and were expected to provide a reliable way of identifying both minor and major adverse events. When they are used, and especially if follow-up interviews with providers can be obtained, voluntary reporting offers the unique potential to explore both the system-related and cognitive aetiologies of diagnostic error.40 ,41

    Unfortunately, these programmes capture only a small fraction of diagnostic errors. Factors that detract from reporting include the time required to submit cases, a natural reluctance to call attention to one's own mistakes, and the ever-present fear of provoking a malpractice suit despite the reassurance that reports can be submitted anonymously. Another barrier to voluntary reporting of diagnostic errors is the lack of a ‘common format’ for reporting. The Agency for Healthcare Research and Quality has taken the lead in developing ‘common formats’ which healthcare organisations can use to report other types of patient safety incidents, but at the present time there is no specific common format tool to report diagnostic errors.

    Methods currently used to identify diagnostic errors in practice

    Diagnostic error rates are being measured in very few, if any, healthcare organisation in the USA. With regard to ambulatory settings, Tsang and colleagues recently reviewed the methods available for measuring adverse events and found that none were helpful in identifying diagnostic errors.42 The situation is no better for capturing diagnostic errors involving inpatients: A recent Inspector General study of 785 hospitalised Medicare beneficiaries using five different approaches identified a 13% incidence of serious adverse events, but not a single episode categorised as diagnostic error.43

    Suggestions on improving measurement

    Given the limitations of the error-detection approaches currently in use, how can healthcare organisations begin to identify diagnostic errors? We suggest three novel approaches:

    Consider using trigger tools

    Various types of trigger tools are now being used in quality programmes,44 the most widely known of which is the ‘Global Trigger Tool’ developed by the Institute for Healthcare Improvement.45 A growing fraction of US healthcare organisations are using this instrument, but it was designed specifically to identify treatment errors, specifically errors of commission, and is poorly suited to detecting diagnostic errors, many of which are errors of omission.46 A more promising approach was recently reported by Singh and colleagues, who designed an electronic trigger to identify patients with an unscheduled hospitalisation within 2 weeks of a primary care visit. The frequency of diagnostic errors in patients meeting this criterion was about 20%, compared with just 2% for unselected patients.47–49 A similar process identifying hospitalisations after an ER ‘treat and release’ visit has recently been described by Newman-Toker et al.50 Electronic surveillance is also effective in identifying diagnostic errors through discrepancies between laboratory and pharmacy records,51 and discovering diagnostic errors through data mining is probably just around the corner. Although the use of trigger tools will not capture all diagnostic errors, their use will substantially enrich the yield compared with random chart reviews and, thus, bring more errors to attention.52

    Encourage and facilitate voluntary or prompted reports from patients

    There is substantial preliminary evidence that patients are acutely aware of the diagnostic error problem.10 ,53 Several authors have called for patients to take a more active role in ensuring the reliability of the diagnostic process54–56 and enlisting their help to identify breakdowns is a logical, practical and simple approach to explore. Weingart and colleagues, for example, surveyed 228 inpatients who reported 20 adverse events, of which 11 were verified in the medical record but none were captured in the hospitals SE-detection programmes.57 Reports from Canada,58 Japan59 and Sweden60 have similarly found that patients are both willing and capable of participating effectively in identifying errors in their care.

    The chief limitation of soliciting error reports from patients is that they will require verification by healthcare providers. As evident from the Weingart study and others,61 not all their safety concerns accurately reflect true lapses in care.

    Encourage and facilitate error reporting from physicians

    At least two different approaches have successfully improved on the power of voluntary error reporting systems to capture diagnostic errors. Phillips and colleagues encouraged physicians, their staff and patients to report all safety concerns over just a constrained 10-week period, and to focus on specific ‘intensive reporting’ days. The approach yielded 935 reports from 10 practices, including 12 instances of suspected diagnostic error.62 Trowbridge was also successful in obtaining diagnostic error reports by acting as a clinical champion to encourage participation from colleagues.63 In pilot work with this process at Maine Medical Center, 36 diagnostic errors were reported over 6 months that otherwise would have escaped detection. The severity of errors uncovered was high; 73% involving moderate or serious harm to the patient. By comparison, the facility identified only six diagnostic errors from their standard, existing systems for identifying medical errors during the same time period.

    Discussion

    We have proposed that measuring the incidence of diagnostic error in everyday practice is an essential requirement of a comprehensive quality management programme.2 Of eight methods used to study diagnostic errors, some are more suitable than others in terms of their potential for providing meaningful data on diagnostic error rates (table 1). Analysing closed malpractice claims can provide only relative data on error rates; estimating absolute rates is not possible because so few true errors actually result in claims, and not all claims reflect true errors. Diagnostic testing audits are meaningful only to the extent that audits examine true clinical impact, because many diagnostic testing errors are ultimately detected and corrected, or discarded as meaningless. Of the remaining methods, each has its own advantages and limitations. Moreover, it is probably safe to assume that each of these approaches will underestimate the actual rate of error, and may identify entirely separate cohorts of errors. This problem is compounded by the different definitions and classification systems of diagnostic error that have been applied in these various studies. The goal of knowing the diagnostic error rate in practice may, therefore, require further research to standardise definitions and other methodologic issues, and combining results from several different approaches.

    Similarly, the eight different approaches offer differing capabilities in their ability to provide insight regarding the aetiology of diagnostic error. Autopsies and second reviews reveal that an error was made, but not why. Using standardised patients is a particularly powerful way to study these factors, because at least some of the variables (case presentation and complexity, for example) can be controlled. Voluntary reports from physicians also provide unique opportunities to gather insights on the cognitive and system-related factors that might have contributed to the error. With the exception of using standardised patients, none of the approaches are well suited to study the human factor issues, such as distractions, fatigue and workload stress, thought to play dominant roles influencing clinical decision making.

    Knowing the incidence of diagnostic error may be less important than being able to measure the likelihood of harm that results.5 Extrapolating from the Class 1 errors (a major discrepancy that likely leads to the patient's death) identified at autopsy, Leape, Berwick and Bates estimated that 80 000 deaths per year might be caused by diagnostic error, including both ambulatory and inpatient errors.64 A recent systematic review of autopsy data concluded that 36 000 deaths a year were due to diagnostic errors in just ICUs alone.65 These estimates, of course, do not include the many instances of non-fatal injury related to misdiagnosis, events that will be far more numerous, and the many instances where the harm is psychological or financial more than physical. Preventability, however, is a difficult parameter to judge, and these estimates may exaggerate the impact of diagnostic error to the extent that it is overestimated.66 Both case review studies and closed claims studies67 ,35 find that diagnostic errors are more likely to cause harm than other patient safety problems.

    In summary, a wide range of different research approaches have been used to estimate diagnostic error rates, all suggesting that the incidence is unacceptably high. Although true incidence data are lacking, a wide variety of research studies suggest breakdowns in the diagnostic process result in a staggering toll of harm and patient deaths. A recent, authoritative review by the AMA of ambulatory patient safety concerns reached the same conclusion.68

    What's missing from these estimates are true incidence data, as typically the denominators are not available or provided. Also missing are the aggregate rates of injury and harm. There is a clear need for additional research to identify better ways to bring diagnostic errors to light. Promising approaches in this regard include the use of trigger tools focusing on diagnostic error, the use of standardised patients, and encouraging both patients and physicians to voluntarily report errors they encounter.

    The most fundamental principle of performance improvement is that ‘You can't fix what you don't measure’. Efforts to begin addressing diagnostic error must begin with measurement. In no area of patient safety is this need more acute than in trying to identify the true incidence of diagnostic errors, and the harm associated with these events.

    References

    1. ↵
      1. Hastie T,
      2. Tibshirani R,
      3. Friedman J
      . The elements of statistical learning. 2nd edn, Springer, 2009.
    2. ↵
      1. Graber M,
      2. Wachter R,
      3. Cassel C
      . Bringing diagnosis into the quality and safety equations. JAMA 2012;308:1211–12.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Graber ML
      . Diagnostic error in medicine: a case of neglect. Jt Comm J Qual Patient Saf 2004;31:112–19.
      OpenUrl
    4. ↵
      1. Wachter R
      . Why diagnostic errors don't get any respect—and what can be done about them. Health Affairs 2010;29:1605–10.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Pronovost PJ,
      2. Colantuoni E
      . Measuring preventable harm. JAMA 2009;301:1273–75.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Berner E,
      2. Graber M
      . Overconfidence as a cause of diagnostic error in medicine. Am J Med 2008;121(5 Suppl Diagnostic Error in Medicine):S2–23.
      OpenUrlCrossRefPubMed
    7. ↵
      1. Elstein A
      . Clinical reasoning in medicine. In: Higgs J, ed. Clinical reasoning in the health professions. Oxford, England: Butterworth-Heinemann Ltd, 1995;49–59.
      1. Shojania K,
      2. Burton E,
      3. McDonald K,
      4. et al
      . The autopsy as an outcome and performance measure; Evidence report/Technology assessment #58 (Prepared by the University of California at San Francisco-Stanford Evidence-based Practice Center under Contract No.290–97–0013). AHRQ Publication No 03-E002. Rockville, MD: Agency for Healthcare Research and Quality, 2002.
      1. Sonderegger-Iseli K,
      2. Burger S,
      3. Muntwyler J,
      4. et al
      . Diagnostic errors in three medical eras: a necropsy study. Lancet 2000;355:2027–31.
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Blendon RJ,
      2. DesRoches CM,
      3. Brodie M,
      4. et al
      . Views of practicing physicians and the public on medical errors. N Engl J Med 2002;347:1933–40.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Singh H,
      2. Thomas E,
      3. Wilson L,
      4. et al
      . Errors of diagnosis in pediatric practice: a multisite survey. Pediatrics 2010;126:70–9.
      OpenUrlAbstract/FREE Full Text
      1. Peabody JW,
      2. Luck J,
      3. Jain S,
      4. et al
      . Assessing the accuracy of administrative data in health information systems. Med Care 2004;42:1066–72.
      OpenUrlCrossRefPubMedWeb of Science
      1. Beam CA,
      2. Layde PM,
      3. Sullivan DC
      . Variability in the interpretation of screening mammograms by US radiologists. Findings from a national sample. Arch Intern Med 1996;156:209–13.
      OpenUrlCrossRefPubMedWeb of Science
      1. Raab SS
      . Variability of practice in anatomic pathology and its effect on patient outcomes. Semin Diagn Pathol 2005;22:177–85.
      OpenUrlCrossRefPubMed
      1. Schiff GD,
      2. Hasan O,
      3. Kim S,
      4. et al
      . Diagnostic error in medicine—analysis of 583 physician-reported errors. Arch Int Med 2009;169:1881–87.
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Gandhi TK,
      2. Kachalia A,
      3. Thomas EJ,
      4. et al
      . Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Ann Intern Med 2006;145:488–96.
      OpenUrlCrossRefPubMedWeb of Science
      1. Charlton I,
      2. Jones K,
      3. Bain J
      . Delay in diagnosis of childhood asthma and its influence on respiratory consultation rates. Arch Dis Child 1991;66:633–35.
      OpenUrlAbstract/FREE Full Text
      1. Kowalski R,
      2. Claasen J,
      3. Kreiter K,
      4. et al
      . Initial misdiagnosis and outcome after subarrachnoid hemorrhage. JAMA 2004;291:866–69.
      OpenUrlCrossRefPubMedWeb of Science
      1. Hogan H,
      2. Healey F,
      3. Neale G,
      4. et al
      . Preventable deaths due to problems in care in English acute hospitals: a retrospectie case record review. BMJ Qual Saf 2012.
      1. Sevdalis N,
      2. Jacklin R,
      3. Arora S,
      4. et al
      . Diagnostic error in a national incident reporting system in the UK. J Eval Clin Pract 2010;16:1276–78.
      OpenUrlCrossRefPubMed
    21. ↵
      Isabel Healthcare. You Gov survey of medical misdiagnosis, 2005.
    22. ↵
      1. MacDonald O
      . Physician Perspectives on Preventing Diagnostic Errors. Quantia MD, 2011.
    23. ↵
      Physician Perspective Survey. Best Doctors, Inc, 2012.
    24. ↵
      1. Gehring K,
      2. Schwappach D,
      3. Battaglila M,
      4. et al
      . Frequency and harm associated with primary care safety incidents. Am J Manag Care 2012;18:e323–e37.
      OpenUrlPubMed
    25. ↵
      1. Weiner S,
      2. Schwartz A,
      3. Weaver F,
      4. et al
      . Contextual errors and failures in indivdualizing patient care. Ann Int Med 2010;153:69–75.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Schwartz A,
      2. Weiner S,
      3. Weaver F,
      4. et al
      . Uncharted territory: measuring costs of diagnostic errors outside the medical record. BMJ Qual Saf 2012.
    27. ↵
      1. Kostopoulou O,
      2. Oudhoff J,
      3. Nath R,
      4. et al
      . Predictors of diagnostic accuracy and safe management in difficult diagnostic problems in family medicine. Med Decis Making 2008;28:668–80.
      OpenUrlAbstract/FREE Full Text
    28. ↵
      1. Fitzgerald R
      . Error in radiology. Clin Radiol 2001;56:938–46.
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Foucar E
      . Error in anatomic pathology. Am J Clin Pathol 2001;116(Suppl Diagnostic Error in Medicine):S34–46.
      OpenUrlCrossRefPubMed
    30. ↵
      1. Westra WH,
      2. Kronz JD,
      3. Eisele DW
      . The impact of second opinion surgical pathology on the practice of head and neck surgery: a decade experience at a large referral hospital. Head Neck 2002;24:684–93.
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      1. Raab SS GD
      . Quality in Cancer Diagnosis. CA Cancer J Clin 2010;60:27.
      OpenUrl
    32. ↵
      1. Singh H,
      2. Arora HS,
      3. Vij MS,
      4. et al
      . Communication outcomes of critical imaging results in a computerized notification system. JAMIA 2007;14:459–66.
      OpenUrlAbstract/FREE Full Text
    33. ↵
      1. Singh H,
      2. Thomas EJ,
      3. Mani S,
      4. et al
      . Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential? Arch Intern Med 2009;169:1578–86.
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      1. Fenn P,
      2. Gray A,
      3. Rivero-Arias O,
      4. et al
      . The epidemiology of error: an analysis of databases of clinical negligence litigation. Manchester, UK: Manchester Centre for Healthcare Management, University of Manchester, 2004.
    35. ↵
      1. Tehrani A,
      2. Lee H,
      3. Mathews S,
      4. et al
      . 25-year summary of US malpractice claims for diagnostic errors 1986–2010; An analysis from the National Practitioner Data Bank. BMJ Qual Saf 2013.
    36. ↵
      1. Newman-Toker D,
      2. Robinson K,
      3. Edlow J
      . Frontline misdiagnosis of cerebrovascular events in the era of modern neuro-imaging: a systematic review. Ann Neurol 2008;64(Suppl 12 Diagnostic Error in Medicine):S17–18.
      OpenUrl
    37. ↵
      1. Kostopoulou O,
      2. Delaney BC,
      3. Munro CW
      . Diagnostic difficulty and error in primary care—a systematic review. Fam Pract 2008;25:400–13.
      OpenUrlAbstract/FREE Full Text
    38. ↵
      1. Singh H,
      2. Daci K,
      3. Petersen LA,
      4. et al
      . Missed opportunities to initiate endoscopic evaluation for colorectal cancer diagnosis. Am J Gastroenterol 2009;104:2543–54.
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. Singh H,
      2. Hirani K,
      3. Kadiyala H,
      4. et al
      . Characteristics and predictors of missed opportunities in lung cancer diagnosis. J Clinical Oncology 2010;28:3307–15.
      OpenUrlCrossRefWeb of Science
    40. ↵
      1. Kostopoulou O,
      2. Delaney B
      . Confidential reporting of patient safety events in primary care: results from a multilevel classification of cognitive and system factors. Qual Saf Health Care 2007;16:95–100.
      OpenUrlAbstract/FREE Full Text
    41. ↵
      1. Graber ML
      . Taking steps towards a safer future: measures to promote timely and accurate medical diagnosis. Am J Med 2008;121(Suppl 5 Diagnostic Error in Medicine):S43–6.
      OpenUrlCrossRefPubMed
    42. ↵
      1. Tsang C,
      2. Majeed A,
      3. Avlin P
      . Routinely recorded patient safety events in primary care: a literature review. Fam Pract 2012;29:8–15.
      OpenUrlAbstract/FREE Full Text
    43. ↵
      1. Levinson D
      . Adverse events in hospitals: national incidence among medicare beneficiaries. Department of Health and Human Services—Office of Inspector General, 2012.
    44. ↵
      1. Resar RK,
      2. Rozich JD,
      3. Classen D
      . Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care 2003;12(Suppl 2 Diagnostic Error in Medicine):ii39–45.
      OpenUrlAbstract/FREE Full Text
    45. ↵
      1. Classen D,
      2. Resar R,
      3. Griffin F,
      4. et al
      . Global Trigger Tool shows that adverse events in hospitals may be ten times greater than previously measured. Health Affairs 2011;30:581–89.
      OpenUrlAbstract/FREE Full Text
    46. ↵
      1. Schildmeijer K,
      2. Nilsson L,
      3. Arestedt K,
      4. et al
      . Assessment of adverse events in medical care: lack of consistency between experienced teams using the global trigger tool. BMJ Qual Saf 2012;31:307–14.
      OpenUrl
    47. ↵
      1. Singh H,
      2. Thomas EJ,
      3. Khan MM,
      4. et al
      . Identifying diagnostic errors in primary care using an electronic screening algorithm. Arch Intern Med 2007;167:302–8.
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      1. Singh H,
      2. Giardina T,
      3. Forjuoh S,
      4. et al
      . Electronic health record-based surveillance of diagnostic errors in primary care. BMJ Qual Saf 2011.
    49. ↵
      1. Singh H,
      2. Traber D,
      3. Meyer A,
      4. et al
      . Types and origins of diagnostic errors in primary care settings. JAMA Internal Med 2013.
    50. ↵
      1. Newman-Toker D,
      2. Moy E,
      3. Valente E,
      4. et al
      . Missed and delayed diagnosis of stroke in emergency department patients with headache or dizziness. Poster presentation: ISPOR 17th Annual Meeting. Washington, DC, 2012.
    51. ↵
      1. Schiff GD,
      2. Kim S,
      3. Krosnjar N,
      4. et al
      . Missed hypothyroidism diagnosis uncovered by linking laboratory and pharmacy data. Arch Intern Med 2005;165:574–7.
      OpenUrlCrossRefPubMedWeb of Science
    52. ↵
      1. Schiff G
      . Finding and fixing diagnosis errors: can triggers help? BMJ Qual Saf 2012;21:89–92.
      OpenUrlFREE Full Text
    53. ↵
      1. Burroughs TE,
      2. Waterman AD,
      3. Gallagher TH,
      4. et al
      . Patient concerns about medical errors in emergency departments. Acad Emerg Med 2005;23:57–64.
      OpenUrl
    54. ↵
      1. Schiff GD
      . Minimizing diagnostic error: the importance of follow-up and feedback. Am J Med 2008;121(5 Suppl Diagnostic Error in Medicine):S38–42.
      OpenUrlPubMed
    55. ↵
      1. Graber ML
      . Reducing diagnostic error in medicine—there's a job for everyone. NPSF Focus Patient Saf 2009;12:6–7.
      OpenUrl
    56. ↵
      1. Shrier I,
      2. Green S,
      3. Solin J,
      4. et al
      . Knowledge of and attitude toward patient confidentiality within three family medicine teaching units. Acad Med 1998;73:710–12.
      OpenUrlCrossRefPubMedWeb of Science
    57. ↵
      1. Weingart S,
      2. Pagovich O,
      3. DZ S,
      4. et al
      . What can hospitalized patients tell us about adverse events? Learning from patient-reported incidents. J Gen Intern Med 2005;20:7.
      OpenUrlCrossRefPubMedWeb of Science
    58. ↵
      1. Pear R
      . New system for patients to report medical mistakes. New York Times, 2012.
    59. ↵
      1. Hasegawa T,
      2. Fujita S,
      3. Seto K,
      4. et al
      . Patient identification and reporting of unsafe events at six hospitals in Japan. Jt Comm J Qual Patient Saf 2011;37:502–08.
      OpenUrlPubMed
    60. ↵
      1. Ohrn A,
      2. Elfstrom J,
      3. Liedgren C,
      4. et al
      . Reporting of sentinel events in Swedish hospitals: a comparison of severe adverse events reported by patients and providers. Jt Comm J Qual Patient Saf 2011;37:495–501.
      OpenUrlPubMed
    61. ↵
      1. Tierney WM
      . Adverse outpatient drug events – a problem and an opportunity. N Engl J Med 2003;348:1587–89.
      OpenUrlCrossRefPubMedWeb of Science
    62. ↵
      1. Phillips RL Jr..,
      2. Bartholomew LA,
      3. Dovey SM,
      4. et al
      . Learning from malpractice claims about negligent, adverse events in primary care in the United States. Qual Saf Health Care 2004;13:121–26.
      OpenUrlAbstract/FREE Full Text
    63. ↵
      1. Trowbridge R,
      2. Salvador O
      . Addressing diagnostic errors: an institutional approach. Focus Patient Saf Newsletter National Patient Saf Found 2010;13:1–5.
      OpenUrl
    64. ↵
      1. Leape L,
      2. Berwick D,
      3. Bates D
      . Counting deaths from medical errors. JAMA 2002;288:2405.
      OpenUrlCrossRefPubMedWeb of Science
    65. ↵
      1. Winters B,
      2. Custer J,
      3. Galvagno S,
      4. et al
      . Diagnostic errors in the intensive care unit: a systematic review of autopsy studies. BMJ Qual Saf 2012;21:894–902.
      OpenUrlAbstract/FREE Full Text
    66. ↵
      1. Shojania K
      . Deaths due to medical error: jumbo jets or just small propeller planes? BMJ Qual Saf 2012;21:709–12.
      OpenUrlFREE Full Text
    67. ↵
      1. Zwaan L,
      2. de Bruijne M,
      3. Wagner C,
      4. et al
      . Patient record review of the incidence, consequences, and causes of diagnostic adverse events. Arch Intern Med 2010;170:1015–21.
      OpenUrlCrossRefPubMed
    68. ↵
      1. Lorincz C,
      2. Drazen E,
      3. Sokol P,
      4. et al
      . Research in ambulatory patient safety 2000–2010: a 10-year review. Chicago, IL: American Medical Association, 2011.
    View Abstract

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.