Original ArticleHealth care utilization: measurement using primary care records and patient recall both showed bias
Introduction
Epidemiologic studies (including general population surveys) often ask subjects to self-report health problems and recall related consultations to primary care. In the UK, this self-reported data can be used for the purposes of population-based needs assessment by Primary Care Trusts who plan and commission health services for their local population. Unmet needs in the community may be identified if there are people with health problems who are not seeking health care, and self-reported data can also be used for costing health care related to specific conditions. An alternative source of health care utilization data is the primary care medical record.
Many studies have found variable levels of agreement on diagnoses between self-reported questionnaire data and medical records [1], [2], [3], [4], [5], [6], [7], [8], [9], [10]. Fewer studies have investigated the agreement between recalled consultation for specific problems and the record of that consultation in the paper or computer records of the patient's general practice [11], [12], [13], [14], [15], [16]. In general, these latter studies have found disagreement. However, none have established whether the discrepancies are predominantly a problem of patient recall or of underrecording in the practice. Identifying the reasons for disagreement and estimating the bias in using either source to obtain the prevalence of consultation will assist researchers and policy makers in planning and interpreting research and in the provision of health care services.
We have therefore assessed (1) the agreement between recalled and recorded general practice consultation using knee pain as an example, and (2) reasons for discrepancies between self-report and practice records by investigating whether such discrepancies disappear if the medical record search is expanded in terms of the time period and the diagnoses covered.
Section snippets
Methods
Six thousand seven hundred ninety-two (adjusted response 77%) subjects returned a completed questionnaire from a general population survey of all adults (n = 8,995) aged 50 and over registered at three general practices in North Staffordshire, UK [17]. Three thousand twenty-three responders reported knee pain and, of these, 2,414 (80%) consented to medical record review and responded to the question on consulting a general practitioner (GP) for their knee pain. These 2,414 respondents form the
Part 1—All subjects
For steps 1–3, only knee disorder contacts recorded in the general practice medical records as being to the GP surgery, a home visit, or by telephone were considered as verifying the recalled consultation. This meant entries of hospital letters, for example, were excluded. After each step, the percentage of those with a recalled consultation who had a recorded consultation identified in the search for that step was determined. This was repeated for those who said they had not consulted. To show
Part 2—Subjects without a recorded consultation in Part 1
All subjects who recalled a knee pain consultation but had no record of a knee disorder consultation in steps 1–3 were selected. An equal number of those said they had not consulted (equally divided across practices) and had no record of a knee disorder in steps 1–3 were randomly selected. All searches for steps 4–7 were performed by KJ and CJ, who were blinded as to whether the subject had recalled a consultation or not.
Part 1: Steps 1–3
Two hundred fourteen (27%) of the 807 responders with a recalled consultation for knee pain had a knee-related Read Code in their medical records for the 12 months before the survey (Table 1). Agreement on consultation between the medical records for this 12-month period and self-report was slight based on Shrout's [20] definitions (κ = 0.30; 95% CI 0.27, 0.34). By extending the search to knee problems mentioned in the text, the number of responders who reported a consultation who also had a
Discussion
Measures of health service utilization may be derived from self-reported population data or from general practice medical records. We have compared the prevalence of self-reported consultation to that recorded in general practice and determined that substantial discrepancy exists. Only 40% of self-reported consulters over 1 year could be verified by examining medical records for the identical time period for knee problems. In contrast, the rate of incorrect reporting of nonconsultation in the
Acknowledgments
We are very grateful to all the survey responders, the North Staffordshire and GP Research Network, and administration staff at Primary Care Sciences Research Centre, Keele University. This study was funded by an NHS Executive (West Midlands) New Blood Research Training Fellowship and the Haywood Rheumatism Research and Development Foundation.
References (30)
- et al.
Self-reports and general practitioner information on the presence of chronic diseases in community dwelling elderly—a study on the accuracy of patients self-reports and on determinants of inaccuracy
J Clin Epidemiol
(1996) - et al.
Cardiovascular diseases and risk factors in a population-based study in The Netherlands: agreement between questionnaire information and medical records
Neth J Med
(1999) - et al.
Validity of a self-reported history of doctor-diagnosed angina
J Clin Epidemiol
(1999) - et al.
The accuracy of self-reported fractures in older people
J Clin Epidemiol
(2002) - et al.
Agreement between self-report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure
J Clin Epidemiol
(2004) - et al.
Comparison of self-reported and medical record health care utilization measures
J Clin Epidemiol
(1996) - et al.
Self-reports of health care utilization compared to provider records
J Clin Epidemiol
(2001) - et al.
Data quality of general practice electronic health records: the impact of a program of assessments, feedback, and training
J Am Med Inform Assoc
(2004) - et al.
Measuring the population impact of knee pain and disability with the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC)
Pain
(2002) - et al.
Comparing self-reported and physician-reported medical history
Am J Epidemiol
(1994)
The validity of self-reports on chronic disease: characteristics of underreporters and implications for the planning of services
Public Health Rev
Validation of patient recall of doctor-diagnosed heart attack and stroke: a postal questionnaire and record review comparison
Am J Epidemiol
Patient–doctor concordance in elderly women's self-reported health and medical records
Methods Inf Med
Agreement between self-report of disease diagnoses and medical record validation in disabled older women: factors that modify agreement
J Am Geriatr Soc
The validity of self-reported physician utilization measures
Med Care
Cited by (44)
Personal exposure to fine particulate matter (PM<inf>2.5</inf>) and self-reported asthma-related health
2023, Social Science and MedicineBest Evidence Osteoarthritis Care: What Are the Recommendations and What Is Needed to Improve Practice?
2022, Clinics in Geriatric MedicineCitation Excerpt :For example, QIs target different joints (eg, hip vs knee OA), whereas others target the type or frequency of treatment,57 or are for use by different health professions (eg, GP or physiotherapist58). QI assessments can be undertaken through an audit of medical records, although QI results based on medical records should be treated with caution as consumer perceptions of their treatment outcome may be different from that of their health care practitioner.59 Therefore, consumer-reported QI sets developed from OA clinical practice guidelines are considered a better option to effectively monitor and evaluate care than medical records.60
Situations and mechanisms of non-contact knee injury in adult netball: A systematic review
2021, Physical Therapy in SportCitation Excerpt :The majority of studies were large scale epidemiological studies designed to investigate the most common injuries in netball (Hopper & Elliott, 1993; Hopper et al., 1995a, 1995b) where identification of the mechanism of injury was a secondary goal and was self-reported. The self-report method risks incomplete or inadequate data because it relies on participants’ memory, or the memory of others, both of which may be inaccurate (Althubaiti, 2016; Gabbe, Finch, Bennell, & Wajswelner, 2003; Jordan, Jinks, & Croft, 2006). It is also important to consider whether there was any bias from the clinicians gathering the data and if they asked leading questions to the participants who may have been unsure how they sustained the injury (Althubaiti, 2016).
Simplified risk prediction indices do not accurately predict 30-day death or readmission after discharge following colorectal surgery
2019, Surgery (United States)Citation Excerpt :Using the number of visits to an emergency room in the 6 months before the index admission would be difficult to assess based on patient recall without the aid of an EMR or large administrative data sets like the one used here. To address this limitation, we converted this original LACE variable to a binary variable for any number of emergency room visits to simplify assessment and decrease error by recall bias.27,28 The impact of this work can be far-reaching.
Methodological challenges collecting parent phone-call healthcare utilization data
2016, Applied Nursing ResearchCitation Excerpt :The Internet is increasingly being used as a means of patient self-report, with good effect (Bhandari & Wagner, 2006). Because systematic errors can occur in both self-report data and provider documentation, a dual source of data collection is likely to yield more robust and accurate data than a single source (Jordan, Jinks, & Croft, 2006; Killeen et al., 2004). Analysis of parent call data to pediatric providers has been shown to be an important indicator of healthcare utilization, as it can identify potential cost savings not otherwise considered in healthcare utilization studies (Jordan et al., 2006; Leibowitz, Day, & Dunt, 2003).
A multisource approach to health care use: concordance between register and self-reported physician visits in the foreign-born population in Finland
2022, BMC Medical Research Methodology