Original ArticleAssisted self-report of health-services use showed excellent reliability in a longitudinal study of older adults
Introduction
Seniors use a variety of health services. Three main sources are available to researchers wishing to measure use. Claims databases, developed for administrative purposes, have long been considered the most complete source of information. However, they do not cover the wide spectrum of services used by the elderly, and their use for research purposes often reveals missing, duplicate, or miscoded data.
The second source, provider records, is not without its limitations mainly because seniors use services delivered by numerous practitioners and organizations. Researchers must be certain to identify records from all the resources used by participants and be aware that there are variations in the quality and necessity of record keeping across sectors and within providers [1].
The third source of information is respondent self-report, an accessible, simple, and cost-effective method. However, because self-report relies on respondents' ability to recollect past events, some have raised concerns for poor recall [2], [3] and telescoping [4]. Concordance between self-report and the other two means of data collection has frequently been studied with claims data or provider records often explicitly considered gold standards [1], [5], [6]. Although other studies recognize that these data sources are not without inaccuracies, conclusions generally state that participant responses are biased toward under- or overreporting utilization, which amounts to blaming self-report for the divergences [2], [3], [7]. However, such disagreements may not solely reflect errors in self-reports. For example, Ritter et al. [4] compared self-reported health-services utilization with computerized utilization records for a health-management organization. They examined 10 cases that showed the largest discrepancies and found that the computerized records were less accurate than self-reports in more than half of the cases.
Different techniques such as shortening and bounding the recall period, using memory aids like calendars and diaries, and interviewer-assisted recall have been shown to be associated with a better accuracy of reporting [8], [9]. Yet efforts to improve the accuracy of self-reports have not been instituted in many studies.
We aimed to evaluate self-report of health-services utilization when efforts are put into place to maximize its accuracy. The strategy, which will be described in the next section, can be viewed as assisted self-report and can easily be implemented in longitudinal studies. The purpose of this study was to examine test–retest and interinterviewer reliability of the method in a population of elders.
Section snippets
Data collection
The reliability study is an add-on to the longitudinal PRISMA (Program of Research to Integrate Services for the Maintenance of Autonomy) study [10] in which 1,501 community-dwelling persons aged 75 years or older were followed for up to 5 years by one of four interviewers. Face-to-face interviews were conducted yearly and participants were contacted bimonthly by phone. At study entry, participants received a calendar and were instructed to record dates of visits to the emergency room or
Results
Of the 50 subjects included in the test–retest study and the 50 subjects recruited for the interinterviewer study, 17 could not complete the 6-month follow-up (10 died, four were institutionalized, and three dropped out), reducing the sample sizes to 43 and 40.
Table 1 summarizes participants' characteristics. The samples for the test–retest and interinterviewer studies did not differ greatly. Fourteen (33%) and 16 (40%) informants were proxies for the test–retest and interinterviewer studies,
Discussion
Utilization of health services is a key outcome variable in a growing number of longitudinal studies measuring the effects of health-care interventions, calculating the associated costs, or evaluating cost-effectiveness. Older adults are often targeted by such studies for which self-report is the most accessible and cost-effective method to collect information on use. Although some researchers voice concerns about elders' recollection abilities, Kronenfeld [19] states that the accuracy of
Acknowledgment
The data reported in this article were collected as part of the PRISMA study, funded by the Canadian Health Services Research Foundation (CHSRF). We thank Myriam Bergeron and Nathalie-Audrey Joly for database management, Annie Lévesque for a first glance at the literature on the subject as well as Marie-Claude Boissé, Valérie Guillot, Isabelle Labrecque, and Dany Simard, who conducted the telephone interviews.
References (21)
- et al.
Comparison of self-reported and medical record health care utilization measures
J Clin Epidemiol
(1996) - et al.
Self-report was a viable method for obtaining health care utilization data in community-dwelling seniors
J Clin Epidemiol
(2005) - et al.
Self-reports of health care utilization compared to provider records
J Clin Epidemiol
(2001) The cross-cultural validity of self-reported use of health care: a comparison of survey and registration data
J Clin Epidemiol
(2000)- et al.
Mini-Mental State: a practical method for grading the cognitive state of patients for the clinician
J Psychiatr Res
(1975) - et al.
High agreement but low kappa: I. The problems of two paradoxes
J Clin Epidemiol
(1990) - et al.
“Precision” and “accuracy”: two terms that are neither
J Clin Epidemiol
(2006) - et al.
Agreement between self-reported and routinely collected health-care utilization data among seniors
Health Serv Res
(2002) - et al.
Older adults' reports of formal care hours and administrative records
Gerontologist
(2004) - et al.
Feasibility and accuracy of a postcard diary system for tracking healthcare utilization of community-dwelling older persons
J Am Geriatr Soc
(1995)