Original Article
Assisted self-report of health-services use showed excellent reliability in a longitudinal study of older adults

https://doi.org/10.1016/j.jclinepi.2006.12.011Get rights and content

Abstract

Objective

Seniors use a wide variety of health services delivered by numerous practitioners and organizations. Self-report is the most accessible and cost-effective method to collect information on their use. It is thus important to demonstrate the reliability of this approach.

Study Design and Setting

As part of a longitudinal study on the effect of an integrated service delivery system, participants (or their proxies) were instructed to use a calendar to record their use of health services. Every 2 months, an interviewer collected use since the last phone contact. A subsample was recontacted by the same or another interviewer to estimate test–retest and interinterviewer reliability, respectively. Data collections were compared using delta and weighted kappa as well as intraclass correlation coefficients.

Results

Almost perfect agreement was obtained for hospitalization, day surgery, visits to general practitioners and medical specialists, help for home maintenance, and use of voluntary services. Agreement was substantial for visits to the emergency room and home help for personal care. For visits to or by nurses and other health professionals, agreement can be qualified as moderate-to-substantial.

Conclusion

Assisted self-report of health-services use by older adults or their proxies through bimonthly phone calls is reliable.

Introduction

Seniors use a variety of health services. Three main sources are available to researchers wishing to measure use. Claims databases, developed for administrative purposes, have long been considered the most complete source of information. However, they do not cover the wide spectrum of services used by the elderly, and their use for research purposes often reveals missing, duplicate, or miscoded data.

The second source, provider records, is not without its limitations mainly because seniors use services delivered by numerous practitioners and organizations. Researchers must be certain to identify records from all the resources used by participants and be aware that there are variations in the quality and necessity of record keeping across sectors and within providers [1].

The third source of information is respondent self-report, an accessible, simple, and cost-effective method. However, because self-report relies on respondents' ability to recollect past events, some have raised concerns for poor recall [2], [3] and telescoping [4]. Concordance between self-report and the other two means of data collection has frequently been studied with claims data or provider records often explicitly considered gold standards [1], [5], [6]. Although other studies recognize that these data sources are not without inaccuracies, conclusions generally state that participant responses are biased toward under- or overreporting utilization, which amounts to blaming self-report for the divergences [2], [3], [7]. However, such disagreements may not solely reflect errors in self-reports. For example, Ritter et al. [4] compared self-reported health-services utilization with computerized utilization records for a health-management organization. They examined 10 cases that showed the largest discrepancies and found that the computerized records were less accurate than self-reports in more than half of the cases.

Different techniques such as shortening and bounding the recall period, using memory aids like calendars and diaries, and interviewer-assisted recall have been shown to be associated with a better accuracy of reporting [8], [9]. Yet efforts to improve the accuracy of self-reports have not been instituted in many studies.

We aimed to evaluate self-report of health-services utilization when efforts are put into place to maximize its accuracy. The strategy, which will be described in the next section, can be viewed as assisted self-report and can easily be implemented in longitudinal studies. The purpose of this study was to examine test–retest and interinterviewer reliability of the method in a population of elders.

Section snippets

Data collection

The reliability study is an add-on to the longitudinal PRISMA (Program of Research to Integrate Services for the Maintenance of Autonomy) study [10] in which 1,501 community-dwelling persons aged 75 years or older were followed for up to 5 years by one of four interviewers. Face-to-face interviews were conducted yearly and participants were contacted bimonthly by phone. At study entry, participants received a calendar and were instructed to record dates of visits to the emergency room or

Results

Of the 50 subjects included in the test–retest study and the 50 subjects recruited for the interinterviewer study, 17 could not complete the 6-month follow-up (10 died, four were institutionalized, and three dropped out), reducing the sample sizes to 43 and 40.

Table 1 summarizes participants' characteristics. The samples for the test–retest and interinterviewer studies did not differ greatly. Fourteen (33%) and 16 (40%) informants were proxies for the test–retest and interinterviewer studies,

Discussion

Utilization of health services is a key outcome variable in a growing number of longitudinal studies measuring the effects of health-care interventions, calculating the associated costs, or evaluating cost-effectiveness. Older adults are often targeted by such studies for which self-report is the most accessible and cost-effective method to collect information on use. Although some researchers voice concerns about elders' recollection abilities, Kronenfeld [19] states that the accuracy of

Acknowledgment

The data reported in this article were collected as part of the PRISMA study, funded by the Canadian Health Services Research Foundation (CHSRF). We thank Myriam Bergeron and Nathalie-Audrey Joly for database management, Annie Lévesque for a first glance at the literature on the subject as well as Marie-Claude Boissé, Valérie Guillot, Isabelle Labrecque, and Dany Simard, who conducted the telephone interviews.

References (21)

There are more references available in the full text version of this article.

Cited by (0)

View full text