Introduction

Chronic obstructive pulmonary disease (COPD) is a chronic, progressive, and incapacitating disease. Traditionally, treatment of COPD is focused on improving or maintaining physiological functioning of the patient. However, in the past decade, it is recognized that besides physiological functioning also symptoms, functional limitations and quality of life (QoL) are important domains of health status (HS) in these patients [1, 2].

Studies have shown that symptoms, functional impairment, and QoL are relatively unrelated to physiological functioning [3–5]. In a previous study, we also have shown that these four main domains of HS were shown to be subdivided into 15 relatively unrelated sub-domains [6]. An individual patient may experience clinically relevant problems in some of these sub-domains, but not in other sub-domains. As a consequence, to tailor treatment to the specific needs of the individual patient, it is necessary to obtain an integral and detailed picture of HS of all sub-domains.

In routine care, physiological functioning is measured by lung function tests. Although, many generic and disease-specific questionnaires are available to measure symptoms, functional impairment and QoL [3, 7], these three main domains commonly are not measured in routine care. A major reason for this is that current questionnaires are not suitable for application in routine care. This has several causes. First, questionnaires commonly consist of many items and are therefore time-consuming. Second, the scoring of questionnaires is often complex and has to be done by hand, which is also time-consuming. Third, the clinical meaning of a particular score is often unclear due to the lack of adequate normative data. Normality cannot be defined by absence of, e.g., symptoms. As patients with COPD are often elderly, the presence of an elevated score can also be the result of normal aging instead of being the result of having COPD. Moreover, symptoms such as fatigue or shortness of breath may be experienced by healthy persons as well. Hence, it is important to know whether a score represents normal functioning or clinically relevant problems. Fourth, in a previous study, we found that existing questionnaires measure only one to three aspects of HS [6]. In addition, we have shown that there is considerable overlap between questionnaires with respect to the specific sub-domains they measure. This implicates that, for an integral and detailed assessment of HS, a combination of several instruments is required in that overlap should be avoided.

Consequently, the following criteria must be fulfilled to permit adequate assessment of HS suitable and useful in routine care: (1) a broad spectrum of aspects of HS has to be measured to obtain a comprehensive and detailed picture; (2) instruments should be as short as possible, but still have enough items to warrant adequate reliability; (3) overlap should be avoided; (4) scoring must be simple and results should be available immediately, preferably this should be automated; and (5) results should be easy to interpret and should indicate if a particular score indicates normal functioning or clinically relevant problems. Such an instrument would provide the clinician with a tool to identify patients who need additional treatment and provides a detailed picture on the type and severity of problems in HS of an individual patient, and thereby can help the clinician in patient-tailored treatment. In addition, it also could be used for outcome assessment, as part of an intervention, and for research studies.

The aim of the present study was to compose a battery of existing questionnaires that fulfills all above-mentioned criteria for clinical applicability in a routine care outpatient setting. In this process we used the Nijmegen Integral Assessment Framework (NIAF) [6] as a guide for the selection of instruments. The NIAF is an evidence-based framework that organizes tests and questionnaires by indicating which sub-domains of HS actually are measured by specific (sub-)scales of various existing instruments. Data of matched control groups were used to determine cut-offs for each instrument to indicate normal functioning versus clinically relevant problems. In addition, existing software for computerized questionnaire completion was adapted specifically to facilitate clinical applicability of the test battery and easy interpretation of results.

Method

Subjects

COPD-study group

For the selection of instruments of the Nijmegen Clinical Screening Instrument (NCSI) we used the data from a sample of COPD patients that are representative for patients with stable COPD (GOLD II and III), with no primary co-morbidity, in routine care at outpatient clinics. This COPD-study group was recruited from three different pulmonary outpatient clinics in the Netherlands: University Lungcenter Dekkerswald of the Radboud University Nijmegen Medical Center, Maas Hospital Boxmeer and Rijnstate Hospital Arnhem. Inclusion criteria were diagnosis of GOLD II/III (FEV1% predicted between 30 and 80%), FEV1/FVC < 70%, and reversibility of obstruction <12%. Exclusion criteria were primary co-morbidity that may dominate HS, recent participation in a rehabilitation program (within previous 6 months), inability to speak or read Dutch, acute exacerbation of COPD, and inability to completely adhere to the research protocol. Screening the patient charts resulted into 361 eligible patients. A pulmonologist asked these patients for permission to be called by the investigator, and 316 (88%) agreed to be called for further information. One hundred sixty-eight patients (47%) participated in this study (see Table 1 for patient characteristics). Reasons for non-participation were diverse; predominantly being too busy, refusing cycle-ergometry, and travel problems. Patients gave informed consent and the local Ethics Committee approved this study.

Table 1 Patient characteristics expressed in number (%) unless stated otherwise of the COPD outpatient study group (OP), patients included in pulmonary rehabilitation (PR), and Healthy controls (HC)
Full size table

Control samples

To determine the score range of the selected instruments that represents clinically relevant problems, we recruited patients with COPD included in a clinical multi-disciplinary pulmonary rehabilitation program at the University Lungcenter Dekkerswald of the Radboud University Nijmegen Medical Center. A key requirement for inclusion in this program is that patients have to experience clinically relevant problems in multiple areas of HS. The decision on this requirement was based on a 3-days intake procedure, in which elaborate assessments, physiological tests, and clinical interviews by seven disciplines (pulmonologist, psychotherapist, physiotherapist, nurse, dietitian, psychomotor therapist, social worker) took place. The results of these assessments and interviews are evaluated in a multi-disciplinary discussion. Inclusion criteria were diagnosis of COPD [8] and clinically relevant problems in multiple areas of HS. Exclusion criteria: mild or isolated problems in HS and inability to speak or read Dutch. Subjects were matched to the COPD-study group by age and sex. See Table 1 for a description of this sample.

To determine the score range of instruments indicating normal functioning, we recruited healthy controls by an advertisement in a regional newspaper. Exclusion criteria were having asthma or COPD, being under regular treatment of any specialist and/or inability to speak or read Dutch. The subjects were screened for absence of chronic illnesses by one of the investigators (LD). Subsequently, lung function testing was performed. Based on all assessments a pulmonologist (JM) decided whether a person could be included or not. Subjects were matched on age and sex to the COPD-study group. See Table 1 for a description of this sample.

Measurements

In a previous study, we defined four domains of HS; physiological functioning, symptoms, functional impairment, and QoL [6]. These four main domains of HS were found to be subdivided into 15 relatively unique sub-domains, which together constitute the NIAF for COPD. See Fig. 1 for a general description of the development and validation of the NIAF. For a detailed description on the development and validation of the NIAF see elsewhere [6].

Fig. 1
figure 1

Main stages of the development of the Nijmegen Integral Assessment Framework (NIAF) for COPD (previous study) and the development of the Nijmegen Clinical Screening Instrument (NCSI) for COPD (present study)

Full size image

In a recent, yet, unpublished study, we found fatigue to be an important symptom in COPD that is relatively independent to the other sub-domains. For that reason, fatigue was included in the framework as a separate sub-domain of the main domain symptoms. Table 2 shows the instruments that measure the sub-domains of HS.

Table 2 Sub-domains of the Nijmegen Integral Assessment Framework (NIAF), their corresponding instruments, subscales/measurements
Full size table

Questionnaire completion was performed by the TestOrganiser, which is a computerized questionnaire system developed by the Department of Medical Psychology and the Department of Instrumental Services of the Radboud University Nijmegen Medical Centre [6]. Questionnaires are presented in the same layout as paper-and-pencil versions, items cannot be skipped, and both scoring and data storage are automated.

Construction of the Nijmegen Clinical Screening Instrument

The NIAF organizes existing instruments by the sub-domains of HS they measure. Each sub-domain was measured by several tests or instruments or subscales of instruments (Table 2), and can be used interchangeably. Based on the following criteria we selected for each sub-domain one or two instruments for inclusion of the NCSI.

  1. A.

    Preliminary selection of instruments

    1. 1.

      The scores on selected instruments should show a correlation of >0.70 with the original NIAF-STS [9].

    2. 2.

      The selected instruments must be completed in as little time as possible (preferably <30 min), in other words a minimum number of items but should show good reliability (Cronbach’s alpha > 0.70).

    3. 3.

      Although all instruments included in the NIAF are clinically relevant, in the selection process of instruments we also considered which instrument was most clinically relevant. These decisions were based on clinical experience of the pulmonologists (JM, YH, RD) and the clinical psychologist (JV).

  2. B.

    Statistical analysis on the preliminary selection

    1. 1.

      For each sub-domain of HS, selected instruments were combined into a sub-domain total score (NCSI-STS) by adding scores of respective instruments. Spearman correlation coefficients between the original NIAF-STS and the new NCSI-STS were calculated and had to be higher than 0.70, which is considered a criterion for instruments to measure the same concept [9, 10].

    2. 2.

      To test possible overlap between the sub-domains the inter-correlations between all NCSI-STS were calculated by spearman coefficients, and should be lower than 0.70.

    3. 3.

      The Cronbach’s alpha reliability coefficients (internal consistency) of each NCSI-STS should be at least be moderate (>0.50) and preferably >0.70 [9].

Construction of normative data

For each subscale, the total score range of the COPD-study group was transformed to C-scores. C-scores are similar to percentile scores, but differentiate more in the extremes of the score range and correct for skewed distributions. The score range is 1–11, and the scores refer to the following percentiles, respectively: 1.2–4.0–10.6–22.7–40.1–59.9–77.3–89.4–96.0–98.8–100%. For each instrument the score belonging to the 80th percentile of the Healthy Controls was used as the maximal score of normal functioning (green colored score range), and the score belonging to the 20th percentile of the pulmonary rehabilitation patients was used as the minimum score representing clinically relevant problems (red colored score range). The area between green and red has been labeled ‘elevated’(yellow).

New features of the TestOrganiser

The TestOrganiser was originally developed for the purpose of data collection in research. In the past 3 years, the TestOrganiser has been implemented in our inpatient and outpatient clinic to develop and test clinical applicability and patient acceptability in routine care. The software of the TestOrganiser was revised in several aspects and new features were developed. These revisions particularly concerned automated data processing. The most important new feature is the graphical presentation of results on the level of an individual patient (the PatientProfileChart) to facilitate ease of interpretation of results for clinical purposes.

Results

No significant differences were found between the COPD-study group and the two control groups with respect to age and sex (Table 1).

Construction of the Nijmegen Clinical Screening Instrument

  1. A.

    Preliminary selection of instruments

The preliminary selection of instruments for the NCSI is shown in Table 2. The instruments in bold were selected for the NCSI. The sub-domains exercise capacity, gas exchange and muscle strength require cycle-ergometry testing and muscle strength tests are too time-consuming for use in a routine care outpatient setting and, therefore, were excluded. The sub-domains expected dyspnea (main domain symptoms) and actual physical activity (main domain functional impairment) were excluded because these tests also are too time-consuming: the PARS-expected dyspnea consist of 20 items, and the accelerometer has to be worn for 12 days.

With respect to the sub-domain dyspnea emotions (main domain symptoms) we included dyspnea-related anxiety instead of dyspnea-related mood despite the higher correlation of the latter, because dyspnea-related anxiety is far more common in COPD than dyspnea-related depressed mood. With respect to the sub-domain subjective symptoms (main domain symptoms), we included the PARS-D global dyspnea burden (1 item) instead of the PARS-D activity (14 items) for reasons of brevity.

  1. B.

    Statistical analysis on the preliminary selection

The correlations between the sub-domain total scores of the NIAF (NIAF-STS) and the NCSI (NCSI-STS) all exceeded 0.70, which indicates that the NCSI-STS are conceptually similar to the NIAF-STS (Table 3). In addition, all Cronbach’s alphas of the NCSI-STS were >0.70, except those of general QoL (0.61) and satisfaction relations (0.64) (Table 3). In general, there was none or at best moderate overlap between the sub-domains of the NCSI-STS as expressed by non-significant to at best moderate inter-correlations (Table 4).

Table 3 Correlations between the sub-domains measured by the NCSI-sub-domain total scores (NCSI-STS) versus the NIAF-sub-domain total scores (NIAF-STS) (P < 0.01), and Cronbach’s alpha reliability coefficients of all NCSI-STS
Full size table
Table 4 Intercorrelations between the Nijmegen Clinical Screening Instrument sub-domain total scores (NCSI-STS)
Full size table

Additional items were added to measure smoking-status (yes/no) and willingness to quit smoking (yes/no).

Construction of normative data

Characteristics of the sub-domains included in the NCSI for all study groups are presented in Table 5. As expected, in healthy controls there was a strong floor effect on disease-related domains: symptoms (except fatigue) and functional impairment. In general, there were no evident problems related to floor and ceiling effects in both COPD groups. As expected, the pulmonary rehabilitation control group showed the highest scores on all sub-domains and healthy controls the lowest.

Table 5 Characteristics of the three groups on the sub-domains of HS; the minimal, maximal scores for each sub-domain, mean (SD), % patients with the lowest possible score (floor), and % of patients with the highest possible score (ceiling)
Full size table

New features of the TestOrganiser

To enable patients with no prior computer experience to complete the questionnaires easily, a simple response board was developed with a minimum of (large) buttons. A network function was integrated that enables immediate access to the results after test completion on every computer in the hospital. The most important new feature is the automatic production of graphical representations of the results: the PatientProfileChart (see Fig. 2). The PatientProfileChart (PPC) provides a graphical presentation of the scores of an individual patient. Each column represents a specific instrument within a sub-domain. All score ranges are based on the reference sample (COPD-study group) and are expressed as C-scores. The x represents the score of the individual patient. First, the x indicates how a patient scores in relation to the general COPD population. For example, the patient in Fig. 2 had a raw score on Depression of 3 which falls in the 7th C-score of the COPD reference sample. This means that 77, 3% of the reference sample had a lower score. Second, the score range of each instrument is divided into colored ranges that allow absolute interpretations. The green score range indicates ‘normal functioning’, the yellow score range indicates ‘mild problems’, and the red score range indicates ‘clinically relevant problems’. The patient in Fig. 2 scored in the yellow area (‘mild problems’). Thus, although this patient a higher score than 77% of the COPD reference sample (7th C-score), still this score did not indicate clinically relevant problems.

Fig. 2
figure 2

The PatientProfileChart; graphical representations of the patients’ results

Full size image

Discussion

In the present study, we developed the NCSI that is short enough to be completed in routine care, but stills provides much detail on a patient’s HS. In addition, the TestOrganiser was adapted to further improve clinical applicability of the NCSI in routine care. To facilitate interpretation of a patient’s scores, we developed the PatientProfileChart that presents results graphically. In addition, we determined cut-offs based on reference groups indicating whether a particular score indicates normal functioning or clinically relevant problems.

Guidelines for treatment of COPD emphasize the importance of maintaining and optimizing HS [8, 11, 12], and describe for every COPD severity stage (based on the degree of airway obstruction; I–IV) what type of treatment is indicated. Pulmonary rehabilitation, for example, is indicated for GOLD stages III–IV. However, HS consists of four main domains: physiological functioning, symptoms, functional impairment, and quality of life [1, 2, 6, 13], divided into at least unique 15 sub-domains [6]. Given the findings of many studies showing that FEV1 is poorly related to symptoms, functional impairment and quality of life [3–5, 14] it is impossible to determine the status of other sub-domains of HS on the basis of FEV1 alone. Thus, FEV1 gives no information on any aspect of HS other than airway obstruction, and as such is a poor indicator for specific interventions such as pulmonary rehabilitation. Consequently, tailoring treatment to the needs of the individual patient requires an integral and detailed picture of the individual patient’s HS by measuring of all four main domains and their many sub-domains.

Many generic and disease-specific instruments have been developed to measure HS in COPD patients [3, 7]. However, most of these instruments are rather lengthy, scoring is time-consuming, commonly measure only few aspects of HS, and in many cases it is unclear whether a score represents normal functioning or clinically relevant problems, due to the lack of normative data. In addition, as existing instruments measure only few aspects of HS, the need for a detailed assessment of HS to enable patient-tailored treatment requires the combination of multiple instruments. However, such a battery of instruments would diminish clinical applicability even further as this would increase problems with regard to the length of instruments, time-consuming scoring procedures, as well as the overlap found between instruments [6].

The need for short questionnaires that allow measurement of symptoms, functional impairment, and quality of life in routine care has been acknowledged by other researchers. Several instruments have been developed for this purpose; the clinical COPD questionnaire (CCQ) [15], the respiratory illness questionnaire—monitoring 10 (RIQ-MON10) [16], and the EuroQOL (EQ-5D) [17]. These instruments are short (six to ten items) and have shown good validity and reliability. However, the CCQ measures only three sub-domains of HS, the RIQ-MON10 measures only two sub-domains, and the EQ-5D measures only three sub-domains. Although, these instruments are short and easy to score, these do not provide a detailed picture of the patient’s HS, and lack normative data indicating normal functioning versus clinically relevant problems. Hence, these instruments still do not satisfy all requirements for clinical applicability as pointed out in the introduction.

We did not develop a new instrument, as we did not want to add to the abundance of instruments already available, but we set out for a new approach that would render existing instruments suitable for use in routine care. We composed a battery of existing instruments with a minimum number of items, but with a maximum of detail of HS, a minimum of overlap between instruments, and good reliability and validity. Although clinical considerations did play a role in the selection process of instruments, decisions were not based on personal preferences of the researchers, or on how frequent a particular instrument is used in COPD research. The selection of instruments from the empirically validated integral assessment framework of HS in COPD [6], the NIAF, was primarily guided by statistical analyses. The NIAF contains 16 sub-domains of HS covering the main domains physiological functioning, symptoms, functional impairment, and quality of life. In addition, this framework provides additional validity information on many existing instruments: it indicates what sub-domain(s) of HS specific instruments measure, and it indicates which instruments measure the same sub-domains of HS.

Although the NCSI enables a quick (15–25 min) and detailed assessment of HS, typical questionnaire problems such as complex scoring procedures and the problem of interpretability of results remained. To provide a solution for these problems the software of the TestOrganiser was adapted. This involved automatic scoring, a network facility, and the development of a special response board. After instructions, additional questions of patients predominantly were related to the content of the questionnaire items, and rarely with regard to computer operating.

The most important new feature of the TestOrganiser is the PatientProfileChart. Immediately after completion of the NCSI, the PatientProfileChart is generated by the TestOrganiser, is available on every authorized computer in the clinic, and can immediately be discussed with the patient. The interpretation of the PatientProfileChart is very easy for the clinician, but also for the patient. A short training for clinicians is sufficient to learn how to interpret the PatientProfileChart. The colored areas of the score range of each instrument indicate whether a patient shows normal functioning in a particular sub-domain of HS or clinically relevant problems.

Psychometric properties of the NCSI are good. The correlations between the NCSI-STS and the corresponding NIAF-STS were high and well above the criterion for conceptual similarity. Within the NCSI there was little overlap between the NCSI-STS as expressed by non-significant to at best moderate inter-correlations. This also indicates that each sub-domain of the NCSI represents a unique aspect of a patient’s HS. The internal consistency of the NCSI-sub-domain total scores in general were good, except the general Qol and satisfaction relations.

Some methodological issues need to be addressed. First, in the present study the test–retest reliability, and the responsiveness-to-change of the instruments used to measure the sub-domains of HS were not tested. However, inclusion of instruments with adequate psychometric properties was one of the selection criteria for the NIAF. For most of the included instruments test–retest reliability and responsiveness-to-change were found adequate in other studies [18, 19, 21, 24]. Second, not all sub-domains of HS are measured by the NCSI. Some sub-domains required measurements that are too time-consuming for use in routine care at an outpatient clinic (e.g. cycle-ergometry testing, accelerometry). Future studies are needed to find alternatives that can be used for measuring those sub-domains in routine care. Nevertheless, the decision on what specific measurements are too time-consuming also may depend on specific settings. The final issue refers to the use of control groups and the need for cut-offs. Normal functioning cannot be defined by absence of symptoms or functional impairment, for example, due to effects of normal ageing. This means that the upper part of the score range in healthy subjects indicates abnormal functioning. Therefore, we used the 80th percentile of healthy controls as the upper limit of normal functioning. Similarly, ‘clinically relevant problems’ cannot be defined by the mere presence of such problems. For example, healthy subjects may experience fatigue or shortness of breath as well. In addition, even patients with multiple and severe problems in HS (the inpatient pulmonary rehabilitation controls) may not have problems in all sub-domains. So we assumed that for each sub-domain the lower part of the score range of the rehabilitation patients overlaps with the score range of normal functioning or mild problems. Therefore, we chose the 20th percentile of the pulmonary rehabilitation patients as the cut-offs for clinically relevant problems. Although the method we used to calculate cut-off scores indicating normal functioning versus clinically relevant problems is an accepted approach [25], to a certain degree these cut-offs remain arbitrary. However, decisions on, for example, which patients do need additional treatment versus those who do not, never depend on one single sub-domain, but on the profile on all sub-domains. Most important criteria in this respect are the number of sub-domains showing clinically relevant problems and discrepancies between the severity of physiological sub-domains versus the sub-domains measuring symptoms, functional impairment, or quality of life. This may render the arbitrariness of cut-offs less problematic. In addition, the clinical relevance of the cut-offs (i.e. the profiles) were clinically tested during 3 years in different settings and proved to be quite accurate.

The NCSI can be used for several clinical purposes. Screening and monitoring. In our centre, every year the patient completes the NCSI during a regular visit. In this way, problems in all four domains of HS are revealed in an early stage.

Decision making. The profile of the PatientProfileChart indicates which type of intervention would be required for this individual patient (e.g. pulmonary nurse, an outpatient or multi-disciplinary pulmonary rehabilitation program). As pointed out above, such decisions are based on the profile of all sub-domains. Additionally, the discussion of the PatientProfileChart with patient and partner elucidates the mechanisms underlying the problems in HS. This provides additional information on which type of intervention is best suited.

Motivational intervention. The NCSI and PatientProfileChart can be used as an intervention to increase the patient’s motivation to adopt adequate health behaviors (e.g. stop smoking, regular exercise) or to enroll in additional treatment (e.g. rehabilitation program). This is simply done by discussing the PatientProfileChart with the patient and his partner. The motivational effect is achieved by several psychological mechanisms ‘hidden’ in this procedure. The most important are firstly, results are presented graphically, which has much greater impact than words, and thereby powerfully increases awareness of the severity of his problems. Secondly, the profile is the resultant of responses of the patient himself and does not reflect the opinion of the clinician. This increases commitment and avoids conflicting opinions. The NCSI can also be used for outcome assessment and research purposes.

In conclusion, in this study we composed a battery of instruments that enables the clinician to obtain a valid, reliable, and detailed picture of a patient’s HS by measuring multiple sub-domains covering all four main domains. In combination with the TestOrganiser and the PatientProfileChart, the NCSI can easily be used in routine care as a guide in patient-tailored treatment.