Abstract
This study focused on repeatability data and minimal important difference (MID) estimates of the endurance shuttle walking test (ESWT).
255 chronic obstructive pulmonary disease patients (forced expiratory volume in 1 s 54.7±13.2% predicted) completed four ESWTs at different times during the 8-week study: two under baseline conditions with tiotropium (1 week apart), one after a single dose and one after 4 weeks of either fluticasone propionate/salmeterol combination or placebo in addition to tiotropium. 97 patients performed all the tests with a portable metabolic system. Reproducibility of test performance and cardiorespiratory response was investigated with the data obtained on the first two ESWTs.
The mean differences between the first two ESWT performances (-6.7±72.2 s and -7.3±113.1 m for endurance time and walking distance, respectively) were not statistically significant. The between-test end-exercise and isotime values for each cardiorespiratory parameter were not significantly different from each other. With the exception of arterial oxygen saturation by pulse oximetry, the repeatability of cardiorespiratory adaptations to ESWT was also confirmed with strong Pearson and intraclass correlation coefficients. Finally, changes of 56–61 s and 70–82 m in endurance time and distance walked, respectively, were perceived by patients.
This study provides methodological information supporting the reliability of the ESWT and suggests MID estimates for this test.
Abstract
This study provides a prospective validation of ESWT reliability and minimal important difference in COPD http://ow.ly/zuLew
Introduction
Exercise intolerance is a hallmark feature of chronic obstructive pulmonary disease (COPD), and leads to a sedentary lifestyle, poor quality of life and even reduced survival [1, 2]. As such, measuring the endurance to constant load exercise has been an important objective of several clinical trials evaluating the impact of pharmacological or rehabilitative interventions in COPD [3–5]. In this context, the endurance shuttle walking test (ESWT) is gaining acceptance due to its minimal requirement for equipment, low cost and ease of administration [6], and to its responsiveness and sensitivity to interventions, at least in single centre studies [7–12]. With the advancement of portable cardiopulmonary monitoring, it is also possible to obtain a thorough physiological assessment during the ESWT. However, some methodological aspects of the ESWT deserve further evaluation. The reproducibility of the ESWT time has been reported in only one study [12], involving a limited sample size and without measurement of cardiopulmonary variables during the test. Another issue is the interpretation of changes in ESWT performance after an intervention. In a prior study from our group, estimates of the minimal important difference (MID) from the patients’ perspective were provided for changes in ESWT performance [13]. In this prior study, it was determined that a change in ESWT performance of 45–85 s or 60–115 m after bronchodilation could be perceived by patients (n=69). Prospective validation of these estimates in a larger number of patients seems warranted given the widespread use of MID values in clinical and administrative decision-making processes.
Therefore, the aims of this prospective study were two-fold: 1) to evaluate the reproducibility of the ESWT performance and of the cardiorespiratory variables during the test; and 2) to prospectively validate MID estimates of ESWT performance in patients with COPD.
Subjects and methods
Ethics statement
The study was conducted in 24 sites, in the USA and Canada, between July 2010 and May 2011. The study was approved by all the local ethics review committees and was conducted in accordance with the Declaration of Helsinki and Good Clinical Practice guidelines.
Subjects
Patients with stable COPD agreed to participate and signed a written informed consent form. Inclusion criteria were as follow: age ≥40 years; a clinical history of COPD; post-bronchodilator forced expiratory volume in 1 s (FEV1) 30–80% predicted; FEV1/forced vital capacity ratio ≤0.70; and a current or past history of ≥10 pack-years of cigarette smoking. Exclusion criteria included: diagnosis of asthma; an exacerbation requiring antibiotics and/or systemic corticosteroids within the 6 weeks preceding inclusion into the study; a body mass index ≥40 kg·m−2; and any clinically significant abnormal ECG.
Study design
This study represents a pre-specified secondary analysis of an 8-week multicentre, placebo-controlled, double blind, randomised, parallel-group study (clinicaltrials.gov identification number NCT01124422) investigating whether the addition of fluticasone/salmeterol fixed-dose combination (FSC) to open label tiotropium further improved walking capacity in comparison to tiotropium alone [5]. figure 1 provides a schematic representation of study design.
Study participants completed a 4-week run-in period of open-label tiotropium (Spiriva; Boehringer Ingelheim GmbH, Ingelheim, Germany). The incremental shuttle walking test (ISWT) was first completed without prior practice. After ISWT completion, patients were familiarised with the ESWT procedures and were asked to walk on the same course for a few minutes at constant speed. After at least 2 weeks of tiotropium therapy, two ESWTs (ESWT 1 and ESWT 2) were performed under baseline condition. A third and fourth ESWT (ESWT 3 and ESWT 4) were performed after the first dose and after 4 weeks of study medications, respectively (FSC 250/50 μg administered via Advair/Seretide DISKUS twice daily (GlaxoSmithKline, Research Triangle Park, NC, USA) or placebo DISKUS twice daily). These additional treatments were administered under a background of open-label tiotropium. As per protocol design, patients with exercise duration >20 min and those in whom ESWT 1 and ESWT 2 exercise duration was >2 min apart were excluded from further study procedures in the original clinical trial [5].
Endurance shuttle walking test
All subjects initially performed an ISWT, as originally described by Singh et al. [14], in order to determine their own maximal walking speed and distance. From the data obtained with the ISWT, peak oxygen uptake (V′O2peak) was estimated and the walking speed corresponding to 85% of estimated V′O2peak was determined in order to define the walking speed for the ESWT, according to the procedure described by Revill et al. [12]. The ESWTs were performed on the same course as the ISWT (10 m course in a flat corridor), in accordance with published guidelines [12]. After a 90-s warm-up period, walking speed increased at the speed corresponding to 85% of estimated V′O2peak and subjects were instructed to walk as long as they could at the pace dictated by the auditory signal. The endurance time was defined as the duration of the ESWT and the distance covered during the test was recorded, excluding the warm-up period. With the objective of standardising exercise testing procedures across participating centres, investigators and research personnel had the opportunity to familiarise themselves with the ESWT by performing a few practice tests under the supervision of an exercise physiologist during the investigator meeting. Research teams were also provided with an instruction manual and the required materials covering the methodological aspects of the ESWT.
Subjective rating of change
After each ESWT and before any feedback was given to the subject about the test result, subjects rated their performance of the day in comparison to the previous one using a 7-point Likert scale. The scale ranged from -3 to +3 and included the following ratings: -3 (large deterioration); -2 (moderate deterioration); -1 (slight deterioration); 0 (no change); +1 (slight improvement); +2 (moderate improvement); and +3 (large improvement). Subjects were asked to answer the following question: “In comparison with your previous test, how would you rate your performance on today’s test using the present scale?”
Physiological measurements
In a subset of participating centres (eight out of 24) and subjects (n=97, 35% of the total study population), physiological measurements were collected during the ESWT, using a portable gas exchange analyser (Oxycon Mobil;, Viasys Healthcare, Care Fusion, Germany). Subjects wore a facemask, which allowed measurements of V′O2, carbon dioxide output (V′CO2), minute ventilation (V′E) and respiratory exchange ratio (RER) on a breath-by-breath basis. The device was calibrated before each test according to manufacturer’s recommended procedures. Arterial oxygen saturation measured by pulse oximetry (SpO2) was monitored during all tests using a finger oximeter. A 12-lead electrocardiogram (Cardiosoft; Corina, Milwaukee, WI, USA) was used to monitor cardiac activity and heart rate throughout each test.
Statistical analysis
The analyses for this study were pre-specified at the beginning of the study. All variables are expressed as mean±sd, unless otherwise stated. For all analyses, the level of significance was set at p=0.05. The reproducibility of the ESWT performance (time and distance) was evaluated by comparing values obtained at ESWT 1 and 2 using a paired t-test. The presence of a systematic bias and the agreement between these two ESWTs was assessed using a Bland–Altman plot [15], and the relationship between exercise performance at ESWTs was evaluated by the Pearson product moment correlation. The analysis of ESWT performance reproducibility was completed with the determination of intraclass correlation coefficient (ICC), a valuable way to determine the degree of association and agreement between variables [16]. End-exercise and submaximal V′O2, V′CO2, V′E, heart rate, SpO2 and RER values were compared using paired t-tests. Time courses of each parameter during ESWTs were compared using two-way repeated measure ANOVA, with “visit” and “isotime” as factors. Isotime was defined as exercise time-points corresponding to 25%, 50%, 75% and 100% of the latest exercise time reached on both ESWTs. When the ANOVA F-ratio was significant, post hoc analysis was then conducted with a Tukey test to perform pairwise multiple comparison procedures. Test–retest reproducibility of cardiorespiratory adaptations to ESWT was also assessed using the ICC and within-subject coefficient of variation.
To provide MID estimates for ESWT time and distance, data from the four ESWTs were used. MID values were estimated using both distribution- and anchor-based approaches, similarly to our previous investigation [13]. For the distribution-based approach, half the sd of the change in ESWT performance provided the MID estimates [17]. In the anchor-based approach, differences in performance between two ESWTs were calculated for consecutive tests and for each patient. Relationships between objective and subjective changes in ESWT performance were established using Pearson product moment correlations. The coefficient correlations between the anchor and target needed to be ≥0.3 for the regression to be valid [18]. If this condition was fulfilled, the association between the subjective rating of change and the corresponding difference in performance was plotted. The resulting slopes, which represented the change needed in walking time or distance for participants’ ratings to move one unit on the Likert perception scale, were considered as estimates of the MID.
All the data were analysed using SPSS (version 21.0; SPSS Inc., Chicago, IL, USA) and GraphPad Prism 6 (GraphPad Software Inc., La Jolla, CA, USA).
Results
Anthropometric and pulmonary data
One patient had an ESWT duration >20 min and was excluded from further study procedures. Because of this, it was not possible to obtain reproducibility and MID data in this individual. Nine patients were excluded from the treatment phase of the original clinical trial because their two baseline ESWTs did not meet the a priori defined reproducibility criteria for the trial (endurance time <2 min apart). 12 other patients performed both baseline ESWTs but were not included in the treatment phase due to other reasons. These 21 patients contributed to the ESWT reproducibility analysis but not to the MID analysis. Lastly, 255 patients were randomised into the treatment phase of the study and could contribute to both aspects of the present report.
As shown in table 1, the characteristics of the entire population (n=276) and the subgroup of subjects in whom physiological measurements were obtained during the ESWTs (n=97) were similar. On average, subjects exhibited moderate-to-severe airflow obstruction.
Reproducibility of ESWT performance
Mean values for ESWT time and distance obtained at ESWT 1 and ESWT 2 are reported in table 2. These ESWT performance parameters were not statistically different between the two tests.
The reproducibility of ESWT performance was also confirmed with Pearson correlations and the Bland–Altman plot (fig. 2). Strong linear relationships were found for endurance times and distances obtained at ESWT 1 and ESWT 2, with correlation coefficients of 0.92 and 0.90, respectively. The graphical interpretation of the Bland–Altman plot further confirms the reproducibility of ESWT performance, with a mean bias (95% CI) of -6.7 s (-148.3–134.8 s) and -7.3 m (-228.9–214.3 m) for endurance time and distance, respectively (fig. 2). These small biases represent less than one shuttle at the fastest walking cadence. The reproducibility of ESWT performance was also assessed using ICC, with an ICC (95% CI) of 0.96 (0.95–0.97) and 0.95 (0.94–0.96) for endurance time and endurance distance, respectively.
Reproducibility of metabolic and cardiorespiratory parameters measured during ESWT
Mean end-exercise values for metabolic and cardiorespiratory parameters were not significantly different between ESWT 1 and ESWT 2 (table 3).
Apart from SpO2, end-exercise values reached during both ESWTs showed strong Pearson correlations coefficients (table 3). The time course of each physiological parameter during ESWT 1 and ESWT 2 was practically identical (fig. 3).
Similarly, ANOVA results show no significant interaction of “visit” and “isotime” factors for the physiological parameters, which means there was no significant isotime difference for these parameters between both ESWTs. ICC and coefficients of variation are provided in tables 4] and 5]. With the exception of SpO2, the reproducibility of all the other cardiorespiratory parameters was excellent at different percentages of isotime or at the maximal level.
MID values determination
Using the distribution-based approach, our data suggest MID values of 61 s or 82 m. The correlation coefficients between the anchor (the subjective rating of change) and the changes in ESWT performance were 0.48 and 0.45 for the ESWT time and distance, respectively (fig. 4). The slopes of these relationships suggested that a change in walking time of 56.3 s (95% CI 48.9–63.6 s) and a change in walking distance of 69.6 m (95% CI 59.5–79.6 m) should be perceived by subjects. These changes represent 15.7% (95% CI 13.2–18.1%) and 14.8% (95% CI 12.6–17.1%) of the baseline value, respectively.
Discussion
The present results indicate high reproducibility of ESWT time and distance in the context of a multicentre clinical trial. In addition, cardiorespiratory parameters obtained during the ESWT were highly reproducible, with the exception of SpO2. Finally, we found that changes in ESWT time and distance of 56–61 s and 70–82 m, respectively, were likely to be perceived by subjects and could therefore be considered clinically meaningful.
Reproducibility of ESWT performance
The use of the ESWT for the evaluation of patients with COPD has become increasingly popular in research and clinical fields [8, 9, 13]. However, the methodological validation of the ESWT is limited. Before the present investigation, the reproducibility of the ESWT was documented only once, in a sample of 11 subjects in the original development study by Revill et al. [12]. Although this paper concluded on the good reproducibility of ESWT performance, the precision of the limits of agreement remained questionable in light of the small sample size, as highlighted by the authors themselves [12]. Accordingly, our study provides an important contribution by highlighting the absence of a significant difference in mean endurance time and distance between two ESWT performed 1 week apart, under stable conditions. The graphical analyses of the Bland–Altman plot confirm the absence of systematic differences across the test performance spectrum (i.e. from low to high performance) and ICC also confirm the strong association between the performances achieved on both ESWTs. These results are in accordance with the conclusions of Revill et al. [12] and thus add weight to these findings. In addition, the reproducibility and limits of agreement of the ESWT over time can now be interpreted with more confidence given the multicentre nature and large sample size of our study and the fact that both tests were conducted 1 week apart.
Nevertheless, an intra-individual variability was observed in ESWT performance, as indicated by the sd of the mean ESWT performance differences between ESWT 1 and ESWT 2 (table 2) and by the limits of agreement of the differences in ESWT performance (fig. 2). The randomness of this variability suggests that it was not related to a learning effect, as confirmed by the Bland–Altman plot. The issue about intra-individual variability in exercise performance was previously discussed for the ESWT [12, 19–21], but also for constant work-rate cycling exercise [22]. Revill et al. [21] reported greater variability in ESWT performance with longer endurance time. One suggested explanation for this observation was the possibility that longer walking times were more sensitive to environmental factors and other influences (motivation, mood, etc.) than shorter tests [21]. In our study, examinations of the Bland–Altman plot (fig. 2) did not reveal such influence of walking duration on the between-test variability in ESWT performance. Day-to-day variations in airway function, differences in medication use, subject encouragement, prior pre-test activity or differences in time-of-day of the assessments were some factors proposed to explain performance variability in constant work-rate cycle exercise tests [22]. Finally, there was a small subgroup of patients (21 patients corresponding to 8% of the total sample size) that had not been included in the treatment phase. Among these 21 patients, nine were unable to perform reproducible ESWTs defined as a <2-min difference between the two baseline tests. Although the inclusion of these individuals did not alter our conclusion regarding the reproducibility of the ESWT, we would recommend excluding them from randomisation in the treatment period of clinical trials aiming at evaluating the impact of therapies on exercise endurance.
Repeatability of cardiorespiratory adaptations to ESWT
Peak values and time course of cardiorespiratory and metabolic parameters were compared between the first two ESWTs performed in 97 out of 255 subjects. No significant difference between mean values was found for any parameter supporting good repeatability of cardiopulmonary physiological variables during ESWT. This contention was further supported by Pearson correlation coefficients, ICC and coefficients of variation values. ICC values reflect the within-subject variance, referenced to as the degree of between-subject variance [23, 24]. It is generally assumed that an ICC near 0 reflects an absence of reliability between measurements and an ICC of 1 indicates a perfect reliability, with a cut-off value of ≥0.75 for the lower limit of the 95% confidence interval for defining a very high level of repeatability [16, 24]. With the exception of oxygen saturation, the majority of ICC values found in our study, presented a lower limit of 95% confidence interval >0.75. To the best of our knowledge, no study has documented the repeatability of cardiorespiratory adaptations in two consecutive ESWTs. O’Donnell et al. [22] reported on the reproducibility of cardiorespiratory parameters during constant work rate cycling exercise. Even if the methodology of ESWT and constant work rate tests are different, both tests are defined as endurance tests; interestingly, the ICC and coefficients of variation observed in our study were similar to the values reported in the study by O’Donnell et al. [22].
Determination of MID values for ESWT
The determination of MID values for ESWT performances was based on two different methods: the distribution-based and the anchor-based approaches. As previously suggested [13, 25, 26], different methods should be combined for the determination of MID values, considering that there is no consensus in favour of a single best methodology for the determination of MID estimate and that this estimate should be viewed as a range of probable values rather than as a discrete value.
In the present study, the distribution-based and anchor-based approaches led to similar MID values, which is reassuring regarding the validity of our estimates. In addition, these values are in accordance with previously suggested MID values obtained in patients with COPD [13]. The present estimates were prospectively determined in a larger sample size, thereby adding weight to previous findings and allowing for better precision of these estimated with reduced 95% CI [13].
Nonetheless, these results need to be interpreted by taking into account the modest relationships between changes in ESWT performance and perception of ESWT change. In fact, one important requirement for anchor-based approach interpretability is that the association between the target (change of ESWT performance) and the anchor (perception of performance change) should be appreciable [27]. In this context, some authors have suggested that correlation coefficients of the relationship should be ≥0.5 for interpreting anchor-based approaches results satisfactorily. In the case of correlation coefficients between 0.3 and 0.5, the analyses can be performed but the interpretation of the results and estimates requires some caution [26, 28]. With correlation coefficients amounting to 0.48 and 0.45 for endurance time and endurance distance, respectively, and considering the present estimates are very similar to what was previously reported, we are confident about the validity of the proposed estimates.
Our MID results were obtained in the context of a pharmacological intervention and as such may not be applicable to other types of interventions [13], although it has been argued that the same MID values should be applied irrespective of the intervention [29]. All patients involved in this clinical trial had a background therapy of tiotropium and whether this may have influenced the MID results is uncertain. The presence of a baseline potent inhaled therapy probably reduced the increase in endurance time seen with the addition of a second inhaled therapy, a situation that could have decreased the strength of the association between changes in endurance time and changes in patients’ perception. Despite this, the wide range of changes in endurance time seen in the trial (fig. 4) suggests that the experimental conditions were adequate to provide valid MID estimates. As in our initial publication on this topic [13], with our anchor we have defined a threshold of change in ESWT that could be perceived by patients. In addition, we refer to this threshold as the MID [30]. The use of additional anchors (e.g. dyspnoea or quality of life questionnaires) would have been useful to ascertain the clinical importance of the observed changes.
One interesting methodological consideration is that patients involved in our trial performed only one ISWT. This study procedure is at variance with the suggestion that two ISWTs should be performed to account for a small but significant learning effect seen with this test [31–37]. This proposed methodology is likely to be important to accurately determine ISWT performance. However, our data would support the idea that performing two ISWTs may not be essential when the purpose of the test is simply to determine the walking speed to be used during the ESWT, as indicated by the good reproducibility of the ESWT coupled with the fact that only one patient reached the maximum 20-min ESWT duration.
In conclusion, this study provides prospective methodological validation of the ESWT by confirming good reproducibility of ESWT performance and by demonstrating the high repeatability of cardiorespiratory measurement under stable condition. With MID values ranging from 56 to 61 s and 70 to 82 m for endurance time and endurance distance, respectively, our study also provides additional information for the clinical interpretation of changes in ESWT performance after a therapeutic intervention. Thus, the results of this study highlight the fact that the ESWT can be considered as a valid alternative for the evaluation of exercise tolerance in COPD, whether in a clinical setting or in the context of scientific research.
Acknowledgments
We would like to thank Glenn D. Carter, Andrea M. Morris, Amanda H. Emmett and Tom J. Ferro (GlaxoSmithKline, Research Triangle Park, NC, USA), and all the research staff for their commitment and involvement in this study. We also want to thank the study subjects for their participation in this project.
Footnotes
Clinical trial: This study is registered at clinicaltrials.gov (identification number NCT01124422).
Conflict of interest: Disclosures can be found alongside the online version of this article at erj.ersjournals.com
- Received February 5, 2014.
- Accepted June 26, 2014.
- ©ERS 2014