Article Text
PDF
Abstract
Objective The ‘single-item measure’ was developed as a short self-report tool for assessing physical activity. The aim of this study was to test the criterion validity of the single-item measure against accelerometry.
Design Participants (n=66, 65% female, age: 39±11 years) wore an accelerometer (ActiGraph GT3X) over a 7-day period and on day 8, completed the single-item measure. The number of days of ≥30 min of accelerometer-determined moderate to vigorous intensity physical activity (MVPA) were calculated using two approaches; first by including all minutes of MVPA and second by including only MVPA accumulated in bouts of ≥10 min (counts/min ≥1952). Associations between the single-item measure and accelerometer were examined using Spearman correlations and 95% limits of agreement. Percent agreement and κ statistic were used to assess agreement between the tools in classifying participants as sufficiently/insufficiently active.
Results Correlations between the number of days of ≥30 min MVPA recorded by the single-item and accelerometer ranged from 0.46 to 0.57. Participants underreported their activity on the single-item measure (−1.59 days) when compared with all objectively measured MVPA, but stronger congruence was observed when compared with MVPA accumulated in bouts of ≥10 min (0.38 days). Overall agreement between the single-item and accelerometry in classifying participants as sufficiently/insufficiently active was 58% (k=0.23, 95% CI 0.05 to 0.41) when including all MVPA and 76% (k=0.39, 95% CI 0.14 to 0.64) when including activity undertaken in bouts of ≥10 min.
Conclusions The single-item measure is a valid screening tool to determine whether respondents are sufficiently active to benefit their health.
Statistics from Altmetric.com
Background
Reliable and valid measurement of physical activity is important for understanding population prevalence of physical activity, determining the extent to which physical activity interventions are reaching inactive groups and assessing the impact of interventions on participants' physical activity levels.1 Self-report questionnaires are a cheap and simple method of collecting data from large groups and are therefore a frequently used approach to physical activity measurement.2 Many self-report physical activity tools are available, which vary in the type of data which are collected and the way in which physical activity is classified.3 Selecting an appropriate data collection tool is a key priority for policy makers and practitioners.1
Self-report tools typically capture details on the frequency (days), intensity (moderate/vigorous) and duration (min) of physical activity and are therefore relatively long. These tools can be time consuming and burdensome for respondents, rendering them inappropriate for use in interventions, particularly in field-based contexts when time and resources are often limited. In order to collect data from a broad range of settings (for example, healthcare, worksites and community settings) and across a range of interventions (such as led walk schemes and ‘active travel’ initiatives), there is a need for shorter measurement tools which are practical and easy to complete.
A recent review identified 14 short or ‘single-item’ physical activity questionnaires.4 These tools have the advantage of being relatively short and quick to complete, however the type of data which these tools provide means that they are often not appropriate for intervention research. For example, one approach has been to ask respondents whether they regularly participate in physical activity, using a binary (yes/no) response scale. Another approach asks respondents to consider whether they are more or less active than their peers. Because these tools do not provide a quantification of the amount of physical activity which respondents undertake, it is not possible to determine baseline activity levels or to measure changes in physical activity over time. Several short tools are available which provide a quantification of physical activity in terms of days and time spent in moderate to vigorous intensity physical activity (MVPA), however these tools typically include two or three questions.
The single-item measure was developed to provide an even shorter and therefore more pragmatic tool for the assessment of physical activity.4 This tool was designed to capture an assessment of physical activity against the national recommendation of at least 30 min of MVPA on 5 or more days of the week,5 using one question only. This is achieved by asking about the number of days in which respondents have done ≥30 min of MVPA. The single-item measure has demonstrated strong repeatability and moderately strong validity against other self-report tools.4 However, further testing using objective measurement was needed to determine the ability of the tool to accurately assess ‘true’ physical activity levels. The aim of the current study therefore was to test the criterion validity of the single-item measure using accelerometry.
Methods
Sample
A standard e-mail was sent to students and staff at Loughborough University, inviting participation in the study. In addition, posters and ‘word of mouth’ were used to advertise the study. In total, 133 people expressed an interest in taking part. Respondents consisted of staff and students at Loughborough University as well as residents from the local community.
Procedure
All 133 people who expressed an interest in the study were invited to attend a one-to-one consultation with a member of the research team (KM). All respondents attended this consultation and were recruited into the study. During the baseline consultation, participants completed a short questionnaire, including contact details and demographic characteristics (age and gender), and were instructed on how to wear the accelerometer (ActiGraph GT3X).
Accelerometers were initialised to collect data in 1-min epochs. Participants were instructed to wear the accelerometer during waking hours, except when in water, for a 7-day period while maintaining their usual daily routine. Consultations lasted approximately 10 min.
On day 8, participants attended a second consultation, during which they returned the accelerometer and completed the ‘past week’ version of the single-item measure via self-administration. The wording of the single-item measure is as follows: ‘In the past week, on how many days have you done a total of 30 min or more of physical activity, which was enough to raise your breathing rate? This may include sport, exercise and brisk walking or cycling for recreation or to get to and from places, but should not include housework or physical activity that may be part of your job’.
The study was approved by the Loughborough University Ethical Advisory Committee and all participants provided written informed consent. Data collection took place between December 2010 and March 2011.
Data analyses
Accelerometer data were cleaned for periods when the monitor was not worn, by excluding consecutive strings of zero-count epochs lasting 20 min or more. Participants were excluded from the analyses if they did not meet the wear time validation criteria of at least 10 h per day on all 7 days.6 ,7 An independent-samples t-test and χ2 test were used to determine whether any differences existed between included and excluded participants in terms of age and gender, respectively.
Using established criteria, accelerometer-determined activity registered as ≥1952 counts/min was considered moderate intensity.8 For this study, all activity undertaken at this level of intensity or higher was classified as MVPA. Independent-samples t-tests were used to determine whether any differences existed between both self-reported and objectively measured physical activity of the included and excluded participants. Due to missing objective data from the excluded participants, mean minutes per day spent in MVPA were calculated from the available days and compared with mean daily minutes of MVPA recorded from the included participants.
The number of days that participants achieved at least 30 min of MVPA, according to the accelerometer, was calculated using two different approaches. First, the total time spent in MVPA each day was calculated regardless of bout duration by summing the number of minutes in which accelerometer counts exceeded 1952 counts/min. Second, the number of days of MVPA totalling ≥30 min was calculated, including activity occurring in sustained bouts of ≥10 min only, reflecting the government recommendation that bouts of at least 10 min may be necessary to benefit health.5 These two approaches are referred to as ‘all MVPA’ and ‘recommended MVPA’, respectively, throughout the paper.
The Shapiro-Wilk test of normality was used to determine the distribution of both the self-report and objective physical activity data. As the data were not normally distributed, Spearman rank correlation coefficients were used to assess associations between responses to the single-item measure and the number of days of ≥30 min of MVPA determined by accelerometry. Bland-Altman 95% limits of agreement were used to calculate the level of agreement between the two tools in terms of the number of days that participants achieved ≥30 min of MVPA.9 In addition, accelerometer data were used to classify participants based on whether or not they were meeting the physical activity recommendation of at least 30 min of MVPA on 5 or more days of the week.5 Percent agreement and κ statistic were used to determine the level of agreement between the single-item measure and accelerometer on this classification. In this study, ‘specificity’ is used to describe sufficiently active participants who were correctly identified by the single-item measure and ‘sensitivity’ is used to describe those who were correctly classified as insufficiently active by the self-report tool.
Results
Testing was completed with 133 healthy volunteers, aged between 19 and 64 years. In total, 66 participants (50%) met the wear-time validation criteria and were included in the analyses. The final sample consisted of 23 (35%) males and 43 (65%) females, aged between 21 and 62 years (mean±SD=39±11 years). The 66 participants in the final sample did not differ from the excluded participants in terms of age (t(130)=−0.339, p=0.735) or gender distribution (χ2(1)=0.993, p=0.319). Similarly, the included and excluded participants did not differ significantly in terms of the number of days that they reported spending ≥30 min in MVPA on the single-item measure (t(131)=0.891, p=0.374), or in terms of their mean daily minutes spent in MVPA assessed by the accelerometer (t(131)=−0.686, p=0.494). Table 1 shows the frequency data for days of ≥30 min of MVPA for the final sample.
Frequency of days of ≥30 min moderate to vigorous intensity physical activity (MVPA) as measured via self-report and accelerometry
When including all objectively measured MVPA in the analyses, there was a significant correlation (r=0.46, p<0.001) between responses on the single-item measure and the accelerometer in terms of the number of days that participants reported achieving ≥30 min of MVPA. The mean (SD) difference between the number of days of ≥30 min of MVPA between the single-item measure and accelerometry was −1.59 (2.08). The 95% limits of agreement were −2.49 to 5.67 days which represents the 95% likely range for the differences in scores between the two tools (figure 1). Summing all minutes of objectively measured MVPA, 40 participants (61%) were classified as achieving the national physical activity recommendation of at least 30 min of MVPA on 5 or more days.5 Overall agreement between the single-item measure and accelerometer on this classification was 58% with a κ statistic of 0.23 (95% CI 0.05 to 0.41). As shown in table 2, the single-item measure correctly identified 88% of participants who were meeting the physical activity recommendation (specificity) and 48% of participants who were not achieving this recommended level (sensitivity).
Difference in days of ≥30 min moderate to vigorous intensity physical activity (MVPA) between the single item measure and accelerometry: all MVPA.
Number (%) of participants classified as sufficiently or insufficiently active as measured by the single-item measure and accelerometry – All MVPA
When only including objectively measured MVPA undertaken in sustained bouts of ≥10 min in the analyses, the correlation between the single-item measure and accelerometer for the number of days that participants reported achieving ≥30 min of MVPA was r=0.57 (p<0.001). The mean (SD) difference between the number of days of ≥30 min of MVPA as measured by the two tools was 0.38 (1.97) and the 95% limits of agreement ranged from −4.23 to 3.48 days (figure 2). When including objectively measured MVPA accumulated in bouts of ≥10 min, just 18 participants were classified as achieving the national physical activity recommendation.5 The single-item measure demonstrated 56% for specificity and 83% for sensitivity, giving an overall agreement of 76% and κ statistic of 0.39 (95% CI 0.14 to 0.64) (table 3).
Difference in days of ≥30 min moderate to vigorous intensity physical activity (MVPA) between the single-item measure and accelerometry: recommended MVPA.
Number (%) of participants classified as sufficiently or insufficiently active as measured by the single-item measure and accelerometry – recommended MVPA
Discussion
Self-report tools remain a popular choice for the assessment of physical activity due to their low cost and ease of use. It is important to ensure that self-report tools undergo rigorous validity testing to ensure that they accurately assess physical activity behaviour. The aim of the current study was to test the criterion validity of a new ‘single-item’ self-report physical activity measure against accelerometry.
In total, 66 participants met the accelerometer wear-time validation criteria of at least 10 h per day on all 7 days, representing 50% of the total sample. Although previous studies have set accelerometer wear-time criteria at ≥10 h per day, participants providing between 3 and 5 days of accelerometer data have often been classified as compliant, and thus included in the analyses.6 ,7 ,10 However, because the current study aimed to assess agreement between a 7-day self-report tool and objectively measured physical activity over the same time period, a full 7 days of accelerometer data were essential for the validation. In a previous study which required 7 days of accelerometer data, 56% of participants met this criterion, suggesting that the level of compliance in the current study is not dissimilar to what has been observed previously.11
Comparison between the number of days of ≥30 min of MVPA recorded by the single-item measure and by accelerometry produced correlation coefficients of 0.46–0.57. In a review of short and single-item self-report physical activity measures, two tools were identified which have been tested against accelerometry.4 Correlations ranged from 0.31 to 0.39,12 suggesting that the single-item measure correlates better with accelerometer data than other previously validated short tools. Previous tools have assessed minutes of moderate and vigorous intensity activity, whereas the single-item measure assesses ‘days of ≥30 min of MVPA’. The more discrete assessment of ‘days’ as opposed to ‘minutes’ on the single-item measure may have contributed to the stronger correlations observed.
Another review which included both long and short tools identified 41 studies which have tested self-report tools against accelerometers.3 The results were mixed, although mean correlations were relatively poor (0.32 and 0.22 for vigorous and moderate intensity activity, respectively). However, these tools quantified physical activity in a variety of ways including frequency, minutes and metabolic equivalent. None of the tools included in this review assessed ‘days’ of physical activity, making direct comparisons between the present study and earlier research problematic.
Bland-Altman 95% limits of agreement were used to explore the strength of agreement between the single-item measure and accelerometer data. When including all objectively measured MVPA, participants typically underreported their physical activity on the single-item measure by an average of almost 2 days. The self-report tool excludes housework and occupational physical activity, whereas accelerometry captures all physical activity regardless of domain. Furthermore, accelerometry detects both incidental and purposeful activity, including lifestyle activities such as walking and stair climbing. Participants may not have considered this type of activity when calculating their response to the single-item measure, which may help to explain the underreporting of physical activity when compared with all objectively measured MVPA. When compared with the number of days of 30 min or more of objectively measured MVPA, including activity occurring in sustained bouts of ≥10 min only, agreement between the two tools was stronger. Although the single-item measure does not specify that reported bouts of physical activity should be a minimum of 10 min in duration, it appears that respondents are able to more accurately recall this type of purposeful physical activity when completing the single-item measure.
It is particularly important that physical activity screening tools are capable of identifying whether or not respondents are achieving recommended physical activity levels. Yet, to date, few studies have explored associations between accelerometer and self-report data using recommended activity levels as cut points. Only two studies were found that used this approach, one tested the International Physical Activity Questionnaire13 and the other tested a short screening tool developed for use in primary care.12 Due to differences in the way physical activity is measured using different tools, for example, the inclusion/exclusion of occupational-related activity and differences in the unit of measurement used that is, frequency, duration or both, these studies do not allow for a direct comparison against the single-item measure. However, they may serve as a useful guide for establishing the relative validity of the single-item measure in comparison to previously validated tools.
Percent agreement between the single-item measure and accelerometer on the classification of participants meeting the physical activity guidelines when including all objectively measured MVPA was 58% with a κ value of 0.23 (95% CI 0.05 to 041). When including objectively measured activity undertaken in sustained bouts of ≥10 min, overall agreement between the single-item measure and accelerometry on this classification was 76% with a κ statistic of 0.39 (95% CI 0.14 to 0.64).
The results of the previous studies showed percent agreement values ranging from 59% (for the short screening tool against accelerometry) to 66% (for the IPAQ against accelerometry). κ statistics were available for the short screening tool and ranged from 18.2 (95% CI 3.9 to 32.6) to 24.3 (95% CI 11.6 to 36.9). The κ values in the current study were therefore similar and slightly higher to what has been reported previously. However, it should be noted that κ values ranging from 0.21 to 0.4 are generally considered as ‘fair’,14 and thus the single-item measure, like other self-report tools, may misclassify the activity levels of some respondents.
The IPAQ study reported the specificity and sensitivity of the tool in comparison with accelerometry data. The IPAQ correctly identified 77% of participants who met recommended physical activity levels but correctly identified only 45% of participants who did not meet the guidelines.13 In comparison, our new single-item measure performed well, correctly identifying 83% of respondents who failed to achieve recommended physical activity levels. Therefore, although the κ values are only considered ‘fair’, the sensitivity results suggest that the single-item measure may be a useful screening tool to determine whether respondents are sufficiently active to benefit their health.
A number of limitations associated with this study should be acknowledged. Males were underrepresented, comprising just 35% of the sample. In addition, a volunteer sample was used. It is possible that these self-selected participants were more physically active and/or had greater interest in learning about their physical activity levels than people who found out about the study but did not volunteer. The sampling approach used in the present study may therefore have implications for the generalisability of the results. Further testing, using different population groups across a range of settings, is recommended to confirm the validity of the single-item measure.
Accelerometers are a useful tool for validating the accuracy of self-report instruments, however these devices are not without limitation. For example, accelerometers do not detect arm movements or resistance exercise and cannot be worn during water-based activities such as swimming.15 Consequently, limitations of both the self-report and objective measures are likely to contribute to the discrepancies in physical activity levels observed between the two tools.
Finally, although the single-item measure appears to provide a reasonable indication of the number of days of ≥30 min of MVPA, this tool does not provide details of the total time spent in physical activity or the types of activities undertaken. Consequently, respondents who undertake longer periods of activity on fewer days may be classified as insufficiently active on the single-item measure. These limitations should be taken into consideration when selecting appropriate data-collection tools, given the specific research questions of interest. In addition, further research is needed to test the responsiveness of the single-item measure to detect changes in physical activity over time and thus provide a useful tool to determine the effectiveness of interventions in encouraging physical activity behaviour change.
Conclusions
The single-item measure was developed as a short screening tool designed to assess physical activity against the national recommendation.5 Correlations between the new single-item measure and accelerometry were stronger than previously reported for many other self-report tools. Agreement between the self-report and objective measure was reasonable, however agreement varied depending on whether all minutes of MVPA or only sustained bouts of ≥10 min were included in the analyses. When including activity undertaken in sustained bouts of ≥10 min, consistent with the government recommendation, the single-item measure correctly identified over 80% of insufficiently active participants. These results suggest that the single-item measure is a useful screening tool to determine respondents' appropriateness for entry into an intervention. Further research is needed to determine the responsiveness of the tool to detect changes in physical activity over time.
What this study adds
The ‘single-item measure’ was developed as a short monitoring tool for assessing physical activity levels. Although this tool has demonstrated strong repeatability and moderately strong validity against other self-report tools, further testing was warranted to determine whether the tool accurately assesses ‘true’ physical activity levels. This is the first study to test the criterion validity of the single-item measure compared with accelerometry.
References
Footnotes
-
Contributors KM and FB conceptualised the study and developed the study protocols. KM collected and analysed the data with guidance from SC. KM prepared the draft manuscript. FB and SC assisted in revising the manuscript. All authors read and approved the final manuscript.
-
Funding This research was funded by Natural England.
-
Competing interests None.
-
Ethics approval Loughborough University Ethical Advisory Committee.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
▸ References to this paper are available online at http://bjsm.bmjgroup.com