Article Text
PDF
Abstract
Background Step counters have been used to observe activity and support physical activity, but there is limited evidence on their accuracy.
Objective The purpose was to investigate the step accuracy of the Fitbit Zip (Zip) in healthy adults during treadmill walking and in patients with cardiac disease while hospitalised at home.
Methods Twenty healthy adults aged 39±13.79 (mean ±SD) wore four Zips while walking on a treadmill at different speeds (1.7–6.1 km/hour), and 24 patients with cardiac disease (age 67±10.03) wore a Zip for 24 hours during hospitalisation and for 4 weeks thereafter at home. A Shimmer3 device was used as a criterion standard.
Results At a treadmill speed of 3.6 km/hour, the relative error (±SD) for the Zips on the upper body was −0.02±0.67 on the right side and −0.09 (0.67) on the left side. For the Zips on the waist, this was 0.08±0.71 for the right side and -0.08 (0.47) on the left side. At a treadmill speed of 3.6 km/hour and higher, the average per cent of relative error was <3%. The 24-hour test for the hospitalised patients showed a relative error of −47.15±24.11 (interclass correlation coefficient (ICC): 0.60), and for the 24-hour test at home, the relative error was −27.51±28.78 (ICC: 0.87). Thus, none of the 24-hour tests had less than the expected 20% error. In time periods of evident walking during the 24 h test, the Zip had an average per cent relative error of <3% at 3.6 km/hour and higher speeds.
Conclusions A speed of 3.6 km/hour or higher is required to expect acceptable accuracy in step measurement using a Zip, on a treadmill and in real life. Inaccuracies are directly related to slow speeds, which might be a problem for patients with cardiac disease who walk at a slow pace.
- Activity monitor
- Step counter
- validation study
- Accuracy
- Exercise
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
The speed test on the treadmill did provide strength to the study, as the speed inaccuracy of the treadmill may hamper test results.
Treadmill studies are important because they may isolate the step accuracy at different speeds.
Treadmill walking is not representative of normal walking, and test persons on a treadmill may produce a non-natural rigid walking gait that might affect accuracy.
The study validated Fitbit Zip (Zips) placed only on the waist and upper body; therefore, the reliability of the Zip at other placements is still unknown.
Introduction
Physical activity is associated with numerous physical and mental health benefits including delayed all-cause mortality, reduced risk of diabetes, cardiovascular diseases and some forms of cancer, lower blood pressure, improved lipoprotein profiles and enhanced insulin sensitivity.1 Furthermore, a physically active lifestyle enhances feelings of energy, well-being and quality of life.1 Physical activity is vital for recovery after cardiac disease, and there is strong evidence for reduced morbidity and mortality when cardiac rehabilitation is exercise based.2–5 Walking is a simple physical activity that improves cardiovascular capability.6–9 Activity monitors, such as step counters, are designed to monitor walking activity, and step counters have been used to observe activity and as a motivational tool to support physical activity in clinical settings.7 ,10–14 The cautious use of step counters in research and clinical settings has been due to the lack of evidence regarding their reliability and validity. In general, accuracy studies have reported counting errors in the step readings of such counters, particularly in cases of low walking speed,15–17 high body mass index18 and individuals with gait disorders and functional limitations.13 ,19 ,20 Step counter accuracy has often been studied in healthy adults and under controlled conditions, for example, on a treadmill.15 ,17 ,20–23 However, treadmill walking is not representative of normal walking. The advantages of treadmill tests are the possibility of determining inaccuracies at different walking speeds, durations, cadences and distances. New step counters on the market are expected to count steps accurately. In particular, Fitbit step counters have received considerable attention.15 ,17 ,20 ,23–28 One of Fitbit's step counters the Fitbit Zip (Zip) is recommend as a valid step counter in measuring free-living physical activity in healthy adults, and Ferguson et al27 found high step accuracy in 24-hour real-life studies of healthy subjects.26 Furthermore, Beevi et al15 has shown high step error rates for the Zip at slow speeds (1–3 km/hour). However, no studies have examined the step accuracy of the Zip on a treadmill at normal speed or in real-life studies of patients with cardiac disease. The purpose of this study was to investigate the accuracy of the Zip step counter in measuring steps taken by healthy adults during treadmill walking (treadmill study). Some studies recommend that an acceptable measurement error under controlled conditions or for research purposes is within ±3%.29–31 Other studies recommend that mean errors of <20% have acceptable validity for clinical purposes.29 ,32
The aim of this study was twofold:
To investigate the accuracy of the Zip step counter in measuring steps taken by healthy adults during treadmill walking (treadmill study). It was hypothesised that a Zip step counter would have a step-count error of ≤3%, compared with a gold standard, when tested by healthy adults on a treadmill.
To use the same motion measurement equipment to investigate the accuracy of the Zip in measuring steps taken by patients with cardiac disease at the hospital and in the patient's home (real-life study). It was hypothesised that a Zip step counter would have a step-count error of ≤20%, compared with a gold standard, when tested by patients in real-life settings.
Methods
Participants
Treadmill study
A sample of 20 healthy participants recruited from November 2014 to January 2015 among voluntary employees at Aalborg University Hospital was included in this study. The inclusion criteria were age ≥18 years, perceived general good health, the ability to walk and run on a treadmill, and the ability to understand verbal and written information. To ensure equal gender distribution, 10 women and 10 men were selected for participation in the study. Subjects who required a walking aid or were pregnant and/or breast feeding were excluded.
Real-life study
From February to August 2015, 24 patients with cardiac disease participated in the real-life study. The inclusion criteria were age ≥18 years, hospitalised with acute coronary syndrome, heart failure (defined by an ejection fraction <40%), coronary artery bypass grafting or valve surgery. Equality between treatment types (surgical or medical) was sought.
Patients were excluded if pregnant, breast feeding or non-Danish speaking and if they had a gait disorder or any other conditions that might affect walking. Thirty-nine potential participants were approached, and 33 subjects agreed to participate. Of these participants, seven withdrew their consent after the first 24 hours of testing due to a lack of interest to complete the study. One participant was discharged from the hospital before completing the test. In total, 24 patients completed the study. The distribution of the participants was 11 patients from the Department of Cardiothoracic Surgery and 13 patients from the Department of Cardiology, Aalborg University Hospital, Denmark.
Procedure
Measures
The Fitbit Zip
The Zip (FITBIT, 405 Howard Street San Francisco, CA 94105) is small (35.6×28.9×9.6 mm) and has a user account connected to a computer or mobile application. The user account displays steps, active minutes, distance and energy expenditure over time, providing the user with knowledge of their own activity. The Zip uses a triaxial accelerometer, and raw acceleration signals are converted into indicators of physical activity (eg, steps) using mathematical formulas (not known to the authors, as the formulas are proprietary to the company).
Shimmer3
Shimmer3 (Shimmer) is a wireless sensor platform (Shimmer Research, Dublin, Ireland). The Shimmer was used as the criterion standard for the measurement of steps because it has been shown to be accurate in sensing steps in adults aged 60 and older33 and healthy male adults.34 The Shimmer contains a triaxial gyroscope and accelerometer for motion sensing.
Treadmill
A Daum Eletronic Ergo_run Premium8 treadmill was used. We found no studies that determined the accuracy of the treadmill speed. Thus, we conducted our own speed test by filming the rolling treadmill, with and without the test person walking on the conveyer belt.
Treadmill study
All participants carried four Zips on elastic belts, two Zips at the heart level on the upper body and two Zips at the waist. The upper body and waist step counters were placed on the right and left sides of the body. The Shimmer was fastened onto an elastic belt and placed on the ankle on the lower left leg with the orientation of the gyro Z-axis perpendicular to the sagittal plane.
All participants walked on the treadmill for 3 min at 10 different speeds, increasing from an expected speed of 1.5 to 6 km/hour with an increase of 0.5 km/hour between each test. The 3-min trials were similar to other research protocols.11 ,29 The sample rate was 256 Hz during the treadmill test. There were 2-min rests between each walking test to record the step counts from each step counter.
Real-life study
At the hospital and in the patient homes
The hospitalised patients were asked to wear the Shimmer on their ankle simultaneously with a Zip placed on the waist. Placement of the Zip on the waist and not the upper body was chosen due to the slightly more accurate step recordings revealed from the treadmill study. The patients wore the Zip and Shimmer for 24 hours, except during bathing. The study was performed for 24 hours, at a mean of 4.3 (range 1–10) days after admission/surgery. All participants were asked to keep an activity diary documenting activities, such as sitting, eating, going for a walk and the time without the devices for the 24-hour measurement. The precise numbers of steps at the start and end of the test were noted in the activity diary. Data from the activity diary were analysed together with data from the Zip and the Shimmer. The sample rate was 50 Hz because this rate was considered adequate for the expected walking speeds and it was necessary to have sufficient disc space and battery capacity for 24-hour recordings. This study was repeated in the patients' homes 4 weeks after the hospital test (mean: day 28.2 (range 26–31)). This length of time was considered appropriate as it was expected that patients' walking activity increased during the first weeks of recovery. Furthermore, all devices were tested for battery status. Before the tests, the participants' weight, height, age and sex were uploaded to each of the step counters. Raw Z-gyroscope data from the Shimmer and recorded step per minute and total steps from the Zips were analysed using Matlab 2014b (The MathWorks, Natick, Massachusetts, USA). Steps per minute were used to analyse step patterns throughout the entire day and the total steps per day.
For the treadmill and real-life studies, the participants were asked to walk 20 steps on the ground at a self-paced speed to verify the accuracy of the step counter recordings within a margin of one step. In case of errors, the step counter was replaced until satisfactory readings were achieved.
Ethical considerations
The study followed the principles as outlined in the Declaration of Helsinki. For the treadmill and real-life study, the study was explained in written and oral formats for the participants, and all participants provided written informed consent. The study was presented to the North Denmark Region Committee on Health Research Ethics. The committee responded by stating that the study fell out of the ethical committee's area of concern, as it aimed to study the accuracy of the Zip and not the Zip's influence on a human being. As a result, this study did not require approval.35
Statistical analysis
The sample size calculation was based on a significance level of 0.05 with a 90% power to identify a mean difference of 3% between the treadmill study and the real-life study. Furthermore, a SD calculated from an accuracy study by Park et al28 was used. Thus, 19 participants should have been included, in the treadmill study and the real-life study. All statistical analyses were performed using STATA 13.1 (Stata Corp., College Station, Texas, USA). Descriptive statistics were expressed as the mean ±SD. For the treadmill test, four relative error scores were measured for each participant at each speed, two for the upper body step counters (ErelUpperRight and ErelUpperLeft) and two for the waist step counters (ErelWaistRight and ErelWaistLeft), using the following formula:
Error scores of zero indicated no difference, a positive error score represented an overestimation of the step counters and a negative error score represented an underestimation.29 For the real-life study, the per cent relative error was measured for all 24-hour recordings (time periods of 24 hours (TP24h)). A 24-hour recording will automatically record activities other than regular walking. Thus, time periods of evident walking were determined, and within these, time periods of 3 min (TP3min) were selected for every patient at every 24-hour measurement to directly compare the walking speed and the step accuracy from the treadmill test. To measure the walking speed (numbers of steps × step length (km)/time(h)) for TP3min, a determination of each patients' step length was made using the following formula: 
36 Time periods of evident walking were identified using the diary and by investigating the accumulated steps graphs for each patient (figure 1).
Accumulated step graph for day 1 (surgical patient number 2 at home). Evident walking was found at around 8:00, 12:00 and 16:00. Twelve o'clock was chosen for the TP3min estimation. TP3min, time periods of 3 min.
The per cent relative error was measured for TP3min. For the treadmill and real-life study, the statistical analysis consisted of Blandt and Altman plots and the interclass correlation coefficient (ICC) to assess the level of agreement between the Shimmer and Zips.
Results
Sample characteristics
The descriptive statistics for the sample population for the treadmill study and the real-life study are provided in table 1.
Descriptive characteristics for participants in the treadmill and real-life studies
The speed test of the conveyer belt revealed an average inaccuracy of 0.2 km/hour at speeds of 1.5–3.5 km/hour and 0.1 at speeds of 4–6 km/hour (figure 2). The precise speeds of the treadmill were 1.7–6.1 km/hour.
Actual speed of the treadmill when test persons walk at treadmill-determined speed of 1.7–6.1 km/hour.
Results from the treadmill study are displayed in table 2. The average per cent relative error at 3.6 km/hour for step counters of the upper body was −0.02 (0.67) on the right side and −0.09 (0.67) on the left side. For the waist step counters, it was 0.08 (0.71) on the right side and −0.08 (0.47) on the left side. At 3.6 km/hour and higher speeds, the average per cent relative error was <3% with a high ICC (all >0.9940). Inaccuracy was primarily due to an underestimation for the upper and waist step counters (figure 3 and table 2).
Mean steps (±SD) for Shimmer and Zip, per cent relative error (Erel), SD and intraclass correlation coefficient (ICC) of Zips placed on the upper body (upper body right and upper body left) and waist (waist right and left) at different speeds on a treadmill
Relative error (SD) between Fitbit and Shimmer for step counters placed at the upper body (right (A), left (B)) and at the waist (right (C), left (D)).
Real-life study
The results of the real-life study are displayed in table 3. The per cent relative error (SD) for the TP24h at the hospital was −47.15±24.11 (ICC: 0.60). For the TP24h at home, the per cent relative error was −27.51±28.78 (ICC: 0.87), meaning that inaccuracy was more than the expected 20%. The per cent relative error for TP3min at the hospital was −24.63±29.95 (ICC: 0.79), and at home, the per cent relative error was −11.43±15.51 (ICC: 0.96), meaning that the inaccuracy was less than the expected 20%. The relative errors for TP24h and TP3min are displayed in figure 4, demonstrating that the Zip mainly underestimated the number of steps.
Per cent relative error (Erel), SD and intraclass correlation coefficient (ICC) for the Zip worn by patients at the hospital and at home for time periods of 24 hours (TP24h) and for time periods of 3 min of evident walking (TP3min)
Relative error (SD) between Fitbit and Shimmer for 24 hour (A) and time periods of 3 min (TP3min) (B).
The treadmill study revealed that the inaccuracies were directly related to slow speed. Inaccuracies of the waist step counter averages from the treadmill study are displayed in figure 5 along with the inaccuracies of the TP3min.
Relative errors for Fitbit Waist on treadmill and in time periods of 3 min (TP3min) real life.
This finding indicated that the step accuracy of the Zip in the real-life study was best at high speeds. Similar to the treadmill study, at 3.6 km/hour and higher speeds, the average per cent relative error was <3%.
The Bland–Altman plots revealed no systematic differences between the Shimmer and the Zip at a speed of 6.1 km/hour. Plots from the real-life study revealed that the difference between the two devices appeared to be random, but at 1.7 km/hour, the difference appeared to be systematic. This finding supports the accuracy of results reported in tables 2 and 3 (figure 6A, B).
(A) Bland–Altman plot for Shimmer and Fitbit at different speeds at the treadmill. (B) Bland–Altman plot for Shimmer and Fitbit in real life.
Continued
Discussion
The treadmill study revealed a satisfactory agreement between the Shimmer and the Zip at a speed of 3.6 km/hour and higher. This same high accuracy has previously been found in other studies of Fitbit step counters.16 ,17 For example, Singh et al revealed that Zips worn on the chest were accurate to within 3% when the walking speed was 2.88 km/hour. When worn on the hip, the Zip was accurate to within 4% at a walking a speed of 2.52 km/hour and above.36 According to Tudor-Locke et al,30 activity monitors should not exceed a 1% error deviation from the criterion standard during walking on a treadmill at a speed of 4.8 km/hour to be considered accurate. In our study, the Zip fulfils this criterion.
In the real-life test, the step inaccuracy was higher than 20% when looking at the 24-hour measurements. Other studies have reported step counter inaccuracy in real-life settings.18 In 24-hour measurements, activities other than walking activity will always be recorded.37 A study by Singh et al revealed a high accuracy at slow speeds when the Zips were placed on the lower limbs.38 This is an interesting finding for a cardiac population, even though placing a Zip on a lower limb might be difficult, as the device is not designed for this placement.
During periods with evident walking (TP3min), inaccuracy decreased with increased walking speed. Increased step-counting error with decreasing walking speed is a condition for step counters,12 ,13 ,39 resulting in a challenge for measuring steps in frail older individuals with walking disabilities.20 A review of ambulant older people (>70 years of age) reported that the mean usual walking speed was between 1.7 and 2.4 km/hour.40 Additionally, Pepera et al41 found that patients with chronic heart failure had a shorter step length and walked more slowly than healthy adults walk during a 6 min walk test. Consistent with this finding, Jehn et al39 ,42 found that self-paced walking speed decreased with increasing severity of heart failure. In the present study, five patients had heart failure, and the mean (±SD) age was 67 (10.03) years. This may be a reason for the inaccuracy of the 24-hour measurements.
The treadmill test was performed with healthy adults under the age of 40 years and an equal distribution of gender. In the real-life study, the participants were patients with cardiac disease, only two women were included, and the mean age was 67 (10.03) years. This might limit transferability of the treadmill tests to real-life studies. In a study by Jehn et al,39 ,43 patients with heart failure performed a treadmill and a real-life test. The authors found satisfactory step accuracy even for slow walking during treadmill tests.39 ,43 Treadmill walking is not representative of normal walking, and it is hypothesised that test persons on a treadmill will produce non-natural rigid walking regardless of health condition or age (which might improve accuracy). This means that the test person's health condition is of less significance during treadmill walking. Consistent with other studies, the present study found that the inaccuracy was primarily due to an underestimation of steps taken.37 ,44 Thus, if the user compares their walking activity to external benchmarks (eg, 10 000 steps per day) the underestimation is likely to be a source of frustration.27 Only Aharbi et al have found that another version of a Fitbit device overestimates steps counted compared with a gold standard.45
Zips may provide 24-hour activity feedback on different variables (steps, active/sedentary minutes, distance and energy expenditure over time), making them appealing for lifestyle interventions and as a tool for promoting physical activity.10 ,38 A complete 24-hour activity record provides a picture of a person's daily life, providing the possibility to tailor activity recommendations and integrate them into a person's daily life. Despite inaccuracies, the long-term use of these devices is expected to demonstrate an increase or decrease in walking over time.46 Thus, step counters may be an effective adjunct to more traditional physical activity promotion strategies.47
Limitations
The manufacturers recommend a number of body placements for the Zip (breast pocket, bra, pant pockets or belt).48 This study did validate the Zips when placed on only the waist or upper body for the treadmill study and when placed on only the waist for the real-life study. The validity of the Zip at other placements is still unknown.
Despite the fact that the results from treadmill studies cannot be directly transferred to real-life settings, they are still important because they can isolate the step accuracy at different speeds and because it is possible to control the environment (speed, placement of devices, time spent walking and so forth).
In the real-life study, the patients were instructed on how to fasten the devices during the 24-hour test. Despite these conditions, we have reason to believe that the Shimmer devices sometimes slide to one side or another. None of the participants reported (verbally or in the diary) any misplacement of the devices, but the gyroscope readings sometimes showed reverse data halfway through the test (as if the Shimmer was turned upside down). This, of course, complicated the data readings, as the researcher had to analyse the data in reverse. Still, this issue did not hamper the validity of the readings.
Conclusion
A speed of 3.6 km/hour or higher is required to obtain an accurate measurement of steps using the Zip, which may be a challenge for patients with cardiac disease who walk at a slow pace. Investigations using the treadmill revealed that the Zip had an average per cent relative error of <3% at speeds of 3.6 km/hour and higher; this inaccuracy was mainly due to an underestimation of the Zip. In the 24-hour real-life tests of patients with cardiac disease, the inaccuracy was higher than the expected 20% except during periods with evident walking in which the Zip showed acceptable accuracy at speeds of 3.6 km/hour or more. Despite the fact that the Zip appears to be inaccurate at slower speeds, the device might have the potential to motivate patients with cardiac disease to become more active as the Zip provides the patients with information on their walking activity over time.
Acknowledgments
We specifically thank the nurses and doctors at the Departments of Cardiothoracic Surgery and Cardiology, Aalborg University Hospital, Denmark for their cooperation during the recruitment of the patients.
References
Footnotes
Contributors CBT, JJA, EES, MG, BID and JH were involved in the design of the study. Recruitment of the participants was performed by CBT, and the data collection and executions of the tests were performed by CBT and JH. Data analysis and interpretation were performed by JH and CBT, and CBT wrote the first draft of the paper. CBT, JJA, EES, MG, BID and JH contributed to the revision of the manuscript, and all authors approved the final manuscript for publication.
Funding This work was supported by the Lundbeck Foundation (grant number FP25/2013). Furthermore, the research was funded by the Eir Research and Business Park, the European Regional Development Fund, the UNIK Partnership, the Danish Council of Research and Innovation, and the Departments of Cardiothoracic Surgery and the Clinical Nursing Research Unit of Aalborg University Hospital and Aalborg University.
Competing interests None declared.
Patient consent Obtained.
Ethics approval The study did not require approval.34
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.