Objectives The purpose of this study was to develop a replicable methodology of mobile ecological momentary assessments (EMAs) to capture prospective physical activity (PA) within free-living social and physical contexts by leveraging citizen-owned smartphones running on both Android and iOS systems.
Design Data were obtained from the cross-sectional pilots of the SMART Platform, a citizen science and mobile health initiative.
Setting The cities of Regina and Saskatoon, Canada.
Participants 538 citizen scientists (≥18 years) provided PA data during eight consecutive days using a custom-built smartphone application, and after applying a rigid inclusion criteria, 89 were included in the final analysis.
Outcome measures EMAs enabled reporting of light, moderate, and vigorous PA, as well as physical and social contexts of PA. Retrospective PA was reported using International Physical Activity Questionnaire (IPAQ). For both measures, PA intensities were categorised into mean minutes of light and moderate-to-vigorous PA per day. Wilcoxon signed ranks tests and Spearman correlation procedures were conducted to compare PA intensities reported via EMAs and IPAQ.
Results Using EMAs, citizen scientists reported 140.91, 87.16 and 70.38 mean min/day of overall, light and moderate-to-vigorous PA, respectively, whereas using IPAQ they reported 194.39, 116.99 and 98.42 mean min/day of overall, light and moderate-to-vigorous PA, respectively. Overall (ρ=0.414, p<0.001), light (ρ=0.261, p=0.012) and moderate-to-vigorous PA (ρ=0.316, p=0.009) were fairly correlated between EMA and IPAQ. In comparison with EMAs, using IPAQ, citizen scientists reported significantly greater overall PA in active transportation (p=0.002) and recreation, sport and leisure-time domains (p=0.003).
Conclusions This digital epidemiological and citizen science methodology adapted mobile EMAs to capture not only prospective PA, but also important physical and social contexts within which individuals accumulate PA. Ubiquitous tools can be leveraged via citizen science to capture accurate active living patterns of large populations in free-living conditions through innovative EMAs.
- public health
- sports medicine
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The methodology addresses current discrepancies in mobile ecological momentary assessments (EMAs; eg, triggering processes, time to follow-up).
The methodology of time-triggered mobile EMAs is effective in recording comprehensive daily prospective physical activity.
The methodology facilitates capture of both physical and social contexts of physical activity prospectively.
The main limitation is the small sample size after applying the inclusion criteria.
All observations are self-reported by citizen scientists.
Advances in mobile technology over the past decade have facilitated the innovation of ecological momentary assessments (EMAs), which are digital epidemiological tools that aid in understanding environmental, social and behavioural processes.1 2 EMAs can capture real-time data that reflect the dynamics of participants’ experiences in their natural environment and thus they are increasingly being used to monitor health behaviours among populations across the life course.3–5 In active living research, evidence indicates that EMAs are a valid, reliable and feasible method of data collection.6 7
EMAs are an advancement over traditional self-report methods as they enable data collection more proximal to the time and place that a behaviour has occurred.2 8 Moreover, EMAs overcome many of the limitations of traditional self-report surveys to provide information regarding specific activity types (eg, watching TV vs video gaming) and capture important factors that influence health behaviours such as mood and environmental perceptions.5 9 10 In measuring physical activity (PA) intensities, EMAs have been shown to minimise recall6 11 and social desirability bias12 of traditional self-report measures.
Several studies have examined the validity of smartphone-based EMAs compared with other objective devices (accelerometers, pedometers) and self-report measures of PA.2 10–16 Overall, estimates from EMAs were found to be highly correlated with accelerometer estimates.13 14 However, this evidence also indicates that PA was over-reported when International Physical Activity Questionnaire (IPAQ) was used2 and that daily PA EMA reports were not significantly associated with their traditional recall measures.12
Currently, there is little evidence of existing EMA methods that capture PA intensities across various physical (leisure-time PA, transit-related PA, occupation-related PA and household/domestic-related PA)15 and social contexts (with family, friends, etc).16 Moreover, there are discrepancies in smartphone-based EMA methodologies, which range from inconsistent EMA triggering processes and varying times of prospective follow-up, to limitations of using identical mobile devices and operating systems.8
The objective of this study is to address current deficiencies in active living EMA approaches by developing a replicable digital epidemiological and citizen science methodology to capture prospective PA within free-living social and physical contexts. This objective will be achieved by leveraging citizen-owned smartphones running on both Android and iOS systems, and by comparing EMA measures with traditional self-report measures of PA within the same cohort.
This study is part of the SMART Platform, which is a mobile health and citizen science initiative for active living surveillance, integrated knowledge translation and policy and real-time interventions.8 17 18 Citizen science is a participatory approach where participants, termed citizen scientists, actively engage in the research process from data collection to knowledge translation, thus improving the probability of longitudinal participant compliance.15 A detailed description of SMART Platform’s methods, including recruitment and data collection strategies, is described in the Platform’s methodology publication.8
The data for this study have been obtained from the 2017 (1 April–May 31) and 2018 (4 January–March 31) cohorts of the SMART Platform,8 which is a prospective investigation designed to capture active living data from adults residing in the two largest urban centres in Saskatchewan, Canada (Regina and Saskatoon). All subjective (via traditional validated surveys and EMAs) and objective data (via smartphones sensors) related to PA, sedentary behaviour, and perception of environment, individual motivation, health outcomes and eudaimonic well-being were obtained through citizen-owned smartphones on 8 consecutive days (figure 1).
Patient and public involvement
Participants in the SMART Platform are ‘citizen scientists’ as they can engage with the researchers at all stages of the research process. Thus, citizen scientists informed the design, research questions and outcome measures. As part of the social media campaign for recruitment, citizen scientists were encouraged to inform their friends about the study. Finally, as integrated knowledge translation is part of the SMART Platform, results are disseminated throughout the study period using the community voices webpage of the Platform’s website: https://www.smartstudysask.com/community-voices.
Recruitment and participants
Citizen scientists for SMART Adult cohorts were recruited online through social media, and in-person from the universities of Regina and Saskatchewan and community centres located in different neighbourhoods in each city to capture a socioeconomically representative sample. Citizen scientists were guided to download a custom-built epidemiological smartphone application (app), specifically adapted for the SMART Platform, which captures data through both Android and iOS platforms. All citizen scientists provided informed consent through the app and confirmed their age (≥18 years) before joining the study.
The two primary measures used in this study are the IPAQ,19 which collects retrospective PA in four physical domains (recreation, active transportation, work and home), and the SMART Platform’s modified EMA, which captures prospective daily PA in both social and physical contexts.
International Physical Activity Questionnaire
IPAQ was deployed at baseline as soon as citizen scientists downloaded the app to self-report physical activities over the past 7 days that were of at least 10 consecutive minutes in duration. These activities were categorised by four domains: (1) recreation (eg, weight training, sports (soccer, hockey, etc), aerobics, running, jogging, swimming, cycling, etc); (2) household (eg, carrying light loads, sweeping, washing windows, raking, etc); (3) transportation (eg, travelling in a train, bus, car, or other kind of motor vehicle, etc) and (4) work (eg, heavy lifting, digging, heavy construction or climbing upstairs, etc). The records included the number of times per week (within the last 7 days) and average minutes per day for each activity.
Adapted daily EMAs
Using the SMART Platform, time-triggered modified EMAs (figure 1A–D) were developed, tested and piloted, before being pushed to citizen scientists’ smartphones between 20:00 and 20:30 on each day for 8 consecutive days. These EMAs were designed to expire at 03:00 the next day. Citizen scientists were asked to report only those physical activities that were of at least 10 min in duration at a time. More importantly, each EMA was designed to not only measure intensity and volume (in minutes) of PA, but also to capture social (ie, with whom they accumulated PA (figure 1C) and physical contexts (ie, where they accumulated PA (figure 1D). This design was achieved by creating a looped linkage, where on entering the type and volume of each activity, the EMA triggered the social and physical context questions.
Derived variables—intensities and volume of PA
International Physical Activity Questionnaire
Thirty-seven questions related to PA were asked and three different categories of intensities were created (light, moderate and vigorous PA) by combining PA across four domains: recreation, household, workplace and active transportation. Moderate and vigorous PA intensities are combined to derive ‘moderate-to-vigorous PA’. After conducting several aggregation techniques, two final intensity variables were derived for IPAQ retrospective PA: mean minutes per day of light and moderate-to-vigorous PA.
Adapted daily EMAs
A similar approach was employed to derive two final intensity variables for EMA prospective PA: mean minutes per day of light and moderate-to-vigorous PA. For example, light PA included walking, light hiking, any light PA/sport (eg, golf bowling etc), yoga and light intensity household chores (eg, washing dishes sweeping laundry gardening). Moderate-to-vigorous PA included moderate-to-vigorous hiking, running, biking, any team sport (football hockey soccer, etc), any other sport or activity (swimming canoeing skiing, etc), weight training, dance/aerobic/cardio exercise and moderate-to-vigorous intensity household chores (eg, shovelling driveways, washing a car, etc).
PA information from the IPAQ and EMAs (based on the question ‘Where did you do this activity?’) was grouped into domains. Domain 1: PA at workplace (IPAQ) and from work (EMAs). Domain 2: transportation PA (IPAQ) and from street (EMAs). Domain 3: housework, house maintenance and caring from family (IPAQ) and from home (EMAs). Domain 4: recreation, sport and leisure-time PA (IPAQ) and from park, gym and sport facility (EMAs).
Social context information was collected via EMA question, ‘With whom did you do this activity?’ for each PA that the participants reported. Categories for social context included ‘by myself, with my dog, with my friend(s), with my parent(s)’, among others.
The inclusion criterion to determine the final sample was dependent on citizen scientists completing the IPAQ, and answering the daily EMA on at least 3 days. Continuous estimates were reported as means with SD and medians with 25th and 75th percentiles, depending on normality. Where estimates were non-normal and positively skewed, median and IQRs were used. Wilcoxon signed ranks tests and Spearman correlation procedures were conducted to compare PA intensities and domain-based PA reported via IPAQ and EMAs. Correlation coefficient values of <0.20, 0.21–0.40, 0.41–0.60, 0.61–0.80 and 0.81–1.0 were considered as weak, fair, moderate, strong and very strong correlation, respectively.20 Analyses were conducted in SPSS V.24.0 (SPSS) with significance set at α <0.05.
After applying the decision rule of including only those citizen scientists who completed IPAQ, and answered the daily EMA on at least 3 days, out of 538 participants, only 89 were included in this study (table 1), among whom 47 identified as women (51.68%), and 26 identified as men (29.21%), and 19.11% (n=17) did not reveal their identity. The final sample had the mean age of 37.15 years (SD=15.92), and a mean body mass index of 28.46 (SD=7.78). The median (25th, 75th percentiles) and the mean (SD) duration of time (min/day) spent in each of the activity intensities (light, moderate and vigorous), as well as overall PA were derived from both IPAQ and EMA measures.
Using EMAs, citizen scientists reported 140.91, 87.16 and 70.38 mean min/day of overall PA, light PA and moderate-to-vigorous PA. The same citizen scientists reported 194.39, 116.99 and 98.42 mean min/day of overall PA, light PA and moderate-to-vigorous PA using the IPAQ (table 2). These findings show that although there are no significant differences between activity intensities reported via EMAs and IPAQ, citizen scientists consistently overestimated their PA using IPAQ in comparison with EMAs. Table 3 demonstrates the correlation between EMA and IPAQ measures to show that overall (ρ=0.414, p<0.001), light (ρ=0.261, p=0.012) and moderate-to-vigorous PA (ρ=0.316, p=0.009) were fairly correlated across both measures. Figure 2 shows the visual representation of these correlations.
Figures 3 and 4 demonstrate the distribution of overall PA accumulated across different physical and social contexts, as reported by citizen scientists using EMAs. Among physical contexts, citizen scientists reported accumulating overall PA predominantly at home (26.4%), on the streets ((20.4%), ie, active transportation), at the gym (13.7%), at work (13.1%) and in parks (12.3%). When it comes to social context, citizen scientists overwhelmingly reported accumulating overall PA by themselves (64.2%), with some reporting being active with friends (14.7%) and relatives (6.3%).
As IPAQ captures PA in four physical domains (workplace, active transportation, household and recreation, sport and leisure time) to compare estimates between EMA and IPAQ, EMA estimates of overall PA accumulated across various physical contexts were categorised to match the physical domains of IPAQ. Using EMAs, citizen scientists reported 20.50, 16.41, 25.33 and 20.88 mean min/day of overall PA across workplace, active transportation, household and recreation, sport and leisure-time domains, respectively. Using IPAQ, the same citizen scientists reported 32.14, 43.97, 38.27 and 145.90 mean min/day of overall PA across workplace, active transportation, household and recreation, sport and leisure-time domains, respectively.
These findings show that in comparison with EMAs, there is a consistent pattern of over-reporting of overall PA across all physical domains when citizen scientists used IPAQ, with statistically significant differences observed in active transportation (p=0.002) and recreation, sport and leisure-time domains (p=0.003; table 4). Table 5 demonstrates correlation between EMA and IPAQ measures for overall PA across four physical domains, with moderate correlation being depicted in household (ρ=0.607, p=0.036), and recreation, sport and leisure-time domains (ρ=0.587, p=0.021).
The objective of this study was to address current deficiencies in PA EMA approaches by developing a novel and replicable methodology of standardised time-triggered smartphone-based EMAs to capture prospective PA within free-living social and physical contexts by leveraging citizen-owned smartphones running on both Android and iOS systems.
We were able to not only develop a novel EMA that can be time-triggered by both iOS and Android devices to capture prospective PA across physical and social contexts to address current gaps in EMA methodologies,21 22 but also compared this EMA measure with IPAQ to highlight potential discrepancies between prospective and retrospective measures in capturing active living in free-living conditions.
Although not statistically significant, irrespective of the intensity of PA (overall PA, light and moderate-to-vigorous PA), citizen scientists consistently over-reported activity with IPAQ in comparison with EMA. However, when PA intensities were compared across the four physical domains (workplace; active transportation; household; and recreation, sport and leisure), PA reported via IPAQ in active transportation; and recreation, sport and leisure domains was significantly greater than PA reported via EMAs.
These findings corroborated a longitudinal validation study by Swendeman et al,23 who concluded that the intermethod reliability between smartphone-based EMAs and their corresponding recall reports was low and no significant associations were observed.23 Another validation study that compared PA EMAs with IPAQ and accelerometer measures concluded that EMA measures correlated better with accelerometers.4 Several studies have been conducted to compare self-report estimates of PA with objective measures (an accelerometer),4 24 25 with evidence suggesting that an ideal approach potentially lies between traditional validated self-report measures and accelerometry,24 especially because accelerometry is unable to capture context.
This is indicative of EMAs being the potential solution for comprehensively capturing PA by minimising recall bias. However, a key gap in current methodologies is that EMAs are used in more controlled experiments, where identical mobile devices running on same operating systems are to participants.26 Moreover, EMA methodologies lack standardisation and sufficient rigour such as inclusion criteria for valid data. A key advancement of our study is including only those participants who completed EMAs on at least 3 days, an inclusion criterion which provides the necessary rigour to arrive at valid data.
EMAs are currently novel methods that are in need of standardisation. We applied a strict inclusion criterion, where we included only participants with PA data on at least 3 out of 8 days in the final analysis, which resulted in exclusion of most participants. We did this even at the risk of reducing our sample size because this rigorous inclusion criterion is an essential step in standardising EMA measures, and obtaining valid and reliable data. This is not very different from accelerometry standardisation methods, where data are considered valid if participants wear accelerometers for at least several hours (eg, 10 hours) on at least 2–3 days in a 1-week study period.27 28
Another gap in current methodologies is the inability of existing EMAs to capture important physical and social contexts within which PA is accumulated. We developed an innovative looped linking mechanism that sequentially triggers questions about type, volume and context of PA. The findings showed that citizen scientists reported accumulating most PA while at home, through active transportation, at the gym, at their work places and in parks.16 29 The distribution of accumulation of overall PA across these physical contexts provides important evidence to develop interventions modifying physical spaces to address physical inactivity.30–33 The findings also showed that most citizen scientists accumulated PA by themselves,29 which points towards informing individual-level interventions that facilitate intrinsic motivation.34–36 Although these findings are not novel by themselves, the methodology of using a single time-triggered EMA per day to capture volume, intensity and physical and social contexts of PA is innovative.
Although EMAs are valid and reliable measures to measure PA, current evidence indicates that there is no gold standard in assessing prospective PA using mobile EMAs.3 5 21 Our study advances a methodology that introduces conceptual and technological advancement (citizen science approach using citizen-owned devices functioning on both iOS and Android systems), scientific rigour (stringent inclusion criteria for valid data) and comprehensiveness of data collection (volume, intensity and contexts). In working towards standardised EMA methodology, future studies need to address the balance between capture of prospective PA and participant burden/compliance in repeatedly responding to EMAs. Future studies should also combine EMAs with objective measurement to measure PA,37 38 to concretely capture PA.
Nevertheless, EMAs for PA measurement have the potential to reliably record active living and could substitute accelerometers when needed.2 In our study we addressed existing gaps in EMA methodology to measure PA by adopting a citizen science approach39 in deploying a comprehensive, yet generic EMA that captures type, volume and context of PA. More importantly, participants used their own smartphones, which operated on either iOS or Android systems. Thus, this methodology is not only replicable, but also expands the scope of leveraging ubiquitous tools such as smartphones40 to conduct ethical surveillance8 41 of PA among large populations. Citizen science approaches are increasingly being considered in active living research,42 and it is important that methodological advancements are in step with conceptual and technological innovations. With more than 3 billion smartphones currently in circulation globally,40 standardised and generic EMA methodologies can enable real-time engagement through crowdsourcing43 44 for ethical active living surveillance.8
Strengths and limitations
The primary strength of the study is the development of novel and replicable methodology to capture prospective PA from large populations using citizen-owned devices. This citizen science approach, if replicated appropriately, can transform surveillance of physical PA among large populations by leveraging citizen owned devices. Implementing such innovative approaches of PA surveillance will be critical to develop appropriate interventions to address global physical inactivity.
In terms of limitations, all observations are self-reported by citizen scientists. The study sample size was also small after applying the inclusion criteria; however, smaller sample sizes are not uncommon in smartphone-based EMA studies.6 Another limitation is that IPAQ and EMAs measured PA in different timeframes. As IPAQ captures data retrospectively and EMAs capture data prospectively, they cannot be issued simultaneously. Nonetheless, although IPAQ could have been issued on day 8, we refrained from such late deployment based on the evidence from our pilots, which showed that compliance to burdensome traditional recall surveys such as IPAQ is much higher when it is issued as close to participant enrolment in the study as possible.
With growth of smartphones projected to only magnify in the future,16 these ubiquitous tools can be leveraged via citizen science to capture accurate active living patterns of large populations in free-living conditions through innovative EMAs. This digital epidemiological and citizen science methodology adapted mobile EMAs to minimise recall bias and capture not only prospective PA, but also important physical and social contexts within which individuals accumulate PA.
The authors acknowledge the entire DEPtH Lab team, and especially Kayla Brodersen for her role in the successful implementation of the SMART Platform.
Contributors TK contributed substantially to the study design, acquisition and interpretation of data, and writing the manuscript. LMC contributed substantially to the acquisition and interpretation of data, and writing the manuscript. All authors read and approved the final manuscript.
Funding This study was funded by the Saskatchewan Health Research Foundation’s Establishment Grant.
Disclaimer This funding body had no role in study design; data collection, analysis and interpretation; or in writing the manuscript.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Patient consent for publication Not required.
Ethics approval All citizen scientists provided informed consent through the app and confirmed their age (≥18 years) before being recruited. Ethics approval was obtained from the universities of Regina and Saskatchewan through a synchronized review protocol (REB # 2017-29).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.