You are here

  • Home
  • Archive
  • Volume 7, Issue 6
  • Effectiveness of recruitment to a smartphone-delivered nutrition intervention in New Zealand: analysis of a randomised controlled trial

Article Text

Article menu

Download PDFPDF

XML
Research methods
Research
Effectiveness of recruitment to a smartphone-delivered nutrition intervention in New Zealand: analysis of a randomised controlled trial
  1. Ekaterina Volkova1,
  2. Jo Michie1,
  3. Callie Corrigan2,
  4. Gerhard Sundborn3,
  5. Helen Eyles1,
  6. Yannan Jiang1,
  7. Cliona Ni Mhurchu1
  1. 1 National Institute for Health Innovation, University of Auckland, Auckland, New Zealand
  2. 2 Toi Tangata, Auckland, New Zealand
  3. 3 Department of Epidemiology and Biostatistics, School of Population Health, University of Auckland, Auckland, New Zealand
  1. Correspondence to Dr Cliona Ni Mhurchu; c.nimhurchu{at}auckland.ac.nz

Abstract

Objectives Delivery of interventions via smartphone is a relatively new initiative in public health, and limited evidence exists regarding optimal strategies for recruitment. We describe the effectiveness of approaches used to recruit participants to a smartphone-enabled nutrition intervention trial.

Methods Internet and social media advertising, mainstream media advertising and research team networks were used to recruit New Zealand adults to a fully automated smartphone-delivered nutrition labelling trial (no face-to-face visits were required). Recruitment of Māori and Pacific participants was a key focus and ethically relevant recruitment materials and approaches were used where possible. The effectiveness of recruitment strategies was evaluated using Google Analytics, monitoring of study website registrations and randomisations, and self-reported participant data. The cost of the various strategies and associations with participant demographics were assessed.

Results Over a period of 13 months, there were 2448 registrations on the study website, and 1357 eligible individuals were randomised into the study (55%). Facebook campaigns were the most successful recruitment strategy overall (43% of all randomised participants) and for all ethnic groups (Māori 44%, Pacific 44% and other 43%). Significant associations were observed between recruitment strategy and age (p<0.001), household size (p<0.001), ethnicity (p<0.001), gender (p=0.005) and interest in healthy eating (p=0.022). Facebook campaigns resulted in the highest absolute numbers of study registrations and randomisations (966 and 584, respectively). Network strategies and Facebook campaigns cost least per randomised participant (NZ$4 and NZ$5, respectively), whereas radio advertising costs most (NZ$179 per participant).

Conclusion Internet and social media advertising were the most effective and least costly approaches to recruiting participants to a smartphone-delivered trial. These approaches also reached diverse ethnic groups. However, more culturally appropriate recruitment strategies are likely to be necessary in studies where large numbers of participants from specific ethnic groups are sought.

Trial registration ACTRN12614000644662; Post-results.

  • recruitment
  • randomised controlled trial
  • smartphone
  • social media
  • nutrition
  • effectiveness
  • ethnicity

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-016198

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Reports on effectiveness and costs of a range of recruitment strategies (internet and social media advertising, mainstream media advertising and research team networks) were used for a smartphone-delivered study.

    • Effectiveness was evaluated using Google Analytics, monitoring of study website registrations and randomisations and self-reported data.

    • Information is provided on participant retention rates and strategies to reduce duplicate and fake registrations.

    • The broad categorisation of self-reported recruitment source data prevented more precise subcategory analysis.

    • Staffing costs associated with recruitment strategies were not assessed.

    Introduction

    Monitoring and reporting of study recruitment strategies and their effectiveness facilitates improvements in design and methods for future randomised controlled trials (RCTs). Systematic reviews provide an indication of successful recruitment strategies and trial design elements such as incentives and open-label design,1 2 and their cost-effectiveness.3 However, applicability and success of various strategies depends on specifics of the trial design, setting and study population.

    Smartphone delivery and measurement of trial interventions4–6 is a relatively new initiative and presents unique recruitment challenges compared with traditional RCTs. In addition to standard trial eligibility criteria, prospective participants must have smartphone access and be adept at using technology. Moreover, attrition rates are potentially higher because enrolment rates may be augmented by a ‘novelty factor’ that decays over time.7 Lack of personal contact with study staff may also increase attrition.7 Such challenges are also relevant to public health and health promotion interventions that increasingly deliver programme components using smartphone technology. Identifying successful recruitment and retention strategies for this type of intervention is, therefore, timely and important.

    In addition to traditional RCT recruitment methods such as advertising via community fliers, newspapers or media,8 newer strategies including Internet and social media advertising9 are being increasingly used. A systematic review found that online recruitment strategies, Facebook in particular, were promising ways to recruit participants for internet and mobile health (mHealth) studies.7 However, limited data were available, and specific gaps identified were reporting of participant retention rates, and methods to identify and manage fake and duplicate registrations.

    Potential greater reach among underserved and diverse population groups is one suggested benefit of online recruitment.8 10 In New Zealand (NZ), Māori (indigenous New Zealanders making up 14.9% of population) and Pacific peoples (7.4%) are priority groups for health intervention programmes due to their disproportionately high prevalence of non-communicable diseases and associated risk factors.11 Adequate representation of these groups in health research is vital but often hard to achieve with generic recruitment strategies, so ethnic-specific tailoring of strategies is recommended.12 13

    Our aim was to assess the effectiveness of a range of approaches used to recruit participants to a large smartphone-delivered nutrition labelling RCT14 and to examine associations of recruitment strategy with ethnicity and other participant demographics.

    Methods

    The Starlight RCT received ethics approval from the University of Auckland Human Participants Ethics Committee (reference number 011390), and the study protocol was published in 2014.15 The trial was registered on the Australian New Zealand Clinical Trials Registry (ACTRN12614000644662). All participants provided informed consent via a questionnaire completed on their smartphone.

    The aim of the trial was to evaluate the effects of two interpretive nutrition labels, traffic light labels and Health Star Rating labels, compared with a non-interpretive label on the healthiness of consumer food purchases. The target was to recruit and randomise 1500 eligible NZ adults, composed of approximately equal numbers of Māori, Pacific and other ethnicities. Eligible participants were aged 18 years or older, owned a smartphone, were the main household shopper and could read and understand English. All components of the trial (screening, consent, registration, randomisation, intervention and data collection) were delivered via a bespoke, automated smartphone application (app).16 Outcome data were collected in the form of scanned household food purchase records. Completion of a run-in period (week 1) with a requirement to record at least 15 food items was a prerequisite for randomisation. The 4-week nutrition labelling intervention comprised randomisation to one of two interpretive nutrition labels (traffic lights17 or Health Star Ratings18) or to a control group (nutrition information panel).19 The primary study outcome was healthiness of all foods and beverages purchased over the intervention period, measured using the Food Standards Australia New Zealand nutrient profiling scoring criterion.20

    Recruitment

    Recruitment commenced in October 2014. A key focus was to attract 500 Māori and 500 Pacific participants, with advice provided by Māori and Pacific team members. Recruitment was open initially to all ethnic groups, but was closed to non-Māori and non-Pacific individuals after 6 months when the target for this group (500 participants) was reached. Recruitment of Māori and Pacific individuals continued for a further 5 months, but at that time it became clear that recruitment targets would not be met within the study timeframes. Therefore, recruitment was reopened to all ethnicities again for 2 months and closed finally in November 2015.

    Recruitment campaigns directed volunteers to a designated trial website to answer a prescreening questionnaire, and a link to download the trial app was then provided. Enrolment occurred via the app, but the prescreening step allowed the study team to monitor registration numbers and ethnicity targets. The trial app was available free of charge in NZ Google Play and iTunes app stores. Information in the app stores also encouraged volunteers to visit the trial website before downloading the app.

    Recruitment materials and advertisements were developed by the research team (see online supplementary file1 A, B). Advertising campaigns were conducted at staggered intervals over the 13-month recruitment period using research team networks, Internet and social media advertising and mainstream media advertising (figure 1). The impact of recruitment strategies was assessed continuously by comparison with monthly recruitment targets.

    Supplementary Material

    Supplementary data
    Figure 1
    Figure 1

    Recruitment rates in response to implementation of key recruitment strategies over time.

    Three media releases were issued by the university communications office (October 2014, November 2014 and 2015). Promotion via research team networks was undertaken using group email lists, word of mouth, staff intranets, personal Facebook pages and hard copy recruitment flier distribution. Electronic recruitment fliers were circulated via email lists at several NZ universities, healthcare providers and relevant non-government organisations. Requests were made to NZ schools, particularly those with high number of Māori and Pacific students, to promote the study via their newsletters with the aim of recruiting parents of students. Paid internet advertising was undertaken using 14 promoted posts on Facebook (see online supplementary file 2), two Google AdWords campaigns (see online supplementary file 3 part 1 and part 2) and one LinkedIn campaign. Other paid advertising included an advertisement in a Māori magazine (including a web banner on their website), promotional advertisements on Pacific radio and representation at relevant events, such as Māori and Pacific cultural festivals and health provider conferences.

    Supplementary Material

    Supplementary data

    Supplementary Material

    Supplementary data

    Strategies to maintain retention

    Participants were sent regular notifications via the study app, SMS text reminders16 and email messages. All registered participants received a NZ$10 gift voucher, and those who completed the trial (defined as completion of the follow-up questionnaire at the end of the 5-week study) received a further NZ$80 gift voucher.

    Registration logic checks

    Because incentives (vouchers) were offered for study participation, repeat registrations by the same participants and re-registration by ineligible participants were identified as risks. To prevent duplicate registrations, individual internet protocol address and email address checks were implemented on the study screening website. Within the app, logic checks for unique and valid email address and valid age (18–100 years) were also implemented. All registrations were checked daily for duplicate surnames, street addresses and phone numbers. All suspected duplicate registrations were investigated by research staff. Confirmed duplicate registrations were contacted with the request that they discontinue the trial.

    Data analysis

    Information on the demographic characteristics of trial participants was collected via a baseline questionnaire completed in the study app.15 Self-reported ethnicity was grouped into three categories: Māori, Pacific and Other. Google Analytics21 was used to track visits to the study website. Data on self-reported recruitment strategy that attracted participants were collected using the question “How did you find out about the study?”  The effectiveness of recruitment strategies was assessed by collating number of participants registered, randomised and completing the trial. Conversion to randomisation was defined as proportion of registered participants randomised, and retention rate was defined as proportion of randomised participants retained. Participants who could not be randomised due to technical issues with the app or their phone were excluded from analysis.

    Self-reported recruitment data were matched with recorded costs for each strategy. Only broad categories were available in the self-reported data. Thus in the cost analysis, promotion or advertising via community events, research team’s personal and professional network, and coverage in mainstream paper media were combined in one category titled ‘network and paper media’ (table 1). It was not possible to directly account for the cost of research staff time on each of the strategy, and thus only the direct costs of each strategy were assessed. Costs were reported as total per strategy, and cost-per-randomised and cost-per-completed participant (NZ$).

    View this table:
    Table 1

    Study recruitment strategies

    The analysis was conducted in IBM SPSS Statistics V.21.0. Simple descriptive statistics were used to describe the number of participants registered and randomised by recruitment strategy, and key demographics. Statistical difference between categories was tested using the χ2 test for categorical variables and analysis of variance for continuous variables. Statistical tests were two sided at 5% significance level.

    Results

    Recruitment summary

    There were 2448 study registrations, of which 1035 were excluded prior to randomisation. Reasons for exclusion were ineligibility based on study inclusion criteria (n=205), failure to complete the study run-in phase (n=727), duplicate registration due to a technical problem with the app (n=47) and non-randomisation of eligible individuals due to a technical problem with the app (n=56). A further 56 individuals were excluded from analysis because they were randomised in error (ie, randomised even though they failed to meet qualifying run-in criteria) as a result of a technical problem with the app. Thus, 1357 individuals (55% registrations) were randomised and included in the main study data analysis: 243 (18%) Māori, 87 (6%) Pacific and 1027 (76%) other ethnic groups. Of those randomised, 1202 (89%) completed the study: 201 Māori (83%), 75 Pacific (86%) and 926 (90%) other.

    The demographic characteristics of the randomised participants, overall and by recruitment strategy, are presented in table 2. The study population was predominantly female (89%) with a mean (SD) age of 33 (9) years. Representation of Māori, Pacific and other ethnicities was similar to their distribution in the NZ population (table 2). However, more than two-thirds of trial participants were tertiary educated (vs 26% in 2013 NZ Census),22 and nearly all (97%) reported that they were moderately to very interested in healthy eating. Household income was also not a representative of the general population, with higher income groups over-represented in the sample (table 2).

    View this table:
    Table 2

    Demographic characteristics of study participants according to the recruitment strategy

    Recruitment strategy was significantly associated with participants’ demographic characteristics, namely age (p<0.001), household size (p<0.001), ethnicity (p<0.001), gender (p=0.005) and interest in healthy eating (p=0.022). Participants recruited via internet/social media were younger on average (31 years), whereas those recruited via radio and newspaper advertising were older (36.5 and 37 years, respectively). A larger proportion of male participants was recruited via strategies focused on personal contact, namely word of mouth and email invitations (14% and 17%, respectively, vs 7%–9% by other strategies). Finally, radio advertising attracted a greater proportion of participants not interested in healthy eating (7% vs 1%–4% by other strategies).

    Recruitment effectiveness

    A cumulative summary of recruitment over time in response to the various strategies is presented in figure 1. The largest peaks in registrations were observed in response to Facebook campaigns (up to 600 new registrations per campaign), followed by promotion via schools and research team networks (50–100 new registrations per campaign).

    Analysis of trial registration website visits using data from Google Analytics and self-reported recruitment source is presented in figure 2. ‘Social media’ and ‘paid search’ (Google AdWords) brought 55% of the website visits (figure 2A). This aligns with self-reported data, which showed that almost 50% of registrations were due to ‘internet’ campaigns (figure 2B). Further examination showed 98% of such visits to the study registration page came from Facebook, and the remainder were from other social media and paid search, including Google AdWords. Therefore, all registrations that reported ‘internet’ as a source were considered as arising from Facebook. The second highest website traffic acquisition was via the ‘direct’ channel (figure 2A), that is, direct visits to the trial webiste not redirected from other websites (likely people using the website address provided in fliers, emails, magazines and newspapers). Self-reported data also showed that over 40% registered participants were reached via networks (‘word of mouth’ and ‘email invitations’) and media coverage (‘newspapers and magazines’).

    Figure 2
    Figure 2

    Trial website visit and registration numbers by recruitment strategy.

    The effectiveness and cost per strategy are presented in table 3. Facebook (paid campaigns and free posts) resulted in the highest absolute registration, randomisation and study completion numbers, both overall and for each ethnic group. However, higher conversion from registration to randomisation rates were achieved by network/media strategies, such as ‘email invitations’ and ‘word of mouth’ (66%–71% vs 60% achieved by Facebook). There was a significant association between recruitment strategy and conversion to randomisation (p=0.011), but not retention. There was also a significant association between ethnicity and recruitment strategies used (p<0.001).

    View this table:
    Table 3

    Effectiveness and cost of study recruitment campaigns

    Promotions via networks and paper media were the least expensive strategy (NZ$4 per randomised participant, table 3), closely followed by Facebook posts (NZ$5 per participant). Radio advertising was the most costly strategy used (NZ$179 per randomised participant).

    Discussion

    This study describes the effectiveness of a range of recruitment strategies used in a smartphone-delivered nutrition labelling intervention trial. Over a period of 13 months, 2104 participants were registered and provided information on recruitment source, of whom 55% were randomised into the study (n=1357). Facebook campaigns were the most successful recruitment strategy, both overall (43% of all randomised participants) and for Māori and Pacific participants (44% each). Although the conversion rate from registration to randomisation for participants recruited via Facebook was not as high as that achieved by network promotions, the vast reach of Facebook (see online supplementary file 3) and ability to target campaigns by demographics, geographic region and interests led to the greatest absolute number of study registrations (n=966) and randomisations (n=584).

    Supplementary Material

    Supplementary data

    Most types of campaign were used several times during the recruitment period. The Pacific radio campaign and advertising in a Māori magazine (and via their website) were only used once; however (due to cost), during the period that study recruitment was open only to Māori and Pacific. Therefore, although it is possible that other ethnic groups might also have been attracted by these campaigns they would have been deemed ineligible on registration, and so these strategies recruited Māori or Pacific participants exclusively.

    The study sample was not a representative of the general population and contained a high proportion of tertiary educated adults (66%). One possible explanation for this is utilisation of the professional networks of the study team and university mailing lists. In addition, most of the paid Facebook campaigns were placed on the University of Auckland Facebook page and while the audience was not limited to the subscribers of this page, interest in University Facebook posts is likely to be higher among tertiary educated adults.

    The overall dropout rate was 11% (completion rate 89%), which is lower than the typical >20% dropout rate expected for lifestyle interventions.7 This could have been due to some of the unique characteristics of this study, including the offer of a quite substantial (NZ$80) financial incentive on completion, the relatively short 5-week study duration, the automated nature of the intervention which removed the need to travel to attend appointments and use of a prerandomisation run-in period. A run-in period allowed participants to become familiar with the smartphone technology, potentially selected more dedicated users and allowed exclusion of users with incompatible devices. In total, more than 700 people were excluded following the run-in phase.

    Overall the results of this study are consistent with previous findings showing that Facebook is an effective research recruitment method,7 radio advertising is less cost effective23 and combined approaches are best.23 Higher female participation rates, particularly of those with higher income and education levels, are also typical for nutrition research.24 A previous study that compared demographic characteristics of participants recruited via either social media or traditional methods found no difference between groups other than in age, which was younger in the social media group.25 A similar association between age and recruitment strategy was apparent in our study (p<0.001), with a lower mean age among participants recruited via Facebook (31 years), compared with those recruited through newspapers and radio (37 and 36 years, respectively). Our analysis also demonstrated a significant association between recruitment strategy and other important demographic characteristics, such as gender, ethnicity and household size. It is important, therefore, to tailor study recruitment approaches to the target population. Our findings also have broader relevance for the health promotion and public health fields which increasingly uses smartphone technology and applications to deliver and monitor healthcare interventions, for education, and to support consumer behaviour change.26 27

    Despite using a wide range of culturally-targeted media-based strategies and additional resources to recruit Māori and Pacific participants, targets were still not met. Face-to-face recruitment building on community networks and connections is a strategy commonly used to recruit Māori and Pacific participants into studies,13 and it was strongly recommended by Māori and Pacific team members as a way to enhance recruitment of Māori and Pacific participants. The fact that ‘word of mouth’ was the second most effective strategy is an indication of the potential effectiveness of such face-to-face community-based recruitment. However, as this trial recruited people from all across NZ in-person recruitment was considered to be too logistically challenging and resource intensive. Furthermore, the potential for in-person recruitment to introduce selection bias was considered a risk to the internal validity of the study since those recruited using face-to-face methods may differ in other important ways from those recruited using alternative strategies as has been observed in other studies.28 For research to be truly representative of Māori and Pacific peoples, it is clear from this work that in future research, recruitment protocols and indeed study design need to be carefully planned and adapted to accommodate different cultural perspectives.

    Studies focused on recruitment of underserved or hard-to-reach populations consistently report that greater resources, more time and targeted strategies are needed to recruit such populations.29 On the other hand, recruitment targets for other ethnicities could potentially be achieved with substantially less cost, thus freeing up more time and resources to focus on more priority population recruitment. Facebook recruitment alone achieved 88% (n=440) of the target for non-Māori and non-Pacific participants, almost eliminating the need to use any other recruitment strategies for this group.

    To our knowledge, this is the first study to examine the success of a range of strategies to recruit participants to a smartphone-delivered study in NZ. Use of diagnostic technology, namely Google Analytics, enabled objective assessment of web-based recruitment strategies and their effectiveness. The current study also provides data on participant retention, which was previously identified as a gap among existing reports on recruitment for web-based and mobile health studies.7 We also describe strategies for reducing duplicate and fake registrations.

    A limitation was the broad grouping used for self-reported recruitment sources, which prevented analysis at a more precise category level. Inability to estimate staffing costs associated with recruitment strategies was another limitation, likely to be particularly important in assessing research team network promotions, a heterogeneous approach that involved reasonably high resource for example, distribution of hard copy advertising fliers.

    In conclusion, recruitment via the internet and social media is comparable in cost and substantially more effective than traditional study recruitment strategies such as mainstream media advertising, and it is effective in reaching diverse ethnic groups. However, additional targeted strategies should be considered where large number of participants from particular ethnic groups are sought.

    Acknowledgments

    We thank the following organisations and teams for promoting our study to their network and assisting with recruitment: Toi Tangata, Le Va, Pacific Medical Association, Tu Kotahi Māori Asthma Trust and Kokiri Marae, University of Otago, Victoria University, Unitec Auckland, National Heart Foundation, Health Promotion Agency, Cancer Society, Stroke Foundation, Pro Care, Agency for Nutrition Action, Auckland University of Technology, Nutrition Foundation, Diabetes NZ and other research groups at the University of Auckland. We also thank the members of the DIET Programme Advisory Group for promoting the study to their networks. We are grateful to Dr Lisa Te Morenga for her feedback on an earlier version of this manuscript.

    References

    1. 1.↵
      2. Treweek S ,
      3. Lockhart P ,
      4. Pitkethly M , et al
      . Methods to improve recruitment to randomised controlled trials: cochrane systematic review and meta-analysis. BMJ Open 2013;3:e002360–26.doi:10.1136/bmjopen-2012-002360
      OpenUrlAbstract/FREE Full Text
    2. 2.↵
      2. Marcano Belisario JS ,
      3. Bruggeling MN ,
      4. Gunn LH , et al
      . Interventions for recruiting smokers into cessation programmes. Cochrane Database Syst Rev 2012;12:CD009187.doi:10.1002/14651858.CD009187.pub2
      OpenUrlPubMed
    3. 3.↵
      2. Huynh L ,
      3. Johns B ,
      4. Liu SH , et al
      . Cost-effectiveness of health research study participant recruitment strategies: a systematic review. Clin Trials 2014;11:576–83.doi:10.1177/1740774514540371
      OpenUrlCrossRefPubMed
    4. 4.↵
      2. Varnfield M ,
      3. Karunanithi M ,
      4. Lee CK , et al
      . Smartphone-based home care model improved use of cardiac rehabilitation in postmyocardial infarction patients: results from a randomised controlled trial. Heart 2014;100:1770–9.doi:10.1136/heartjnl-2014-305783
      OpenUrlAbstract/FREE Full Text
    5. 5.↵
      2. Payne HE ,
      3. Lister C ,
      4. West JH , et al
      . Behavioral functionality of mobile apps in health interventions: a systematic review of the literature. JMIR Mhealth Uhealth 2015;3:e20.doi:10.2196/mhealth.3335
      OpenUrl
    6. 6.↵
      2. Wayne N ,
      3. Ritvo P
      . Smartphone-enabled health coach intervention for people with diabetes from a modest socioeconomic strata community: single-arm longitudinal feasibility study. J Med Internet Res 2014;16:e149.doi:10.2196/jmir.3180
      OpenUrlCrossRefPubMed
    7. 7.↵
      2. Lane TS ,
      3. Armin J ,
      4. Gordon JS
      . Online recruitment methods for web-based and mobile health studies: a review of the literature. J Med Internet Res 2015;17:e183.doi:10.2196/jmir.4359
      OpenUrlCrossRefPubMed
    8. 8.↵
      2. Loxton D ,
      3. Powers J ,
      4. Anderson AE , et al
      . Online and offline recruitment of young women for a longitudinal health survey: findings from the Australian longitudinal study on women's health 1989-95 cohort. J Med Internet Res 2015;17:e109.doi:10.2196/jmir.4261
      OpenUrlCrossRefPubMed
    9. 9.↵
      2. Ramo DE ,
      3. Rodriguez TM ,
      4. Chavez K , et al
      . Facebook recruitment of young adult smokers for a cessation trial: methods, metrics, and lessons learned. Internet Interv 2014;1:58–64.doi:10.1016/j.invent.2014.05.001
      OpenUrlCrossRefPubMed
    10. 10.↵
      2. Ünlü Ince B ,
      3. Cuijpers P ,
      4. van 't Hof E , et al
      . Reaching and recruiting Turkish migrants for a clinical trial through Facebook: a process evaluation. Internet Interventions 2014;1:74–83.doi:10.1016/j.invent.2014.05.003
      OpenUrl
    11. 11.↵
      Ministry of Health.  New Zealand Health Survey: annual update of key findings. Wellington: BMJ Publishing Group, 2013.
    12. 12.↵
      2. Selak V ,
      3. Crengle S ,
      4. Elley CR , et al
      . Recruiting equal numbers of indigenous and non-indigenous participants to a 'polypill' randomized trial. Int J Equity Health 2013;12:44.doi:10.1186/1475-9276-12-44
      OpenUrlCrossRefPubMed
    13. 13.↵
      2. Mhurchu CN ,
      3. Blakely T ,
      4. Funaki-Tahifote M , et al
      . Inclusion of indigenous and ethnic minority populations in intervention trials: challenges and strategies in a New Zealand supermarket study. J Epidemiol Community Health 2009;63:850–5.doi:10.1136/jech.2008.081109
      OpenUrlAbstract/FREE Full Text
    14. 14.↵
      2. Ni Mhurchu C ,
      3. Volkova E ,
      4. Jiang Y , et al
      . Effects of interpretive nutrition labels on consumer food purchases: the Starlight randomized controlled trial. Am J Clin Nutr 2017;105:695–704.doi:10.3945/ajcn.116.144956
      OpenUrlAbstract/FREE Full Text
    15. 15.↵
      2. Volkova E ,
      3. Neal B ,
      4. Rayner M , et al
      . Effects of interpretive front-of-pack nutrition labels on food purchases: protocol for the starlight randomised controlled trial. BMC Public Health 2014;14:968.doi:10.1186/1471-2458-14-968
      OpenUrlCrossRefPubMed
    16. 16.↵
      2. Volkova E ,
      3. Li N ,
      4. Dunford E , et al
      . "Smart" RCTs: development of a smartphone app for fully automated nutrition-labeling intervention trials. JMIR Mhealth Uhealth 2016;4:e23.doi:10.2196/mhealth.5219
      OpenUrl
    17. 17.↵
      Guide to creating a front of pack (FoP) nutrition label for pre-packed products sold through retail outlets. 2013 https://www.gov.uk/government/publications/front-of-pack-nutrition-labelling-guidancehttp://www.webcitation.org/6bOrPp9z5.
    18. 18.↵
    19. 19.↵
    20. 20.↵
    21. 21.↵
    22. 22.↵
      Statistics New Zealand. New Zealand Census of Population and Dwellings: BMJ Publishing Group, 2013. http://www.stats.govt.nz/Census/2013-census.aspx.
    23. 23.↵
      2. Lam E ,
      3. Partridge SR ,
      4. Allman-Farinelli M
      . Strategies for successful recruitment of young adults to healthy lifestyle programmes for the prevention of weight gain: a systematic review. Obes Rev 2016;17:178–200.doi:10.1111/obr.12350
      OpenUrl
    24. 24.↵
      2. Cowburn G ,
      3. Stockley L
      . Consumer understanding and use of nutrition labelling: a systematic review. Public Health Nutr 2005;8:21–8.doi:10.1079/PHN2005666
      OpenUrlCrossRefPubMedWeb of Science
    25. 25.↵
      2. Frandsen M ,
      3. Walters J ,
      4. Ferguson SG
      . Exploring the viability of using online social media advertising as a recruitment method for smoking cessation clinical trials. Nicotine Tob Res 2014;16:247–51.doi:10.1093/ntr/ntt157
      OpenUrlCrossRefPubMedWeb of Science
    26. 26.↵
      2. Bert F ,
      3. Giacometti M ,
      4. Gualano MR , et al
      . Smartphones and health promotion: a review of the evidence. J Med Syst 2014;38:9995.doi:10.1007/s10916-013-9995-7
      OpenUrlCrossRefPubMed
    27. 27.↵
      2. Kratzke C ,
      3. Cox C
      . Smartphone technology and apps: rapidly changing health promotion. Global J Health Educ Promo 2012;15.
    28. 28.↵
      2. Blakely T ,
      3. Ni Mhurchu C ,
      4. Jiang Y , et al
      . Do effects of price discounts and nutrition education on food purchases vary by ethnicity, income and education? Results from a randomised, controlled trial. J Epidemiol Community Health 2011;65:902–8.doi:10.1136/jech.2010.118588
      OpenUrlAbstract/FREE Full Text
    29. 29.↵
      2. Bonevski B ,
      3. Randell M ,
      4. Paul C , et al
      . Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol 2014;14:42.doi:10.1186/1471-2288-14-42
      OpenUrlCrossRefPubMed

    Footnotes

    • Contributors EV and CNM formulated the idea and planned the reported poststudy analysis; CNM had leadership responsibility for the main study design and execution; EV and HE contributed substantially to design of the main study; EV and JM managed day-to-day conduct of the study and oversaw data collection; EV and YJ performed the statistical analyses of the study data; CC and GS contributed to study design and revised the paper critically from cultural perspectives; EV wrote the paper and had primary responsibility for final content. All authors provided critical review and commentary on the draft manuscript and approved the final manuscript.

    • Funding This was an investigator-initiated study funded by a Health Research Council of New Zealand programme grant (13/724). The Health Research Council had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

    • Competing interests None declared.

    • Patient consent Obtained.

    • Ethics approval University of Auckland Human Participants Ethics Committee.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Additional data are available by emailing CNM at c.nimhurchu@auckland.ac.nz.