mHealth: providing a mindfulness app for women with chronic pelvic pain in gynaecology outpatient clinics: qualitative data analysis of user experience and lessons learnt

Objectives To determine whether a pre-existing smartphone app to teach mindfulness meditation is acceptable to women with chronic pelvic pain (CPP) and can be integrated into clinical practice within the National Health Service (NHS) CPP pathways, and to inform the design of a potential randomised clinical trial. Design A prestudy patient and public involvement (PPI) group to collect feedback on the acceptability of the existing app and study design was followed by a three-arm randomised feasibility trial. In addition, we undertook interviews and focus groups with patients and staff to explore app usability and acceptability. We also obtained participant comments on the research process, such as acceptability of the study questionnaires. Setting Two gynaecology clinics within Barts Health NHS, London, UK. Participants Patients with CPP lasting ≥6 months with access to smartphone or personal computer and understanding of basic English. Intervention The intervention was mindfulness meditation content plus additional pain module delivered by a smartphone app. Active controls received muscle relaxation content from the same app. Passive (waiting list) controls received usual care. Main outcome measures Themes on user feedback, app usability and integration, and reasons for using/not using the app. Results The use of the app was low in both active groups. Patients in the prestudy PPI group, all volunteers, were enthusiastic about the app (convenience, content, portability, flexibility, ease of use). Women contributing to the interview or focus group data (n=14), from a ‘real world’ clinic (some not regular app users), were less positive, citing as barriers lack of opportunities/motivation to use the app and lack of familiarity and capabilities with technology. Staff (n=7) were concerned about the potential need for extra support for them and for the patients, and considered the app needed organisational backing and peer acceptance. Conclusion The opinions of prestudy PPI volunteers meeting in their private time may not represent those of patients recruited at a routine clinic appointment. It may be more successful to codesign/codevelop an app with typical users than to adapt existing apps for use in real-world clinical populations. Trial registration number ISRCTN10925965.

To determine whether a pre-existing smartphone App to teach Mindfulness Meditation is acceptable to women with chronic pelvic pain (CPP) and can be integrated into clinical practice within NHS CPP pathways. To inform the design of a potential randomised clinical trial.

Design
A pre-study patient and public involvement (PPI) group to collect feedback on the acceptability of the existing app and study design was followed by a three-arm randomised feasibility trial.
In addition, we undertook interviews and focus groups with patients and staff to explore app usability and acceptability.

Setting
Two gynaecology clinics within Barts Health NHS, London, UK.

Participants
Patients with CPP lasting >6 months with access to smartphone or PC and understanding of basic English.

Intervention
The intervention was mindfulness meditation content plus additional pain module delivered by smartphone app, active controls received muscle relaxation content by the same app. Passive (waiting list) controls received usual care.

Main outcome measures
Themes on user feedback, app usability and integration and reasons for using/not using the app.

Results
App use was low in both active groups.
Patients in the pre-study PPI group, all volunteers, were enthusiastic about the app (convenience, content, portability, flexibility, ease of use). Women contributing to the interview or focus group data (n=14), from a 'real world' clinic, (some not regular app users) were less positive, citing as barriers lack of opportunities/motivation to use the app, and lack of familiarity Smartphone health apps, as one form of mhealth (2), are popular in the UK, our study setting, with more than two-thirds of the UK population using smartphones (3,4,5). Health apps are one of the fastest growing app categories, thus numbers of users are still increasing (6).
Currently these apps are usually developed either by researchers or (in the majority) by commercial companies, without collaboration between these groups (7,8). The lack of interaction between researchers and commercial developers in the field of pain-related apps has led to a situation where commercially available apps have not been scientifically validated and apps that have been developed from research projects are not commercially available (9).
We were interested in using an app to support women with chronic pelvic pain (CPP) in a clinical setting, where validation of an intervention is important to ensure best care. CPP is defined as a subjective physical and emotional experience of pain in the pelvic area that has been present for at least six months that may or may not have an identifiable pathology. CPP affects up to 24% of women worldwide (10) and accounts for 20% of gynaecological clinic referrals. (11,12) It has considerable impact on patients' quality of life and their income (13), and annual costs to the NHS have been estimated at approximately £326 million. (14) CPP is especially common in younger women, who may be categorised as digital natives, making an app-based intervention particularly appropriate. Despite costly interventions, CPP is often resistant to surgical and medical treatment and appears to respond better to a multimodal, holistic approach, (15) with a focus on coping strategies. Apps with such a focus have been shown to be beneficial in various conditions (16). As one example, evidence from uncontrolled trials (17,18) suggests positive effects of mindfulness meditation (MM) as a coping strategy in CPP. We therefore chose to evaluate MM delivered via an app to women with CPP as our intervention.
Mindfulness meditation (MM) depends on activating the psychological state of mindfulness.
This refers to an awareness that emerges by way of paying attention intentionally and nonjudgementally, in the present moment, to the unfolding of the moment-by-moment experience.
Generally two main complementary approaches have been used for mindfulness meditation 1) exercises focusing attention and 2) monitoring of experiences in the present moment. While systematic reviews show that mindfulness meditation may have positive effects on depression, quality of life and pain symptoms in patients with chronic pain (16,19,20) none of the reviewed papers included meditation delivered via mobile phone apps or women with CPP.
Evaluation of an existing app is often appropriate (21) and is both quicker and more costeffective than designing an app from scratch. We chose to evaluate an existing commercial app platform that teaches mindfulness by guided meditation (Headspace ®), with a ten day basic meditation module followed by a pain module specifically designed for MEMPHIS, the name we gave to the study. The Headspace app was publicly nominated favourite health app We undertook a feasibility study to assess whether or not to proceed with a full randomised controlled trial of a modified Headspace meditation app for women with CPP. In this paper we report on the qualitative interview and focus group data from this study; the protocol and quantitative results have been published/ submitted (1,25).

Methods
The MEMPHIS trial was a three-arm parallel randomised feasibility trial approved by Camden and Kings Cross Research Ethics Committee in 2016 (15/LO/1967). Objectives for the qualitative part of this study were to consider: 1) The acceptability, use and usability of the app in the intended service user population and for health care professionals (doctors, health care assistants, clinical and research nurses) 2) The feasibility of integrating such an app into existing healthcare pathways.

Patient and Public Involvement
We held a Patient and Public Involvement (PPI) group workshop before the study to discuss acceptability of the Headspace app and help us design our study. Women attending the Royal London Hospital CPP clinic were invited to volunteer to use the unmodified Headspace app for a week and then feed back in an evening discussion group. Women were not involved in the design of the modified app. Two patient representatives provided support from the study design stage through recruitment to the interpretation of the results and regularly attended Trial Management Group meetings. The trial recruited at two outpatient gynaecology clinics within Barts Health NHS trust in two separate deprived areas of inner East London. Female patients with new or follow-up gynaecology appointments were assessed for eligibility by a researcher in clinic, having been posted a Patient Information Sheet. Women were eligible if they had been suffering with CPP for 6 months or more and had at least a basic understanding of the English language. Women were excluded if they did not meet these criteria or they did not have access to a smartphone or personal computer or were currently using the Headspace app (there were very few of the latter). All patients gave full and informed consent to be randomised and data was collected through all stages of the study.

Study recruitment
For the study of quantitative data, 90 patients were allocated randomly in a 1:1:1 ratio to the mindfulness meditation app, a muscle relaxation app active control or the usual care arm.
Patients in the two active arms were asked to download the modified app in the clinic with support from a research staff member and were sent a questionnaire about app usability, an analysis of which is reported in a companion paper. (25) We used these data to inform topic guides for the qualitative part of the study, i.e. our outlines of key issues and areas of questioning that were used to guide our semi-structured interviews and focus groups with patients and staff.

Interviews and focus groups
All patients in the two active arms were invited to one of two focus groups at their own study site after the 6-month follow up. We offered telephone interviews as an alternative. All staff participating in the study were invited to attend a staff focus group overseen by the patient  Theory (NPT) toolkit; NPT is a theory of implementation practices (26). Patients were also asked to 'walk through' the app with researchers and comment on its different specific features, (27) as well as feeding back on our study process. Walkthroughs are often used in developing technologies such as mhealth; patients 'walked through' use of the app, articulating their thought processes while they did so (27). This helped to identify issues or barriers to use of the app from the users' points of view without the need for technical discussions. Results for the walkthrough, showing comments on different features specific to the usability of the intervention app used in our study are shown in Appendix 1; walkthroughs were undertaken by two patients. All data were audio-recorded at point of collection and transcribed, with personal identifying data removed from transcripts. Raw data were stored in a Primary Care Clinical Trial Unit database to clinical trial standards.

Analysis
Analysis was carried out blinded as to which study app was used, and deployed the immersion-crystallisation method (28). Thus, the lead qualitative researcher immersed herself in the data, reading transcripts carefully, then writing down articulated or crystallised patterns or themes that related to the aims and research questions of the study. These were discussed with another researcher from the team, and themes modified as appropriate. This process was repeated until all the data had been examined and all patterns that had been noticed were articulated, discussed and substantiated with exemplar extracts. This approach was considered appropriate since we had a small dataset and we were not aiming to develop conceptual themes but rather to inform the design and development of a randomised controlled trial for the modified app.
We used the SRQR checklist when writing our report (29).

Results
We screened 488 women between May and September 2016 for their eligibility to participate in the study. After exclusions, 90 women gave full consent to participate and were randomised to the intervention arm (31 women), the active control arm (30 women) or the usual care arm (29 women).

Demographics
Women were aged a mean of 35 years, 66% were employed and overall approximately 50% had stayed in full time education until at least the age of 20 though the proportion was least, at 36.7% in the intervention arm. Overall 44% were of white ethnicity though the proportion was lowest in the intervention arm (35.7%) and highest in the usual care arm (53.6%). The second most common self-reported ethnic groups were 'Southern Asian' and 'Black'. Women in the intervention arm were most likely to have experienced CPP for 3-5 years (40.3% of this group), women in the usual care group for over 10 years (42.9% of this group). More women had pain for longer than two years in the intervention arm than in either of the other two arms.
All women reported a high pain intensity, with means of 6.8 to 6.9 in the previous week (on a scale from 1-10). Other demographic data are reported in Forbes et al. (25) Our qualitative sample was taken from the two active arms but we did not record separate demographic data for the women in this smaller sample.

Pre-study PPI group
The ten women in the pre-study PPI group were self-selected local women who were familiar with using apps. They anticipated no technical issues even in women who were not used to apps. They considered that the Headspace app would be successfully adopted by patients taking part in the study, given that a smartphone, like CPP, is 'always with you'. They praised the flexibility of the app, welcomed its portability and were unanimous in saying it was easy to slip off for 10 minutes when at work to use it. As a result, they found they could use it at times when they most needed pain relief as well as to prevent pain and found the app helpful in relieving pain and stress. The group reported being able to meditate without the app, once they had tried it with the app; however, they still preferred to use the app because they found the voice soothing.

App usage in the study
Patient adherence to the app was less than expected from our pre-study PPI group discussions. (25) Only 36% of meditation app patients and 46% of the active control patients used the app at least once. (25)

Thematic analysis
Qualitative data were obtained from 14 patients; 12 preferred a telephone interview, two attended face-to-face interviews, one participant at the University attached to one of the recruiting clinics and one participant at the other recruiting hospital. Patients chose not to attend focus groups. Seven people attended the staff focus group: two recruiting nurses, three clinic nurses one consultant and a representative from the pelvic pain support network.
The qualitative analysis revealed three main themes from all participants combined (the analyst remained blind to app group) regarding usability, and four subthemes:

Familiarity and capabilities with app technology
Around half of the patients were sufficiently familiar with technology and apps to be comfortable using the study apps. However, six participants reported difficulty with them because they were "not very good at technology", or were unsure how to get started or use the app effectively without help.

I am not good with technical some things that is why the problems I had arisen, right okay. So I consulted with my daughter and she helped me work it out… so I don't try everything. (patient 1002)
One patient (1001) was not used to technical app language; 'help' suggested emotional support to her, for example. Two more changed handsets and therefore did not continue with the app. In all cases these technical difficulties appeared to lead to abandoning of the app or restricted use of its functionality.
Five patients having technical problems suggested possible solutions such as a 'class' or group for first time users, a YouTube orientation video, or a pictorial leaflet. This might include an introduction to meditation and mindfulness as well as the app itself. One woman commented: "If your market is targeting people who are not using apps then you are going to have to get together and find ways to do this", (patient 1041); she also suggested we could get ideas from other apps on the market in this regard. Given their experiences in the study, staff were concerned about additional staff time needed to support women in using the app. This would sit in tension with one of the original rationales behind choosing an app as the mode of delivery, which was to increase the effective use of staff contact time with patients. Language barriers might compound problems.

Motivations to use the app
Staff, though unaware of the low sustained app use in the study, felt it would be common sense to hold occasional motivating meetings with patients if the intention was for them to use the app long term. The patient data suggested the main motivators or lack of motivation for using the app in our sample, which could be drawn on in such meetings, and which we now consider.

Perceived benefits
Three patients said they only entered the trial to help others through research but were already using alternative forms of pain control. They explained that this meant they were not motivated to actually use the app, perceiving the relative benefit to be small. The failure of such altruism to extend to using the app is a recognised phenomenon in clinical trials that has been called 'weak altruism'. (30) Thus, only one of these three patients persevered. There was no clear pattern regarding the impact of current pain on app use by patients. Six said they used it regardless of pain intensity -sometimes developing a daily routine -while four only used it when in severe pain or expecting to be (e.g. during menstruation). This cyclical or intermittent use in some patients should be considered when looking at our study outcomes. (25) The Headspace app requires regular use to learn and benefit from psychological techniques. To address this, healthcare professional alerts have been effective in other studies, (31) whilst Headspace only has a reminder function that the user can set.
This was often not sufficient, as one patient said even with this feature, "To be quite honest I

used it a couple of times and then forgot. And then I [remembered it and] used it more frequently." (patient 1036)
One patient said medication was not working but the app did, though she was not sure whether this was 'mind over matter' (1065), which was her term for a placebo effect. Three others said it did not reduce their pain. Other benefits were considered good reason for using the app even when participants did not feel that it reduced pain intensity. Alternative or unanticipated benefits were not formally measured or taken into account in the study's effectiveness outcomes (25). For example, ten patients valued the way the app helped them to relax or destress or focus and re-assess their life; three of these specifically said they used it to induce a relaxed state to get to sleep. Notably the active control was a relaxation app, so it may be that most of the patients interviewed were in this study arm; we were blind to this. One participant said she did not like the focus on pain per se as her condition impacted on various areas of her life. Even when the app was positively received, women might stop using it because it was too powerful, and they had gained the change they wanted: Three patients found the app put them more in tune with their bodies and their breathing, (two of these were among those who also found the app de-stressing) while another found yoga better for that. Six patients, like pre-study PPI group members, also learned to use techniques from the app to alter their stress patterns without the app, having tried it, for example in traffic or by sitting down and taking time out or for general relaxation. Four of these came from the group of ten patients that reported de-stressing as the app's main benefit, and it is not clear whether they had the intervention app or the active control.

Relation to other therapies
Two patients preferred 'pure' meditation. Another considered the app to be "very much about meditation". It may be that the former two were in the active control and the latter in the intervention arm. An alternative therapy practitioner and two further patients reported that they preferred yoga; one said this was because it focussed on each part of the body in turn.
Three patients thought the app was useful as an adjunct to other methods rather than a replacement for them, for example physical interventions such as Pilates, or listening to classical music. Life getting in the way Seven patients revealed they preferred to use the app in the evenings because of other life commitments. This meant they did not always use it as a direct response to pain, reducing its potential for contemporaneous effect. One patient who used it in response to pain but only used it once or twice blamed this on having no spare time because of juggling work and children; however, another patient managed despite such commitments.

Barriers to integration for staff
Staff believed that the biggest barrier to clinical adoption of the app was a possible lack of support from the host organisation. It might also be hard to integrate the app within existing professional work practices if the staff in the position of offering the app to patients failed to see its relative advantage over other interventions. Collection of feedback on the app's effectiveness would be necessary for staff to support sustained use. It was felt that staff would need training on how to introduce the app to women in practice, and that complexity and high staff turnover could impede sustained use. An app was also seen as impersonal compared with face-to-face contact, which was more favoured by staff.

Participant comments on the research process
The study questionnaires that were used for the main quantitative outcome measures (25) were acceptable to patients except for some discomfort with a question about sex, which patients considered a delicate question that was missing a 'no sex' option. Most preferred a

Discussion
Our study adds to the limited evidence on mHealth app user behaviour and experience (32,33). We were able to explore reasons for low app usage in our feasibility study. (25) The thematic analysis of qualitative data from this study suggests that the low app use in the trial occurred because many patients were not familiar with apps in general or lacked capabilities with technology. Women also stated limited motivation to use the app because of a lack of perceived benefit, or a lack of opportunity to use the app due to Wi-Fi issues or due to other commitments.
Similar findings were reported by Laurie et al. (33)  Our study has also shown that clear consideration of unexpected benefits should be included in future studies and these can be informed by our finding that benefits for patients may be more diffuse than anticipated (e.g. app relieving stress rather than pain). Though we were blind as to whether the patients we interviewed were in the active arm or the intervention arm of the study, there is indication from responses (especially in the patients who did a walkthrough) that many with these extra benefits were in the intervention arm. Our data also suggest that staff benefits may be less than anticipated, as participants sometimes needed a lot of support and scaffolding in technology use at least initially.
Young age, co-morbid anxiety and low educational attainment are predictors for dropping out of web-based interventions according to studies in the field of depression. (46,47)  Our data suggest that it is important to involve real world end users in the agile design or development or modification of apps in close collaboration with researchers and commercial app developers. (8) Although the evaluation of existing apps has been recommended as a cost-effective and rapid process, (21) our findings suggest that in actual clinical practice these may be problematic.

Strengths and weaknesses of the study and in relation to other studies
One strength of this study is that it creates much-needed evidence in the field of evaluating existing health apps in a clinic population (7,9,21) and recording user experience. This provides us with lessons to be learned.
Researchers conducting interviews and focus groups were: a senior mixed methods medical sociology researcher, a recruiting nurse, a representative from the pelvic pain support network and an experienced health psychologist. Findings were similar across the data and the different backgrounds of the researchers therefore does not appear to have influenced findings. The main analysis was undertaken by the medical sociologist and so the concordance with the COM-B model is not due to background discipline bias.
We were able to recruit successfully, and we obtained valuable information from patients with CPP, who were recruited from a deprived urban area of the UK as typical local clinical patients.
However we report a marked discrepancy between the attitudes of the pre-study PPI group of volunteer patients from the local area, who actively put themselves forward for a 7-day trial of the app, and the participants asked to take part when they attended clinics. The opinions of pre-study PPI volunteers meeting in their private time may not be representative of the opinions of patients recruited at a routine clinic appointment. Women in the PPI group were used to using apps, which had led them to be interested in the study in the first place. Whereas women in the PPI group had all trialled the app at home and work without support from us, many patients from clinic were unable to use their phone beyond calls, texts and photos.
To our knowledge the present observation on failure of PPI work to translate into practice in a trial has not been formally reported before, and is lacking from a recent comprehensive systematic review.(48) PPI involvement is a stipulated requirement when applying for some funding, and the present research findings should be taken into account when drafting guidelines for future PPI involvement in study planning. PPI groups are able to provide significant help and advice in any study but our findings shows the value of adding agile codevelopment as a requirement for app intervention development as likely to provide a more effective intervention than one informed by PPI alone.

Implications for clinicians and policymakers
Given the patchy use of the app and the way that some participants did not manage to unlock its full functionality, and an indication of diffusion of benefit, more work is needed to see whether the app reduces pain per se. This study is a good example of the need to move away from 'one size fits all' behavioural interventions. Future studies should do more work on implementation before doing an effectiveness trial. This will enable researchers to be more nuanced about saying who the app is effective for, if at all.
Strategies to involve busy, less motivated, and less technologically experienced individuals in PPI and lay app design groups need to be further developed. These groups should include considerable scaffolding, which we have shown extends to study involvement by patients.
More care is also needed to obtain PPI input that is representative of the study group with typical users of the target group, taking into account their capabilities, opportunities and motivational aspects. Moreover, we can confirm a recent review suggesting that health apps should be co-designed with users (8), rather than presenting them with a pre-existing app.
These implications for our study are also generalisable to other technology studies.
Our study did not show the app was ineffective but rather that we could not show any possible effectiveness. Thus, further work is needed on MM app effectiveness in clinical subgroups such as our local users, bearing in mind also that diffuse benefits were found.

Research in context
What is known CC was a researcher and contributed to the study design BK and GF provided statistical support and ran the statistical analysis.
CR and LS led the interview and focus group field work and analysis. All authors provided support throughout the trial and contributed towards the final paper.

Role of the funding source
The UK National Institute of Health Research, Research for Patient Benefit (RfPB No. PB-PG1013-32025) funded the MEMPHIS study. The funder had no role in the study design, in the collection, analysis, and interpretation of the data, in the writing of this report, or in the decision to submit the paper for publication. The first and last authors vouch for the integrity, completeness and accuracy of the data and analyses, and for the fidelity of this report to the protocol and statistical analysis plan. The views and opinions expressed herein are those of the authors and do not necessarily reflect those of the RfPB, NIHR, NHS or the Department of Health.

Competing interests
We have read and understood BMJ policy on declaration of interests.

Transparency
Three participants were frustrated and even 'angry' and 'patronised' at having to do the same (basic) meditation several times before moving up a stage.
There was no indication of goals, duration of each level or gamification rewards.
Goal setting is recommended by psychological theory and could be as simple as using "day 1 of 30" rather than simply Day 1.
Changing emojis could show progress, for example from a sad face to a smiley face to a heart; the inbuilt progress function was not felt sufficiently motivating.

Introduction
Experienced app users suggested the introduction could be improved for initiates overview of the entire app and its levels suggested for the start.
One participant had done mindfulness before but thought an introduction to mindfulness might be helpful for others.

Instructions to authors
Complete this checklist by entering the page numbers from your manuscript where readers will find each of the items listed below.
Your article may not currently address all the items on the checklist. Please modify your text to include the missing information. If you are certain that an item does not apply, please write "n/a" and provide a short explanation.
Upload your completed checklist as an extra file when you submit to a journal.
In your methods section, say that you used the SRQR reporting guidelines, and cite them as:

Objective
To determine whether a pre-existing smartphone App to teach Mindfulness Meditation is acceptable to women with chronic pelvic pain (CPP) and can be integrated into clinical practice within NHS CPP pathways. To inform the design of a potential randomised clinical trial.

Design
A pre-study patient and public involvement (PPI) group to collect feedback on the acceptability of the existing app and study design was followed by a three-arm randomised feasibility trial. In addition, we undertook interviews and focus groups with patients and staff to explore app usability and acceptability. We also obtained participant comments on the research process, such as acceptability of the study questionnaires.

Setting
Two gynaecology clinics within Barts Health NHS, London, UK.

Participants
Patients with CPP lasting >6 months with access to smartphone or PC and understanding of basic English.

Intervention
The intervention was mindfulness meditation content plus additional pain module delivered by smartphone app, active controls received muscle relaxation content by the same app.
Passive (waiting list) controls received usual care.

Main outcome measures
Themes on user feedback, app usability and integration and reasons for using/not using the app.

Results
App use was low in both active groups.
Patients in the pre-study PPI group, all volunteers, were enthusiastic about the app (convenience, content, portability, flexibility, ease of use). Women contributing to the were less positive, citing as barriers lack of opportunities/motivation to use the app, and lack of familiarity and capabilities with technology. Staff (n=7) were concerned about the potential need for extra support for patients and staff and considered the app needed organisational backing and peer acceptance.

Conclusion
The opinions of pre-study PPI volunteers meeting in their private time may not represent those of patients recruited at a routine clinic appointment.
It may be more successful to co-design/co-develop an app with typical users than to adapt existing apps for use in real-world clinical populations.

Trial registration and funding
The trial (ISRCTN 10925965) was funded by the UK National Institute of Health Research, Research for Patient Benefit programme (RfPB PB-PG-1013-32025).

Introduction
Smartphone health apps, as one form of mhealth (1), are popular in the UK, our study setting. With more than two-thirds of the UK population using smartphones (2,3,4), health apps are one of the fastest growing app categories, thus numbers of users are still increasing (5). Currently these apps are usually developed either by researchers or (in the majority) by commercial companies, without collaboration between these groups (6,7). The lack of interaction between researchers and commercial developers in the field of painrelated apps has led to a situation where commercially available apps have not been scientifically validated and apps that have been developed from research projects are not commercially available (8). We were interested in using an app to support women with chronic pelvic pain (CPP) in a clinical setting, where validation of an intervention is important to ensure best care. CPP is defined as a subjective physical and emotional experience of pain in the pelvic area that has been present for at least six months that may or may not have an identifiable pathology (9).
CPP affects up to 24% of women worldwide (10) and accounts for 20% of gynaecological clinic referrals. (11,12) It has considerable impact on patients' quality of life, including their mental health and their income (13) due to loss of working days and diminished work capacity. Annual costs to the NHS have been estimated at approximately £326 million (14).
For endometriosis alone, which is just one cause of CPP, a European study of over 900 women showed average annual total costs per woman of €9579. Costs of productivity loss of €6298 were double the health care costs of €3113 per woman. The latter were due to surgery (29%), monitoring tests (19%) and hospitalization (18%) and physician visits (16%) (15).
Despite costly interventions, CPP is often resistant to surgical and medical treatment and appears to respond better to a multimodal, holistic approach, (16) with a focus on coping strategies. A systematic review of randomised controlled trials (RCTs) by authors has identified mindfulness meditation (MM) as an effective coping strategy in other chronic pain conditions (17). In addition, evidence from uncontrolled trials suggests positive effects of MM for CPP, such as an increased ability to control pain, improvements in mental health, emotional well-being, work and family life and social functioning (18,19), but these have never been examined in an RCT.
We therefore chose to evaluate MM delivered via an app to women with CPP as our intervention. CPP is especially common in younger women, who may be categorised as digital natives, making an app-based intervention particularly appropriate in this group.
In MM the aim is to keep focussed on one's own breathing. Whenever attention wanders to intrusive thoughts and feelings these are simply taken notice of in a neutral way, that is, Evaluation of an existing app is often appropriate (24) and is both quicker and more costeffective than designing an app from scratch. We chose to evaluate an existing commercial app platform that teaches mindfulness by guided meditation (Headspace ®), with a ten day We undertook a feasibility study (28) to assess whether or not to proceed with a full randomised controlled trial of the modified Headspace meditation app for women with CPP.
In the current paper we report on the qualitative interview and focus group data from this study; the protocol and quantitative results have been published/ submitted (29,30). Our aim in this analysis was to determine whether a pre-existing smartphone app to teach MM is acceptable to women with CPP and can be integrated into clinical practice within NHS CPP pathways.

Methods
The MEMPHIS trial was a three-arm parallel randomised feasibility trial approved by Objectives for the qualitative part of this study, using interviews and focus groups were to consider: 1) The acceptability, use and usability of the app in the intended service user population and for health care professionals (doctors, health care assistants, clinical and research nurses).
2) The feasibility of integrating such an app into existing healthcare pathways.
3) Feedback on the research process.
We follow the ISO 9241-11 (https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en) concept of technology usability (user friendliness) as the extent to which the app could be satisfactorily used by participants to meditate. By acceptability we mean whether participants could see a reason for using the app when given in clinic, and would be happy to use it for meditation.

Outcomes
The outcomes of this analysis were inductively derived descriptive themes on acceptability, use and usability of the app and feasibility of integrating it into existing pathways. Women in the mindfulness meditation group received access to a 60-day progressive mindfulness meditation course delivered via the Headspace app. The first 10 days of the course taught basics of mindfulness meditation. Following this, participants were able to access the module on meditation which was targeted at for chronic pain. This module had been specifically made for this study. Session length was 10 minutes for the first 10 days, 15 min up to day 20 and 20 min up to day 60. The active control group received access to a series of muscle relaxation sessions. These sessions were identical every day, except that their duration increased to mirror the increasing duration of the meditation content being listened to by the intervention group. Usage data are reported elsewhere (30).

Patient and Public Involvement
We held a Patient and Public Involvement (PPI) group workshop before the study to discuss acceptability of the Headspace app and help us design our study. Women attending the Royal London Hospital CPP clinic were invited to volunteer for a week of using the unmodified (normal commercially available) Headspace app (which did not have the pain module at the time we undertook our pre study workshop) and then feed back on their experiences with the app in an evening discussion group. Women were not involved in the design of the modified app. The focus for the PPI group was on the use of the generic MM app.
Two patient representatives provided support from the study design stage through recruitment to the interpretation of the results and regularly attended Trial Management Group meetings.

Study recruitment and eligibility
The trial recruited at two outpatient gynaecology clinics within Barts Health NHS trust in two separate deprived areas of inner East London. Female patients with new or follow-up gynaecology appointments were assessed for eligibility by a researcher in clinic, having been posted a Patient Information Sheet. Women were eligible if they had been suffering with CPP for 6 months or more and had at least a basic understanding of the English language, sufficient to follow instructions, as assessed during discussion about the study for informed consent; no women were excluded on this basis. Women were excluded if they did not meet these criteria or they did not have access to a smartphone or personal computer or were currently using the Headspace app (there were very few of the latter, according to the impression of the recruiting nurses). All patients gave full and informed consent to be randomised and data were collected through all stages of the study. All healthcare professionals and research nurses involved in the two clinics were also invited to take part in the feasibility study as the only eligibility criterion for staff. Full enrolment data are provided in Forbes et al (30). A key difference of these patients from those in the PPI group was that their focus was on managing their pain, with the app given explicitly as part of their clinic management support.
For the study of quantitative data, 90 patients were allocated randomly in a 1:1:1 ratio to the mindfulness meditation app, a muscle relaxation app active control or the usual care arm (for full details see (29)). Patients in the two active arms were asked to download the modified app in the clinic with support from a research staff member and were sent a questionnaire about app usability, an analysis of which is reported in a companion paper. (30) We used data from the app usability questionnaire to inform topic guides for the qualitative part of the study. This outlined key usability issues that had been uncovered, to guide our semistructured interviews and focus groups with patients and staff.
All women in the intervention and active control arms were eligible for the qualitative component of the study, and all staff participating in the study. All patients in the intervention and active control arms were invited to one of two focus groups at their own study site after the 6-month follow up. We offered telephone interviews as an alternative. Patients were asked to 'walk through' the app with researchers, articulating their thought processes while they did so and commenting on its different specific features, (31). Walkthroughs are often used in developing technologies such as mhealth. This helped to identify app usability issues or barriers to use of the app from the users' points of view without the need for technical discussions. Results for the walkthrough, showing comments on different features specific to the usability of the intervention app used in our study are shown in Appendix 1; walkthroughs were undertaken by two patients. Patients also discussed with us their experiences around app usability and acceptability.

Within-study interviews and focus groups
Staff were invited to attend a staff focus group overseen by the patient representative and facilitated by a researcher. In addition to considering app usability and acceptability, members of the staff focus group (consultants, health care assistants, clinical and research nurses and a representative from the pelvic pain support network were eligible) were asked about the ease of integration into existing NHS pathways. Part of the staff discussion was free flowing with open-ended questions, which gave us patient-focussed information on app acceptability, and part was structured using questions developed from the Normalisation Process Theory (NPT) toolkit in the way recommended by the NPT developers (32). For example, we asked whether staff could see a purpose to the app in clinical practice, as adding something different, which corresponds to the NPT toolkit question 'Participants distinguish the intervention from current ways of working'. Since this was a semi-structured approach questions were not rigidly worded. This helped us to consider the feasibility of integration of the app into practice. NPT is a theory of implementation practices that was initially developed for consideration of technology implementation and is in common use (32).

Analysis
Analysis of within-study focus groups and interviews was carried out blinded as to which study app was used, and deployed the immersion-crystallisation method (33). Thus, the lead qualitative researcher immersed herself in the data, reading transcripts carefully, then writing down articulated or crystallised patterns or themes that related to the aims and research questions of the study. These were discussed with another researcher from the team, and themes modified as appropriate. This process was repeated until all the data had been examined and all patterns that had been noticed were articulated, discussed and substantiated with exemplar extracts. This approach was considered appropriate since we had a small dataset and we were not aiming to develop conceptual themes but rather to inform the design and development of a randomised controlled trial for the modified app.
We used the SRQR checklist when writing our report (34).

Results
We screened 488 women between May and September 2016 for their eligibility to participate in the study. After exclusions, 90 women gave full consent to participate and were randomised to the intervention arm (31 women), the active control arm (30 women) or the usual care arm (29 women).

Demographics
Women in the main feasibility trial (30) were aged a mean of 35 years, 66% were employed and overall approximately 50% had stayed in full time education until at least the age of 20 though the proportion was least, at 36.7% in the intervention arm. Overall 44% were of white  sample was taken from the two active arms and was comprised of 16% of trial participants and 23% of those eligible for the qualitative study. We did not record separate demographic data for the women in this smaller sample.

Pre-study PPI group
The ten women in the pre-study PPI group were self-selected local women who were familiar with using apps and focused on app use per se. They anticipated no technical issues even in women who were not used to apps. They considered that the Headspace app would be successfully adopted by patients taking part in the study, given that a smartphone, like CPP, is 'always with you'. They praised the flexibility of the app, welcomed its portability and were unanimous in saying it was easy to slip off for 10 minutes when at work to use it. As a result, they found they could use it at times when they most needed pain relief as well as to prevent pain and found the app helpful in relieving pain and stress. The group reported being able to meditate without the app, once they had tried it with the app; however, they still preferred to use the app because they found the voice soothing.

App usage in the study
Patient usage of the app was less than expected from our pre-study PPI group discussions.
Only 36% of meditation app patients and 46% of the active control patients used the app at least once. (30)

Thematic analysis of within-study data
Qualitative data were obtained from 14 study patients; 12 preferred a telephone interview, two attended face-to-face interviews, one participant at the University attached to one of the recruiting clinics and one participant at the other recruiting hospital. Patients chose not to attend focus groups. Four of the patients were from the active control arm and 10 from the intervention arm. The two women we met face to face had both used the intervention and neither had progressed beyond the training stage, something that we cannot discount for other participants and which may help to explain reports of lack of effect on pain. Seven people attended the staff focus group: two recruiting nurses, three clinic nurses one consultant and a representative from the pelvic pain support network. These are explored below. As the PPI group data were not research data we did not analyse them for themes.
While we initially combined active control and intervention groups in our analysis, we then looked for instances where there was a difference between these two groups. Only where we found this difference in any theme or statement have we specified which group women belonged to.

Familiarity and capabilities with app technology
Around half of the patients were sufficiently familiar with technology and apps to be comfortable using the study apps. However, six participants (all using the intervention, which was more complex than the active control) reported difficulty because they were "not very good at technology" (patient 1002, intervention), or were unsure how to get started or use the app effectively without help. control) changed handsets and therefore did not continue with the app. In all cases these technical difficulties appeared to lead to abandoning of the app or restricted use of its functionality.
Five patients having technical problems suggested possible solutions such as a 'class' or group for first time users, a YouTube orientation video, or a pictorial leaflet. This might include an introduction to meditation and mindfulness as well as the app itself. One woman commented: "If your market is targeting people who are not using apps then you are going to have to get together and find ways to do this", (patient 1041, intervention); she also suggested we could get ideas from other apps on the market in this regard.
Given their experiences in the study, staff were concerned about additional staff time needed to support women in using the app. This would sit in tension with one of the original rationales behind choosing an app as the mode of delivery, which was to increase the effective use of staff contact time with patients. Language barriers might compound problems.

Motivations to use the app
Staff, though unaware of the low sustained app use in the study, felt it would be common sense to hold occasional motivating meetings with patients if the intention was for them to use the app long term. The patient data suggested the main motivators or lack of motivation for using the app in our sample, which could be drawn on in such meetings, and which we now consider.

Perceived benefits
Three intervention arm patients said they only entered the trial to help others through research but were already using alternative forms of pain control. They explained that this meant they were not motivated to actually use the app, perceiving the relative benefit to be One patient said medication was not working but the app did, though she was not sure whether this was 'mind over matter' (1065, intervention), which was her term for a placebo effect. Three others said it did not reduce their pain; all three were using the intervention app. The remaining patients considered other benefits were good reason for using the app even when they did not feel that it reduced pain intensity. Alternative or unanticipated benefits were not formally measured or taken into account in the study's effectiveness  (30). For example, ten patients valued the way the app helped them to relax or destress or focus and re-assess their life; three of these specifically said they used it to induce a relaxed state to get to sleep. Notably the active control was a relaxation app; however this benefit was also reported by many women in the intervention arm. One participant (active control) said she did not like the focus on pain per se as her condition impacted on various areas of her life. Even when the intervention app was positively received, women might stop using it because it was too powerful, and they had gained the change they wanted: Three intervention app patients found the app put them more in tune with their bodies and their breathing, (two of these were among those who also found the app de-stressing) while another found yoga better for that. Six patients, like pre-study PPI group members, also learned to use techniques from the app to alter their stress patterns without the app, having tried it, for example in traffic or by sitting down and taking time out or for general relaxation.

Relation to other therapies
Two patients (both active control) preferred 'pure' meditation, another considered the app to be "very much about meditation" (1041, intervention), which is in keeping with the arms they were in. An alternative therapy practitioner and two further patients reported that they preferred yoga. One (active control) said this was because it focussed on each part of the body in turn. Life getting in the way Seven patients revealed they preferred to use the app in the evenings because of other life commitments. This meant they did not always use it as a direct response to pain, reducing its potential for contemporaneous effect. One patient who used it in response to pain but only used it once or twice blamed this on having no spare time because of juggling work and children (active control); however, another patient (intervention) managed despite such commitments the fact that she was in the intervention arm may have played a role.

Barriers to integration for staff
Staff believed that the biggest barrier to clinical adoption of the app was a possible lack of support from the host organisation. It might also be hard to integrate the app within existing professional work practices if the staff in the position of offering the app to patients failed to see its relative advantage over other interventions. Collection of feedback on the app's effectiveness would be necessary for staff to support sustained use. It was felt that staff would need training on how to introduce the app to women in practice, and that complexity and high staff turnover could impede sustained use. An app was also seen as impersonal compared with face-to-face contact, which was more favoured by staff.

Participant comments on the research process
The study questionnaires that were used for the main quantitative outcome measures (30) were acceptable to patients except for some discomfort with a question about sex, which patients considered a delicate question that was missing a 'no sex' option. Most preferred a paper form reflecting their lack of affinity with technology. There were no indications that the study design or study processes had contributed to the participants' lack of engagement with the apps -with a caveat around support with the technology as mentioned above -though we did not systematically consider this. A full summary of patient comments on the study design and procedures is given in Appendix 1.

Discussion
Our study adds to the limited evidence on mHealth app user behaviour and experience (36,37). The pre-study PPI group (young women, of a generation who were familiar with using apps and who were asked to focus on the study design use of the app) liked the idea of delivering the intervention via an app, praising the contemporary design and flexibility.
Hence we expected a similar positive attitude from trial participants, who were of a comparable age, and we assumed would be keen on using apps. Participant feedback revealed that this assumption was too simplistic.
Using our qualitative data, we were able to explore reasons for low app usage that had been recorded in our feasibility study. (30) Our thematic analysis suggests that the low app use in the trial occurred because many patients were not familiar with apps in general or lacked capabilities with technology. This was particularly true for the more complex intervention app. The other themes we report did not differ between groups (although the three cases of weak altruism' all occurred in the intervention arm) which suggests more generic issues that The advantages and disadvantages of using the app stand-alone were also illustrated by our data. Some suggestions made by participants to improve usage, such as more guidance at the start, seem obvious in hindsight. But they had not been considered because of the feedback from the pre-study PPI group and the commercial success of Headspace. The use of community contacts may be a helpful alternative.(40) Social support can create a community of practice, help to clarify expectations,(41) and improve health outcomes (as shown for example in internet based psychological treatment for depression (42).
The data suggest that for successful app use we need to understand what motivates individuals with clinical need to use the app for clinical reasons (which our PPI group did not focus on) and target this, for example by setting appropriate expectations. Incentivisation might also improve motivation. This could be achieved through app gamification (7), or encouragement through integration with patient-clinician face-to-face encounters, which was lacking in our study since the app was used stand-alone. The present study provided extensive initial technical support but no coaching and incentivising, in keeping with the protocol. Future app studies should take this into account. Participants in our study may Lack of engagement after recruitment, or good initial engagement but minimal or inconsistent use, have been reported in other studies (44,45), including in Headspace trials other than our own (46,47). Settings were a university and a psychiatric inpatients clinic, both in the USA. Inconsistent app use was noted by Wen (48)  None of these studies included a qualitative component. Yet, each of the Headspace study groups were very different, and so will likely have differed in motivations, contexts for opportunity to use the app, and incentivisations (50). While these aspects were not considered in the other studies, our use of qualitative research has enabled us to explore these in more depth. Our findings suggest these are important considerations in any study of app use and therefore this study makes a contribution to the field. For example, some of the groups in other studies may have differed from ours in likelihood of using mhealth apps in the first place, and familiarity with technology. In-patients may have more time to use the app and more support -and may also have had specific barriers to app use, such as related to setting and to illness. Patients in the qualitative part of our study tended not to use apps on a regular basis (or at least apps other than simple games), and in terms of our themes, also represented in the COM-B model, may be said to have few capabilities in technology use. They therefore do not represent the typical users of the Headspace app in a commercial setting. Accessing the app regularly requires energy, time and effort, but patients with CPP often suffer from fatigue and anxiety as co-morbidities, perhaps whilst having to juggle family life and work.
Therefore, this may be seen as a challenging clinical population in which to trial an app.
Our study has also shown that clear consideration of unexpected benefits should be included in future studies and these can be informed by our finding that benefits for patients may be more diffuse than anticipated (e.g. app relieving stress rather than pain). Such benefits were found in the active control as well as the intervention arm and so it may be that they represent a placebo effect though the effect could equally be real. Our data also suggest that staff benefits may be less than anticipated, as participants sometimes needed a lot of support and scaffolding in technology use at least initially.
Young age, co-morbid anxiety and low educational attainment are predictors for dropping out of web-based interventions according to studies in the field of depression. (51,52) This may be true despite regular phone support (52) though our participants all considered active motivational support from staff or app support groups would have improved app use. Our intervention arm participants were particularly likely to be young and with low educational attainment.
Our data suggest that it is important to involve real world end users in the agile design or development or modification of apps in close collaboration with researchers and commercial app developers. (7) Although the evaluation of existing apps has been recommended as a cost-effective and rapid process, (24) our findings suggest that in actual clinical practice these may be problematic.

Strengths and weaknesses of the study and in relation to other studies
One strength of this study is that it creates much-needed evidence in the field of evaluating existing health apps in a clinic population (6,8,24) and recording user experience. This provides us with lessons to be learned.
Researchers conducting interviews and focus groups were: a senior mixed methods medical sociology researcher, a recruiting nurse, a representative from the pelvic pain support network and an experienced health psychologist. Findings were similar across the data and the different backgrounds of the researchers therefore does not appear to have influenced findings. The main analysis was undertaken by the medical sociologist and so the concordance with the COM-B model is not due to background discipline bias.
We were able to recruit successfully, and we obtained valuable information from patients with CPP, who were recruited from a deprived urban area of the UK as typical local clinical patients.
However, we report a marked discrepancy between the attitudes of the pre-study PPI group of volunteer patients from the local area, who actively put themselves forward for a 7-day trial of the app, and the participants asked to take part when they attended clinics. The opinions of pre-study PPI volunteers meeting in their private time may not be representative of the opinions of patients recruited at a routine clinic appointment. Women in the PPI group were used to using apps, which had led them to be interested in the study in the first place.
Whereas women in the PPI group had all trialled the app at home and work without support from us, many patients from clinic were unable to use their phone beyond calls, texts and photos. Moreover, most of the women we interviewed used the intervention app. We can only speculate as to why this is so but it does mean that concordances and divergences across the intervention and active control arm do need to be treated with circumspection. To our knowledge the present observation on failure of PPI work to translate into practice in a trial has not been formally reported before, and is lacking from a recent comprehensive systematic review.(53) PPI involvement is a stipulated requirement when applying for some funding, and the present research findings should be taken into account when drafting guidelines for future PPI involvement in study planning. PPI groups are able to provide significant help and advice in any study but our findings shows the value of adding agile codevelopment as a requirement for app intervention development as likely to provide a more effective intervention than one informed by PPI alone. Moreover, there is a difference between app use for active clinical management (as with our study participants), and consideration of the potential for app use for this (as with our PPI group).

Implications for clinicians and policymakers
Given the patchy use of the app and the way that some participants did not manage to unlock its full functionality, and an indication of diffusion of benefit, more work is needed to see whether the app reduces pain per se. This study is a good example of the need to move away from 'one size fits all' behavioural interventions. Future studies should do more work on implementation before doing an effectiveness trial. This will enable researchers to be more nuanced about saying who the app is effective for, if at all.
Strategies to involve busy, less motivated, and less technologically experienced individuals in PPI and lay app design groups need to be further developed. These groups should include considerable scaffolding, which we have shown extends to study involvement by patients. More care is also needed to obtain PPI input that is representative of the target group, taking into account their capabilities, opportunities and motivational aspects. It may be useful to give the PPI group a small condition management task that emulates what trial participants will be required to do. Moreover, we can confirm a recent review suggesting that health apps should be co-designed with users (7), rather than presenting them with a

Author contributions
EB led the study as the CI. EB and CR were the main authors of the grant application for this study, and co-lead authors of the current paper. All other authors contributed to the study design and initial protocol, provided support throughout the trial and contributed towards the final paper. CR led on the PPI and CR and LS led the interview and focus group field work and analysis reported here. CR, LS, SN, CC, JD and JB were all involved in the field work.

Role of the funding source
The

Competing interests
We have read and understood BMJ policy on declaration of interests.

Instructions to authors
Complete this checklist by entering the page numbers from your manuscript where readers will find each of the items listed below.
Your article may not currently address all the items on the checklist. Please modify your text to include the missing information. If you are certain that an item does not apply, please write "n/a" and provide a short explanation.
Upload your completed checklist as an extra file when you submit to a journal.
In your methods section, say that you used the SRQR reporting guidelines, and cite them as: To determine whether a pre-existing smartphone App to teach Mindfulness Meditation is acceptable to women with chronic pelvic pain (CPP) and can be integrated into clinical practice within NHS CPP pathways. To inform the design of a potential randomised clinical trial.

Design
A pre-study patient and public involvement (PPI) group to collect feedback on the acceptability of the existing app and study design was followed by a three-arm randomised feasibility trial. In addition, we undertook interviews and focus groups with patients and staff to explore app usability and acceptability. We also obtained participant comments on the research process, such as acceptability of the study questionnaires.

Setting
Two gynaecology clinics within Barts Health NHS, London, UK.

Participants
Patients with CPP lasting >6 months with access to smartphone or PC and understanding of basic English.

Intervention
The intervention was mindfulness meditation content plus additional pain module delivered by smartphone app, active controls received muscle relaxation content by the same app.
Passive (waiting list) controls received usual care.

Main outcome measures
Themes on user feedback, app usability and integration and reasons for using/not using the app.

Results
App use was low in both active groups.
Patients in the pre-study PPI group, all volunteers, were enthusiastic about the app (convenience, content, portability, flexibility, ease of use). Women contributing to the were less positive, citing as barriers lack of opportunities/motivation to use the app, and lack of familiarity and capabilities with technology. Staff (n=7) were concerned about the potential need for extra support for patients and staff and considered the app needed organisational backing and peer acceptance.

Conclusion
The opinions of pre-study PPI volunteers meeting in their private time may not represent those of patients recruited at a routine clinic appointment.
It may be more successful to co-design/co-develop an app with typical users than to adapt existing apps for use in real-world clinical populations.

Trial registration and funding
The trial (ISRCTN 10925965) was funded by the UK National Institute of Health Research,

Introduction
Smartphone health apps, as one form of mhealth (1), are popular in the UK, our study setting. With more than two-thirds of the UK population using smartphones (2,3,4), health apps are one of the fastest growing app categories, thus numbers of users are still increasing (5). Currently these apps are usually developed either by researchers or (in the majority) by commercial companies, without collaboration between these groups (6,7). The lack of interaction between researchers and commercial developers in the field of painrelated apps has led to a situation where commercially available apps have not been scientifically validated and apps that have been developed from research projects are not commercially available (8). We were interested in using an app to support women with chronic pelvic pain (CPP) in a clinical setting, where validation of an intervention is important to ensure best care. CPP is defined as a subjective physical and emotional experience of pain in the pelvic area that has been present for at least six months that may or may not have an identifiable pathology (9).
CPP affects up to 24% of women worldwide (10) and accounts for 20% of gynaecological clinic referrals. (11,12) It has considerable impact on patients' quality of life, including their mental health and their income (13) due to loss of working days and diminished work capacity. Annual costs to the NHS have been estimated at approximately £326 million (14).
For endometriosis alone, which is just one cause of CPP, a European study of over 900 women showed average annual total costs per woman of €9579. Costs of productivity loss of €6298 were double the health care costs of €3113 per woman. The latter were due to surgery (29%), monitoring tests (19%) and hospitalization (18%) and physician visits (16%) (15).
Despite costly interventions, CPP is often resistant to surgical and medical treatment and appears to respond better to a multimodal, holistic approach, (16) with a focus on coping strategies. A systematic review of randomised controlled trials (RCTs) by authors has identified mindfulness meditation (MM) as an effective coping strategy in other chronic pain conditions (17). In addition, evidence from uncontrolled trials suggests positive effects of MM for CPP, such as an increased ability to control pain, improvements in mental health, emotional well-being, work and family life and social functioning (18,19), but these have never been examined in an RCT.
We therefore chose to evaluate MM delivered via an app to women with CPP as our intervention. CPP is especially common in younger women, who may be categorised as digital natives, making an app-based intervention particularly appropriate in this group.
In MM the aim is to keep focussed on one's own breathing. Whenever attention wanders to intrusive thoughts and feelings these are simply taken notice of in a neutral way, that is,  Evaluation of an existing app is often appropriate (24) and is both quicker and more costeffective than designing an app from scratch. We chose to evaluate an existing commercial app platform that teaches mindfulness by guided meditation (Headspace ®), with a ten day We undertook a three-arm parallel randomised feasibility trial (MEMPHIS) (28) to assess whether or not to proceed with a full randomised controlled trial of the modified Headspace meditation app for women with CPP. In the current paper we report on the qualitative interview and focus group data from this study; the protocol and quantitative results have been  (29,30). Our aim for the qualitative part of this study was to determine whether a pre-existing smartphone app to teach MM is acceptable to women with CPP and can be integrated into clinical practice within NHS CPP pathways. Objectives were to consider: 1) The acceptability, use and usability of the app in the intended service user population and for health care professionals (doctors, health care assistants, clinical and research nurses) 2) The feasibility of integrating such an app into existing healthcare pathways 3) The usefulness of having a distinct patient group to advise us on the study design.

Outcomes
The outcomes of this analysis were inductively derived descriptive themes on acceptability, use and usability of the app and feasibility of integrating it into existing pathways. We follow the ISO 9241-11 (https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en) concept of technology usability (user friendliness) as the extent to which the app could be satisfactorily used by participants to meditate. By acceptability we mean whether participants could see a reason for using the app when given in clinic, and would be happy to use it for meditation.

Intervention Procedures
Women in the mindfulness meditation group received access to a 60-day progressive mindfulness meditation course delivered via the Headspace app. The first 10 days of the course taught basics of mindfulness meditation. Following this, participants were able to access the module on meditation which was targeted at for chronic pain. This module had min up to day 20 and 20 min up to day 60. The active control group received access to a series of muscle relaxation sessions. These sessions were identical every day, except that their duration increased to mirror the increasing duration of the meditation content being listened to by the intervention group. Usage data are reported elsewhere (30).

Patient and Public Involvement
We held a Patient and Public Involvement (PPI) group workshop before the study to discuss acceptability of the Headspace app and help us design our study. Women attending the Royal London Hospital CPP clinic were invited to volunteer for a week of using the unmodified (normal commercially available) Headspace app (which did not have the pain module at the time we undertook our pre study workshop) and then feed back on their experiences with the app in an evening discussion group. Women were not involved in the design of the modified app. The focus for the PPI group was on the use of the generic MM app.
Two patient representatives provided support from the study design stage through recruitment to the interpretation of the results and regularly attended Trial Management Group meetings.

Study recruitment and eligibility
The trial recruited at two outpatient gynaecology clinics within Barts Health NHS trust in two separate deprived areas of inner East London. Female patients with new or follow-up gynaecology appointments were assessed for eligibility by a researcher in clinic, having been posted a Patient Information Sheet. Women were eligible if they had been suffering with CPP for 6 months or more and had at least a basic understanding of the English language, sufficient to follow instructions, as assessed during discussion about the study for informed consent; no women were excluded on this basis. Women were excluded if they did not meet these criteria or they did not have access to a smartphone or personal computer or were currently using the Headspace app (there were very few of the latter, according to the impression of the recruiting nurses). All patients gave full and informed consent to be randomised and data were collected through all stages of the study. All healthcare professionals and research nurses involved in the two clinics were also invited to take part in the feasibility study as the only eligibility criterion for staff. Full enrolment data are provided in Forbes et al (30). A key difference of these patients from those in the PPI group was that their focus was on managing their pain, with the app given explicitly as part of their clinic management support.
For the study of quantitative data, 90 patients were allocated randomly in a 1:1:1 ratio to the mindfulness meditation app, a muscle relaxation app active control or the usual care arm (for full details see (29)). Patients in the two active arms were asked to download the modified app in the clinic with support from a research staff member and were sent a questionnaire about app usability, an analysis of which is reported in a companion paper. (30) We used data from the app usability questionnaire to inform topic guides for the qualitative part of the study. This outlined key usability issues that had been uncovered, to guide our semistructured interviews and focus groups with patients and staff.
All women in the intervention and active control arms were eligible for the qualitative component of the study, and all staff participating in the study.

Within-study interviews and focus groups
All patients in the intervention and active control arms were invited to one of two focus groups at their own study site after the 6-month follow up. We offered telephone interviews as an alternative. Patients were asked to 'walk through' the app with researchers, articulating their thought processes while they did so and commenting on its different specific features, free flowing with open-ended questions, which gave us patient-focussed information on app acceptability, and part was structured using questions developed from the Normalisation Process Theory (NPT) toolkit in the way recommended by the NPT developers (32). For example, we asked whether staff could see a purpose to the app in clinical practice, as adding something different, which corresponds to the NPT toolkit question 'Participants distinguish the intervention from current ways of working'. Since this was a semi-structured approach questions were not rigidly worded. This helped us to consider the feasibility of integration of the app into practice. NPT is a theory of implementation practices that was initially developed for consideration of technology implementation and is in common use (32).
All data were audio-recorded at point of collection and transcribed, with personal identifying data removed from transcripts. Raw data were stored in a Primary Care Clinical Trial Unit database to clinical trial standards. Analysis of within-study focus groups and interviews was carried out blinded as to which study app was used, and deployed the immersion-crystallisation method (33). Thus, the lead qualitative researcher immersed herself in the data, reading transcripts carefully, then writing down articulated or crystallised patterns or themes that related to the aims and research questions of the study. These were discussed with another researcher from the team, and themes modified as appropriate. This process was repeated until all the data had been examined and all patterns that had been noticed were articulated, discussed and substantiated with exemplar extracts. This approach was considered appropriate since we had a small dataset and we were not aiming to develop conceptual themes but rather to inform the design and development of a randomised controlled trial for the modified app.

Analysis
We used the SRQR checklist when writing our report (34).

Ethics
The MEMPHIS trial was a three-arm parallel randomised feasibility trial approved by Camden and Kings Cross Research Ethics Committee in 2016 (15/LO/1967).

Results
We screened 488 women between May and September 2016 for their eligibility to participate in the study. After exclusions, 90 women gave full consent to participate and were randomised to the intervention arm (31 women), the active control arm (30 women) or the usual care arm (29 women).

Demographics
Women in the main feasibility trial (30) were aged a mean of 35 years, 66% were employed and overall approximately 50% had stayed in full time education until at least the age of 20 though the proportion was least, at 36.7% in the intervention arm. Overall 44% were of white ethnicity though the proportion was lowest in the intervention arm (35.7%) and highest in the usual care arm (53.6%). The second most common self-reported ethnic groups were

Pre-study PPI group
The ten women in the pre-study PPI group were self-selected local women who were familiar with using apps and focused on app use per se. They anticipated no technical issues even in women who were not used to apps. They considered that the Headspace app would be successfully adopted by patients taking part in the study, given that a smartphone, like CPP, is 'always with you'. They praised the flexibility of the app, welcomed its portability and were unanimous in saying it was easy to slip off for 10 minutes when at work to use it. As a result, they found they could use it at times when they most needed pain relief as well as to prevent pain and found the app helpful in relieving pain and stress. The group reported being able to meditate without the app, once they had tried it with the app; however, they still preferred to use the app because they found the voice soothing.

App usage in the study
Patient usage of the app was less than expected from our pre-study PPI group discussions.
Only 36% of meditation app patients and 46% of the active control patients used the app at least once. (30)

Thematic analysis of within-study data
Qualitative data were obtained from 14 study patients; 12 preferred a telephone interview, two attended face-to-face interviews, one participant at the University attached to one of the recruiting clinics and one participant at the other recruiting hospital. Patients chose not to attend focus groups. Four of the patients were from the active control arm and 10 from the intervention arm. The two women we met face to face had both used the intervention and neither had progressed beyond the training stage, something that we cannot discount for other participants and which may help to explain reports of lack of effect on pain. Seven people attended the staff focus group: two recruiting nurses, three clinic nurses one consultant and a representative from the pelvic pain support network. These are explored below. As the PPI group data were not research data we did not analyse them for themes.
While we initially combined active control and intervention groups in our analysis, we then looked for instances where there was a difference between these two groups. Only where we found this difference in any theme or statement have we specified which group women belonged to.

Familiarity and capabilities with app technology
Around half of the patients were sufficiently familiar with technology and apps to be comfortable using the study apps. However, six participants (all using the intervention, which was more complex than the active control) reported difficulty because they were "not very good at technology" (patient 1002, intervention), or were unsure how to get started or use the app effectively without help. include an introduction to meditation and mindfulness as well as the app itself. One woman commented: "If your market is targeting people who are not using apps then you are going to have to get together and find ways to do this", (patient 1041, intervention); she also suggested we could get ideas from other apps on the market in this regard.
Given their experiences in the study, staff were concerned about additional staff time needed to support women in using the app. This would sit in tension with one of the original rationales behind choosing an app as the mode of delivery, which was to increase the effective use of staff contact time with patients. Language barriers might compound problems.

Motivations to use the app
Staff, though unaware of the low sustained app use in the study, felt it would be common sense to hold occasional motivating meetings with patients if the intention was for them to use the app long term. The patient data suggested the main motivators or lack of motivation for using the app in our sample, which could be drawn on in such meetings, and which we now consider.

Perceived benefits
Three intervention arm patients said they only entered the trial to help others through research but were already using alternative forms of pain control. They explained that this meant they were not motivated to actually use the app, perceiving the relative benefit to be One patient said medication was not working but the app did, though she was not sure whether this was 'mind over matter' (1065, intervention), which was her term for a placebo effect. Three others said it did not reduce their pain; all three were using the intervention app. The remaining patients considered other benefits were good reason for using the app even when they did not feel that it reduced pain intensity. Alternative or unanticipated benefits were not formally measured or taken into account in the study's effectiveness Three intervention app patients found the app put them more in tune with their bodies and their breathing, (two of these were among those who also found the app de-stressing) while another found yoga better for that. Six patients, like pre-study PPI group members, also learned to use techniques from the app to alter their stress patterns without the app, having tried it, for example in traffic or by sitting down and taking time out or for general relaxation.

Relation to other therapies
Two patients (both active control) preferred 'pure' meditation, another considered the app to be "very much about meditation" (1041, intervention), which is in keeping with the arms they were in. An alternative therapy practitioner and two further patients reported that they preferred yoga. One (active control) said this was because it focussed on each part of the body in turn. Life getting in the way Seven patients revealed they preferred to use the app in the evenings because of other life commitments. This meant they did not always use it as a direct response to pain, reducing its potential for contemporaneous effect. One patient who used it in response to pain but only used it once or twice blamed this on having no spare time because of juggling work and children (active control); however, another patient (intervention) managed despite such commitments the fact that she was in the intervention arm may have played a role.

Barriers to integration for staff
Staff believed that the biggest barrier to clinical adoption of the app was a possible lack of support from the host organisation. It might also be hard to integrate the app within existing professional work practices if the staff in the position of offering the app to patients failed to see its relative advantage over other interventions. Collection of feedback on the app's effectiveness would be necessary for staff to support sustained use. It was felt that staff would need training on how to introduce the app to women in practice, and that complexity and high staff turnover could impede sustained use. An app was also seen as impersonal compared with face-to-face contact, which was more favoured by staff.

Participant comments on the research process
The study questionnaires that were used for the main quantitative outcome measures (30) were acceptable to patients except for some discomfort with a question about sex, which patients considered a delicate question that was missing a 'no sex' option. Most preferred a paper form reflecting their lack of affinity with technology. There were no indications that the study design or study processes had contributed to the participants' lack of engagement with the apps -with a caveat around support with the technology as mentioned above -though we did not systematically consider this. A full summary of patient comments on the study design and procedures is given in Appendix 1.

Discussion
Our study adds to the limited evidence on mHealth app user behaviour and experience (36,37). The pre-study PPI group (young women, of a generation who were familiar with using apps and who were asked to focus on the study design use of the app) liked the idea of delivering the intervention via an app, praising the contemporary design and flexibility.
Hence we expected a similar positive attitude from trial participants, who were of a comparable age, and we assumed would be keen on using apps. Participant feedback revealed that this assumption was too simplistic.
Using our qualitative data, we were able to explore reasons for low app usage that had been recorded in our feasibility study. (30) Our thematic analysis suggests that the low app use in the trial occurred because many patients were not familiar with apps in general or lacked capabilities with technology. This was particularly true for the more complex intervention app. The other themes we report did not differ between groups (although the three cases of weak altruism' all occurred in the intervention arm) which suggests more generic issues that The advantages and disadvantages of using the app stand-alone were also illustrated by our data. Some suggestions made by participants to improve usage, such as more guidance at the start, seem obvious in hindsight. But they had not been considered because of the feedback from the pre-study PPI group and the commercial success of Headspace. The use of community contacts may be a helpful alternative.(40) Social support can create a community of practice, help to clarify expectations,(41) and improve health outcomes (as shown for example in internet based psychological treatment for depression (42).
The data suggest that for successful app use we need to understand what motivates individuals with clinical need to use the app for clinical reasons (which our PPI group did not focus on) and target this, for example by setting appropriate expectations. Incentivisation might also improve motivation. This could be achieved through app gamification (7), or encouragement through integration with patient-clinician face-to-face encounters, which was lacking in our study since the app was used stand-alone. The present study provided extensive initial technical support but no coaching and incentivising, in keeping with the protocol. Future app studies should take this into account. Participants in our study may Lack of engagement after recruitment, or good initial engagement but minimal or inconsistent use, have been reported in other studies (44,45), including in Headspace trials other than our own (46,47). Settings were a university and a psychiatric inpatients clinic, both in the USA. Inconsistent app use was noted by Wen (48)  None of these studies included a qualitative component. Yet, each of the Headspace study groups were very different, and so will likely have differed in motivations, contexts for opportunity to use the app, and incentivisations (50). While these aspects were not considered in the other studies, our use of qualitative research has enabled us to explore these in more depth. Our findings suggest these are important considerations in any study of app use and therefore this study makes a contribution to the field. For example, some of the groups in other studies may have differed from ours in likelihood of using mhealth apps in the first place, and familiarity with technology. In-patients may have more time to use the app and more support -and may also have had specific barriers to app use, such as related to setting and to illness. Patients in the qualitative part of our study tended not to use apps on a regular basis (or at least apps other than simple games), and in terms of our themes, also represented in the COM-B model, may be said to have few capabilities in technology use. They therefore do not represent the typical users of the Headspace app in a commercial setting. Accessing the app regularly requires energy, time and effort, but patients with CPP often suffer from fatigue and anxiety as co-morbidities, perhaps whilst having to juggle family life and work.
Therefore, this may be seen as a challenging clinical population in which to trial an app.
Our study has also shown that clear consideration of unexpected benefits should be included in future studies and these can be informed by our finding that benefits for patients may be more diffuse than anticipated (e.g. app relieving stress rather than pain). Such benefits were found in the active control as well as the intervention arm and so it may be that they represent a placebo effect though the effect could equally be real. Our data also suggest that staff benefits may be less than anticipated, as participants sometimes needed a lot of support and scaffolding in technology use at least initially.
Young age, co-morbid anxiety and low educational attainment are predictors for dropping out of web-based interventions according to studies in the field of depression. (51,52) This may be true despite regular phone support (52) though our participants all considered active motivational support from staff or app support groups would have improved app use. Our intervention arm participants were particularly likely to be young and with low educational attainment.
Our data suggest that it is important to involve real world end users in the agile design or development or modification of apps in close collaboration with researchers and commercial app developers. (7) Although the evaluation of existing apps has been recommended as a cost-effective and rapid process, (24) our findings suggest that in actual clinical practice these may be problematic.

Strengths and weaknesses of the study and in relation to other studies
One strength of this study is that it creates much-needed evidence in the field of evaluating existing health apps in a clinic population (6,8,24) and recording user experience. This provides us with lessons to be learned.
Researchers conducting interviews and focus groups were: a senior mixed methods medical sociology researcher, a recruiting nurse, a representative from the pelvic pain support network and an experienced health psychologist. Findings were similar across the data and the different backgrounds of the researchers therefore does not appear to have influenced findings. The main analysis was undertaken by the medical sociologist and so the concordance with the COM-B model is not due to background discipline bias.
We were able to recruit successfully, and we obtained valuable information from patients with CPP, who were recruited from a deprived urban area of the UK as typical local clinical patients.
However, we report a marked discrepancy between the attitudes of the pre-study PPI group of volunteer patients from the local area, who actively put themselves forward for a 7-day trial of the app, and the participants asked to take part when they attended clinics. The opinions of pre-study PPI volunteers meeting in their private time may not be representative of the opinions of patients recruited at a routine clinic appointment. Women in the PPI group were used to using apps, which had led them to be interested in the study in the first place.
Whereas women in the PPI group had all trialled the app at home and work without support from us, many patients from clinic were unable to use their phone beyond calls, texts and photos. Moreover, most of the women we interviewed used the intervention app. We can only speculate as to why this is so but it does mean that concordances and divergences across the intervention and active control arm do need to be treated with circumspection.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y   24 To our knowledge the present observation on failure of PPI work to translate into practice in a trial has not been formally reported before, and is lacking from a recent comprehensive systematic review.(53) PPI involvement is a stipulated requirement when applying for some funding, and the present research findings should be taken into account when drafting guidelines for future PPI involvement in study planning. PPI groups are able to provide significant help and advice in any study but our findings shows the value of adding agile codevelopment as a requirement for app intervention development as likely to provide a more effective intervention than one informed by PPI alone. Moreover, there is a difference between app use for active clinical management (as with our study participants), and consideration of the potential for app use for this (as with our PPI group).

Implications for clinicians and policymakers
Given the patchy use of the app and the way that some participants did not manage to unlock its full functionality, and an indication of diffusion of benefit, more work is needed to see whether the app reduces pain per se. This study is a good example of the need to move away from 'one size fits all' behavioural interventions. Future studies should do more work on implementation before doing an effectiveness trial. This will enable researchers to be more nuanced about saying who the app is effective for, if at all.
Strategies to involve busy, less motivated, and less technologically experienced individuals in PPI and lay app design groups need to be further developed. These groups should include considerable scaffolding, which we have shown extends to study involvement by patients. More care is also needed to obtain PPI input that is representative of the target group, taking into account their capabilities, opportunities and motivational aspects. It may be useful to give the PPI group a small condition management task that emulates what trial participants will be required to do. Moreover, we can confirm a recent review suggesting  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  that health apps should be co-designed with users (7), rather than presenting them with a pre-existing app. These implications for our study are also generalisable to other technology studies.
What the study adds 1. A mindfulness meditation app may not necessarily be taken up by patients with CPP even when it is a commercial success in the general population. Considerable supportive scaffolding may be needed.
2. Use of existing commercial apps in actual clinical practice may be problematic compared with the agile development of apps with collaboration between researchers, clinicians, developers and end users.
3. Chronic Pain patients are interested in alternatives to drug or surgical treatments and further research is required in this area, including with MM, the benefits of which may extend beyond pain relief itself.
4. PPI groups may be more motivated to use an intervention than a real world clinical group as they are volunteers who are interested in the research topic. This may be particularly problematic for interventions that require considerable capability or motivation in use.

Author contributions
EB led the study as the CI. EB and CR were the main authors of the grant application for this study, and co-lead authors of the current paper. FR, SJCT, JD, JB, SN and LS contributed to the study design and initial protocol. All authors provided support throughout the trial and contributed towards the final paper. CR led on the PPI and CR and LS led the interview and focus group field work and analysis reported here. CR, LS, SN, CCC, JD and JB were all involved in the field work.

Role of the funding source
The  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y   26 the collection, analysis, and interpretation of the data, in the writing of this report, or in the decision to submit the paper for publication. The first and last authors vouch for the integrity, completeness and accuracy of the data and analyses, and for the fidelity of this report to the protocol and statistical analysis plan. The views and opinions expressed herein are those of the authors and do not necessarily reflect those of the RfPB, NIHR, NHS or the Department of Health.

Competing interests
We have read and understood BMJ policy on declaration of interests.

Data sharing
The data are collected from a small number of people which could compromise their identity if shared with others. Therefore we are not making them available except under exceptional circumstances which will be determined by the custodian of the data (Elizabeth Ball) on an individual basis.

Graphics
The pictures were often not appropriate as they focused on the problem (e.g. tooth pain logo) rather than something positive or soothing. Unanticipated problems could arise according to one participant; she found it hard to lose weight and felt the food icons a) did not represent success and b) reminded her of eating.
One participant found the pictures hard to see.
Three participants were frustrated and even 'angry' and 'patronised' at having to do the same (basic) meditation several times before moving up a stage.
There was no indication of goals, duration of each level or gamification rewards.
Goal setting is recommended by psychological theory and could be as simple as using "day 1 of 30" rather than simply Day 1.
Changing emojis could show progress, for example from a sad face to a smiley face to a heart; the inbuilt progress function was not felt sufficiently motivating.

Introduction
Experienced app users suggested the introduction could be improved for initiates overview of the entire app and its levels suggested for the start.
One participant had done mindfulness before but thought an introduction to mindfulness might be helpful for others.
Based on the SRQR guidelines.

Instructions to authors
Complete this checklist by entering the page numbers from your manuscript where readers will find each of the items listed below.
Your article may not currently address all the items on the checklist. Please modify your text to include the missing information. If you are certain that an item does not apply, please write "n/a" and provide a short explanation.
Upload your completed checklist as an extra file when you submit to a journal.

Design
A pre-study patient and public involvement (PPI) group to collect feedback on the acceptability of the existing app and study design was followed by a three-arm randomised feasibility trial. In addition, we undertook interviews and focus groups with patients and staff to explore app usability and acceptability. We also obtained participant comments on the research process, such as acceptability of the study questionnaires.

Setting
Two gynaecology clinics within Barts Health NHS, London, UK.

Participants
Patients with CPP lasting >6 months with access to smartphone or PC and understanding of basic English.

Intervention
The intervention was mindfulness meditation content plus additional pain module delivered by smartphone app, active controls received muscle relaxation content by the same app.
Passive (waiting list) controls received usual care.

Main outcome measures
Themes on user feedback, app usability and integration and reasons for using/not using the app.

Results
App use was low in both active groups.
Patients in the pre-study PPI group, all volunteers, were enthusiastic about the app (convenience, content, portability, flexibility, ease of use). Women contributing to the were less positive, citing as barriers lack of opportunities/motivation to use the app, and lack of familiarity and capabilities with technology. Staff (n=7) were concerned about the potential need for extra support for patients and staff and considered the app needed organisational backing and peer acceptance.

Conclusion
The opinions of pre-study PPI volunteers meeting in their private time may not represent those of patients recruited at a routine clinic appointment.
It may be more successful to co-design/co-develop an app with typical users than to adapt existing apps for use in real-world clinical populations.

Trial registration and funding
The trial (ISRCTN 10925965) was funded by the UK National Institute of Health Research, Research for Patient Benefit programme (RfPB PB-PG-1013-32025).

Introduction
Smartphone health apps, as one form of mhealth (1), are popular in the UK, our study setting. With more than two-thirds of the UK population using smartphones (2,3,4), health apps are one of the fastest growing app categories, thus numbers of users are still increasing (5). Currently these apps are usually developed either by researchers or (in the majority) by commercial companies, without collaboration between these groups (6,7). The lack of interaction between researchers and commercial developers in the field of painrelated apps has led to a situation where commercially available apps have not been scientifically validated and apps that have been developed from research projects are not commercially available (8). We were interested in using an app to support women with chronic pelvic pain (CPP) in a clinical setting, where validation of an intervention is important to ensure best care. CPP is defined as a subjective physical and emotional experience of pain in the pelvic area that has been present for at least six months that may or may not have an identifiable pathology (9).
CPP affects up to 24% of women worldwide (10) and accounts for 20% of gynaecological clinic referrals. (11,12) It has considerable impact on patients' quality of life, including their mental health and their income (13) due to loss of working days and diminished work capacity. Annual costs to the NHS have been estimated at approximately £326 million (14).
For endometriosis alone, which is just one cause of CPP, a European study of over 900 women showed average annual total costs per woman of €9579. Costs of productivity loss of €6298 were double the health care costs of €3113 per woman. The latter were due to surgery (29%), monitoring tests (19%) and hospitalization (18%) and physician visits (16%) (15).
Despite costly interventions, CPP is often resistant to surgical and medical treatment and appears to respond better to a multimodal, holistic approach, (16) with a focus on coping strategies. A systematic review of randomised controlled trials (RCTs) by authors has identified mindfulness meditation (MM) as an effective coping strategy in other chronic pain conditions (17). In addition, evidence from uncontrolled trials suggests positive effects of MM for CPP, such as an increased ability to control pain, improvements in mental health, emotional well-being, work and family life and social functioning (18,19), but these have never been examined in an RCT.
We therefore chose to evaluate MM delivered via an app to women with CPP as our intervention. CPP is especially common in younger women, who may be categorised as digital natives, making an app-based intervention particularly appropriate in this group.
In MM the aim is to keep focussed on one's own breathing. Whenever attention wanders to intrusive thoughts and feelings these are simply taken notice of in a neutral way, that is,  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  Evaluation of an existing app is often appropriate (24) and is both quicker and more costeffective than designing an app from scratch. We chose to evaluate an existing commercial app platform that teaches mindfulness by guided meditation (Headspace ®), with a ten day basic meditation module followed by a pain module specifically designed for the MEMPHIS (Mindfulness meditation using a smart-phone application for women with chronic pelvic pain) study. The Headspace app was publicly nominated favourite health app of 2013,(25) has a 5 star user rating in the Apple™ app shop and has scored top in a systematic review of 23 mindfulness apps using the Mobile Application Rating Scale (visual aesthetics, engagement, functionality or information quality) (26). Headspace had reportedly seen over 15 million downloads up to mid-2018 when our study began. (27) To our knowledge the Headspace app in its original or modified form has not been assessed in any other pain conditions.
We undertook a three-arm parallel randomised feasibility trial (MEMPHIS) (28) to assess whether or not to proceed with a full randomised controlled trial of the modified Headspace meditation app for women with CPP. In the current paper we report on the qualitative interview and focus group data from this study; the protocol and quantitative results have been  (29,30). Our aim for the qualitative part of this study was to determine whether a pre-existing smartphone app to teach MM is acceptable to women with CPP and can be integrated into clinical practice within NHS CPP pathways. Objectives were to consider: 1) The acceptability, use and usability of the app in the intended service user population and for health care professionals (doctors, health care assistants, clinical and research nurses) 2) The feasibility of integrating such an app into existing healthcare pathways 3) The usefulness of having a distinct patient group to advise us on the study design.

Outcomes
The outcomes of this analysis were inductively derived descriptive themes on acceptability, use and usability of the app and feasibility of integrating it into existing pathways. We follow the ISO 9241-11 (https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en) concept of technology usability (user friendliness) as the extent to which the app could be satisfactorily used by participants to meditate. By acceptability we mean whether participants could see a reason for using the app when given in clinic, and would be happy to use it for meditation.

Intervention Procedures
Women in the mindfulness meditation group received access to a 60-day progressive mindfulness meditation course delivered via the Headspace app. The first 10 days of the course taught basics of mindfulness meditation. Following this, participants were able to access the module on meditation which was targeted at for chronic pain. This module had min up to day 20 and 20 min up to day 60. The active control group received access to a series of muscle relaxation sessions. These sessions were identical every day, except that their duration increased to mirror the increasing duration of the meditation content being listened to by the intervention group. Usage data are reported elsewhere (30).

Patient and Public Involvement
We held a Patient and Public Involvement (PPI) group workshop before the study to discuss acceptability of the Headspace app and help us design our study. Women attending the Royal London Hospital CPP clinic were invited to volunteer for a week of using the unmodified (normal commercially available) Headspace app (which did not have the pain module at the time we undertook our pre study workshop) and then feed back on their experiences with the app in an evening discussion group. Women were not involved in the design of the modified app. The focus for the PPI group was on the use of the generic MM app.
Two patient representatives provided support from the study design stage through recruitment to the interpretation of the results and regularly attended Trial Management Group meetings.

Study recruitment and eligibility
The trial recruited at two outpatient gynaecology clinics within Barts Health NHS trust in two separate deprived areas of inner East London. Female patients with new or follow-up gynaecology appointments were assessed for eligibility by a researcher in clinic, having been posted a Patient Information Sheet. Women were eligible if they had been suffering with CPP for 6 months or more and had at least a basic understanding of the English language, sufficient to follow instructions, as assessed during discussion about the study for informed consent; no women were excluded on this basis. Women were excluded if they did not meet these criteria or they did not have access to a smartphone or personal computer or were currently using the Headspace app (there were very few of the latter, according to the impression of the recruiting nurses). All patients gave full and informed consent to be randomised and data were collected through all stages of the study. All healthcare professionals and research nurses involved in the two clinics were also invited to take part in the feasibility study as the only eligibility criterion for staff. Full enrolment data are provided in Forbes et al (30). A key difference of these patients from those in the PPI group was that their focus was on managing their pain, with the app given explicitly as part of their clinic management support.
For the study of quantitative data, 90 patients were allocated randomly in a 1:1:1 ratio to the mindfulness meditation app, a muscle relaxation app active control or the usual care arm (for full details see (29)). Patients in the two active arms were asked to download the modified app in the clinic with support from a research staff member and were sent a questionnaire about app usability, an analysis of which is reported in a companion paper. (30) We used data from the app usability questionnaire to inform topic guides for the qualitative part of the study. This outlined key usability issues that had been uncovered, to guide our semistructured interviews and focus groups with patients and staff.
All women in the intervention and active control arms were eligible for the qualitative component of the study, and all staff participating in the study.

Within-study interviews and focus groups
All patients in the intervention and active control arms were invited to one of two focus groups at their own study site after the 6-month follow up. We offered telephone interviews as an alternative. Patients were asked to 'walk through' the app with researchers, articulating their thought processes while they did so and commenting on its different specific features,  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  free flowing with open-ended questions, which gave us patient-focussed information on app acceptability, and part was structured using questions developed from the Normalisation Process Theory (NPT) toolkit in the way recommended by the NPT developers (32). For example, we asked whether staff could see a purpose to the app in clinical practice, as adding something different, which corresponds to the NPT toolkit question 'Participants distinguish the intervention from current ways of working'. Since this was a semi-structured approach questions were not rigidly worded. This helped us to consider the feasibility of integration of the app into practice. NPT is a theory of implementation practices that was initially developed for consideration of technology implementation and is in common use (32).
All data were audio-recorded at point of collection and transcribed, with personal identifying data removed from transcripts. Raw data were stored in a Primary Care Clinical Trial Unit database to clinical trial standards. Analysis of within-study focus groups and interviews was carried out blinded as to which study app was used, and deployed the immersion-crystallisation method (33). Thus, the lead qualitative researcher immersed herself in the data, reading transcripts carefully, then writing down articulated or crystallised patterns or themes that related to the aims and research questions of the study. These were discussed with another researcher from the team, and themes modified as appropriate. This process was repeated until all the data had been examined and all patterns that had been noticed were articulated, discussed and substantiated with exemplar extracts. This approach was considered appropriate since we had a small dataset and we were not aiming to develop conceptual themes but rather to inform the design and development of a randomised controlled trial for the modified app.

Analysis
We used the SRQR checklist when writing our report (34).

Ethics
The MEMPHIS trial was a three-arm parallel randomised feasibility trial approved by Camden and Kings Cross Research Ethics Committee in 2016 (15/LO/1967).

Results
We screened 488 women between May and September 2016 for their eligibility to participate in the study. After exclusions, 90 women gave full consent to participate and were randomised to the intervention arm (31 women), the active control arm (30 women) or the usual care arm (29 women).

Demographics
Women in the main feasibility trial (30) were aged a mean of 35 years, 66% were employed and overall approximately 50% had stayed in full time education until at least the age of 20 though the proportion was least, at 36.7% in the intervention arm. Overall 44% were of white ethnicity though the proportion was lowest in the intervention arm (35.7%) and highest in the usual care arm (53.6%). The second most common self-reported ethnic groups were  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y 12 'Southern Asian' and 'Black'. Women in the intervention arm were most likely to have experienced CPP for 3-5 years (40.3% of this group), women in the usual care group for over 10 years (42.9% of this group). More women had pain for longer than two years in the intervention arm than in either of the other two arms. All women reported a high pain intensity, with means of 6.8 to 6.9 in the previous week (on a scale from 1-10) (30). These and other demographic data are reported in more detail in Forbes et al. (30) Our qualitative sample was taken from the two active arms and was comprised of 16% of trial participants and 23% of those eligible for the qualitative study. We did not record separate demographic data for the women in this smaller sample.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y 13

Pre-study PPI group
The ten women in the pre-study PPI group were self-selected local women who were familiar with using apps and focused on app use per se. They anticipated no technical issues even in women who were not used to apps. They considered that the Headspace app would be successfully adopted by patients taking part in the study, given that a smartphone, like CPP, is 'always with you'. They praised the flexibility of the app, welcomed its portability and were unanimous in saying it was easy to slip off for 10 minutes when at work to use it. As a result, they found they could use it at times when they most needed pain relief as well as to prevent pain and found the app helpful in relieving pain and stress. The group reported being able to meditate without the app, once they had tried it with the app; however, they still preferred to use the app because they found the voice soothing.

App usage in the study
Patient usage of the app was less than expected from our pre-study PPI group discussions.
Only 36% of meditation app patients and 46% of the active control patients used the app at least once. (30)

Thematic analysis of within-study data
Qualitative data were obtained from 14 study patients; 12 preferred a telephone interview, two attended face-to-face interviews, one participant at the University attached to one of the recruiting clinics and one participant at the other recruiting hospital. Patients chose not to attend focus groups. Four of the patients were from the active control arm and 10 from the intervention arm. The two women we met face to face had both used the intervention and neither had progressed beyond the training stage, something that we cannot discount for other participants and which may help to explain reports of lack of effect on pain. Seven people attended the staff focus group: two recruiting nurses, three clinic nurses one consultant and a representative from the pelvic pain support network.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  These are explored below. As the PPI group data were not research data we did not analyse them for themes.
While we initially combined active control and intervention groups in our analysis, we then looked for instances where there was a difference between these two groups. Only where we found this difference in any theme or statement have we specified which group women belonged to.

Familiarity and capabilities with app technology
Around half of the patients were sufficiently familiar with technology and apps to be comfortable using the study apps. However, six participants (all using the intervention, which was more complex than the active control) reported difficulty because they were "not very good at technology" (patient 1002, intervention), or were unsure how to get started or use the app effectively without help.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y 15 control) changed handsets and therefore did not continue with the app. In all cases these technical difficulties appeared to lead to abandoning of the app or restricted use of its functionality.
Five patients having technical problems suggested possible solutions such as a 'class' or group for first time users, a YouTube orientation video, or a pictorial leaflet. This might include an introduction to meditation and mindfulness as well as the app itself. One woman commented: "If your market is targeting people who are not using apps then you are going to have to get together and find ways to do this", (patient 1041, intervention); she also suggested we could get ideas from other apps on the market in this regard.
Given their experiences in the study, staff were concerned about additional staff time needed to support women in using the app. This would sit in tension with one of the original rationales behind choosing an app as the mode of delivery, which was to increase the effective use of staff contact time with patients. Language barriers might compound problems.

Motivations to use the app
Staff, though unaware of the low sustained app use in the study, felt it would be common sense to hold occasional motivating meetings with patients if the intention was for them to use the app long term. The patient data suggested the main motivators or lack of motivation for using the app in our sample, which could be drawn on in such meetings, and which we now consider.

Perceived benefits
Three intervention arm patients said they only entered the trial to help others through research but were already using alternative forms of pain control. They explained that this meant they were not motivated to actually use the app, perceiving the relative benefit to be  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  There was no clear pattern regarding the impact of current pain on app use by patients. Six said they used it regardless of pain intensity -sometimes developing a daily routine -while four only used it when in severe pain or expecting to be (e.g. during menstruation). This cyclical or intermittent use in some patients -which was irrespective of study arm -should be considered when looking at our main study outcomes. (30) The Headspace app requires regular use to learn and benefit from psychological techniques. To address this, healthcare professional alerts have been effective in other studies, (36) whilst Headspace only has a reminder function that the user can set. This was often not sufficient, as one patient said even with this feature, "To be quite honest I used it a couple of times and then forgot. And then I [remembered it and] used it more frequently." (patient 1036, intervention) One patient said medication was not working but the app did, though she was not sure whether this was 'mind over matter' (1065, intervention), which was her term for a placebo effect. Three others said it did not reduce their pain; all three were using the intervention app. The remaining patients considered other benefits were good reason for using the app even when they did not feel that it reduced pain intensity. Alternative or unanticipated benefits were not formally measured or taken into account in the study's effectiveness  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y 17 outcomes (30). For example, ten patients valued the way the app helped them to relax or destress or focus and re-assess their life; three of these specifically said they used it to induce a relaxed state to get to sleep. Notably the active control was a relaxation app; however this benefit was also reported by many women in the intervention arm. One participant (active control) said she did not like the focus on pain per se as her condition impacted on various areas of her life. Even when the intervention app was positively received, women might stop using it because it was too powerful, and they had gained the change they wanted: Three intervention app patients found the app put them more in tune with their bodies and their breathing, (two of these were among those who also found the app de-stressing) while another found yoga better for that. Six patients, like pre-study PPI group members, also learned to use techniques from the app to alter their stress patterns without the app, having tried it, for example in traffic or by sitting down and taking time out or for general relaxation.

Relation to other therapies
Two patients (both active control) preferred 'pure' meditation, another considered the app to be "very much about meditation" (1041, intervention), which is in keeping with the arms they were in. An alternative therapy practitioner and two further patients reported that they preferred yoga. One (active control) said this was because it focussed on each part of the body in turn.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y 18 Three intervention app patients thought the app was useful as an adjunct to other methods rather than a replacement for them, for example physical interventions such as Pilates, or listening to classical music.

Opportunities to use the app
Technology issues getting in the way Staff pointed out that not all patients had smartphones (not appreciating that PCs/tablets were alternatives allowed in this study). Some patients lacked the storage space to load the app on their phones. There were also issues with Wi-Fi connectivity when staff tried to help the patients load the app within the hospital sites. Possible solutions that staff suggested were to lend patients phones and to have group upload sessions in a location with good Wi-Fi signal -though they acknowledged the resource implications.
Life getting in the way Seven patients revealed they preferred to use the app in the evenings because of other life commitments. This meant they did not always use it as a direct response to pain, reducing its potential for contemporaneous effect. One patient who used it in response to pain but only used it once or twice blamed this on having no spare time because of juggling work and children (active control); however, another patient (intervention) managed despite such commitments the fact that she was in the intervention arm may have played a role.

Barriers to integration for staff
Staff believed that the biggest barrier to clinical adoption of the app was a possible lack of support from the host organisation. It might also be hard to integrate the app within existing professional work practices if the staff in the position of offering the app to patients failed to see its relative advantage over other interventions. Collection of feedback on the app's effectiveness would be necessary for staff to support sustained use. It was felt that staff would need training on how to introduce the app to women in practice, and that complexity  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y   19 and high staff turnover could impede sustained use. An app was also seen as impersonal compared with face-to-face contact, which was more favoured by staff.

Participant comments on the research process
The study questionnaires that were used for the main quantitative outcome measures (30) were acceptable to patients except for some discomfort with a question about sex, which patients considered a delicate question that was missing a 'no sex' option. Most preferred a paper form reflecting their lack of affinity with technology. There were no indications that the study design or study processes had contributed to the participants' lack of engagement with the apps -with a caveat around support with the technology as mentioned above -though we did not systematically consider this. A full summary of patient comments on the study design and procedures is given in Appendix 1.

Discussion
Our study adds to the limited evidence on mHealth app user behaviour and experience (36,37). The pre-study PPI group (young women, of a generation who were familiar with using apps and who were asked to focus on the study design use of the app) liked the idea of delivering the intervention via an app, praising the contemporary design and flexibility.
Hence we expected a similar positive attitude from trial participants, who were of a comparable age, and we assumed would be keen on using apps. Participant feedback revealed that this assumption was too simplistic.
Using our qualitative data, we were able to explore reasons for low app usage that had been recorded in our feasibility study. (30) Our thematic analysis suggests that the low app use in the trial occurred because many patients were not familiar with apps in general or lacked capabilities with technology. This was particularly true for the more complex intervention app. The other themes we report did not differ between groups (although the three cases of weak altruism' all occurred in the intervention arm) which suggests more generic issues that  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y   20 can be transferable to other app use situations. For example, women stated limited motivation to use the app because of a lack of perceived benefit, or a lack of opportunity to use the app due to Wi-Fi issues or due to other commitments. Headspace app exposure. Like us, they reported barriers of busy lives, failure to establish a routine and a lack of perceived benefit; all users in their study tried the app at least once hoping it could deliver a quick fix but were disappointed if this did not happen. In our study many patients failed to perceive a benefit from using the app. Hence excuses stating other commitments may mask a deeper lack of motivation linked to perceptions of benefits.(39) The advantages and disadvantages of using the app stand-alone were also illustrated by our data. Some suggestions made by participants to improve usage, such as more guidance at the start, seem obvious in hindsight. But they had not been considered because of the feedback from the pre-study PPI group and the commercial success of Headspace. The use of community contacts may be a helpful alternative.(40) Social support can create a community of practice, help to clarify expectations,(41) and improve health outcomes (as shown for example in internet based psychological treatment for depression (42).
The data suggest that for successful app use we need to understand what motivates individuals with clinical need to use the app for clinical reasons (which our PPI group did not focus on) and target this, for example by setting appropriate expectations. Incentivisation might also improve motivation. This could be achieved through app gamification (7), or encouragement through integration with patient-clinician face-to-face encounters, which was lacking in our study since the app was used stand-alone. The present study provided extensive initial technical support but no coaching and incentivising, in keeping with the protocol. Future app studies should take this into account. Participants in our study may  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  Lack of engagement after recruitment, or good initial engagement but minimal or inconsistent use, have been reported in other studies (44,45), including in Headspace trials other than our own (46,47). Settings were a university and a psychiatric inpatients clinic, both in the USA. Inconsistent app use was noted by Wen (48) in junior doctors who used self-guided Headspace. Morrison Wylde (49) compared face-to-face MM with headspace use in novice paediatric nurses. However, unlike our study there were no recorded dropouts/non-users and also no record of whether or how long the app was used for which is an important omission.
None of these studies included a qualitative component. Yet, each of the Headspace study groups were very different, and so will likely have differed in motivations, contexts for opportunity to use the app, and incentivisations (50). While these aspects were not considered in the other studies, our use of qualitative research has enabled us to explore these in more depth. Our findings suggest these are important considerations in any study of app use and therefore this study makes a contribution to the field. For example, some of the groups in other studies may have differed from ours in likelihood of using mhealth apps in the first place, and familiarity with technology. In-patients may have more time to use the app and more support -and may also have had specific barriers to app use, such as related to setting and to illness.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y   22 Patients in the qualitative part of our study tended not to use apps on a regular basis (or at least apps other than simple games), and in terms of our themes, also represented in the COM-B model, may be said to have few capabilities in technology use. They therefore do not represent the typical users of the Headspace app in a commercial setting. Accessing the app regularly requires energy, time and effort, but patients with CPP often suffer from fatigue and anxiety as co-morbidities, perhaps whilst having to juggle family life and work.
Therefore, this may be seen as a challenging clinical population in which to trial an app.
Our study has also shown that clear consideration of unexpected benefits should be included in future studies and these can be informed by our finding that benefits for patients may be more diffuse than anticipated (e.g. app relieving stress rather than pain). Such benefits were found in the active control as well as the intervention arm and so it may be that they represent a placebo effect though the effect could equally be real. Our data also suggest that staff benefits may be less than anticipated, as participants sometimes needed a lot of support and scaffolding in technology use at least initially.
Young age, co-morbid anxiety and low educational attainment are predictors for dropping out of web-based interventions according to studies in the field of depression. (51,52) This may be true despite regular phone support (52) though our participants all considered active motivational support from staff or app support groups would have improved app use. Our intervention arm participants were particularly likely to be young and with low educational attainment.
Our data suggest that it is important to involve real world end users in the agile design or development or modification of apps in close collaboration with researchers and commercial app developers. (7) Although the evaluation of existing apps has been recommended as a  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y   23 cost-effective and rapid process, (24) our findings suggest that in actual clinical practice these may be problematic.

Strengths and weaknesses of the study and in relation to other studies
One strength of this study is that it creates much-needed evidence in the field of evaluating existing health apps in a clinic population (6,8,24) and recording user experience. This provides us with lessons to be learned.
Researchers conducting interviews and focus groups were: a senior mixed methods medical sociology researcher, a recruiting nurse, a representative from the pelvic pain support network and an experienced health psychologist. Findings were similar across the data and the different backgrounds of the researchers therefore does not appear to have influenced findings. The main analysis was undertaken by the medical sociologist and so the concordance with the COM-B model is not due to background discipline bias.
We were able to recruit successfully, and we obtained valuable information from patients with CPP, who were recruited from a deprived urban area of the UK as typical local clinical patients.
However, we report a marked discrepancy between the attitudes of the pre-study PPI group of volunteer patients from the local area, who actively put themselves forward for a 7-day trial of the app, and the participants asked to take part when they attended clinics. The opinions of pre-study PPI volunteers meeting in their private time may not be representative of the opinions of patients recruited at a routine clinic appointment. Women in the PPI group were used to using apps, which had led them to be interested in the study in the first place.
Whereas women in the PPI group had all trialled the app at home and work without support from us, many patients from clinic were unable to use their phone beyond calls, texts and photos. Moreover, most of the women we interviewed used the intervention app. We can only speculate as to why this is so but it does mean that concordances and divergences across the intervention and active control arm do need to be treated with circumspection.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60   F  o  r  p  e  e  r  r  e  v  i  e  w  o  n  l  y   24 To our knowledge the present observation on failure of PPI work to translate into practice in a trial has not been formally reported before, and is lacking from a recent comprehensive systematic review.(53) PPI involvement is a stipulated requirement when applying for some funding, and the present research findings should be taken into account when drafting guidelines for future PPI involvement in study planning. PPI groups are able to provide significant help and advice in any study but our findings shows the value of adding agile codevelopment as a requirement for app intervention development as likely to provide a more effective intervention than one informed by PPI alone. Moreover, there is a difference between app use for active clinical management (as with our study participants), and consideration of the potential for app use for this (as with our PPI group).

Implications for clinicians and policymakers
Given the patchy use of the app and the way that some participants did not manage to unlock its full functionality, and an indication of diffusion of benefit, more work is needed to see whether the app reduces pain per se. This study is a good example of the need to move away from 'one size fits all' behavioural interventions. Future studies should do more work on implementation before doing an effectiveness trial. This will enable researchers to be more nuanced about saying who the app is effective for, if at all.
Strategies to involve busy, less motivated, and less technologically experienced individuals in PPI and lay app design groups need to be further developed. These groups should include considerable scaffolding, which we have shown extends to study involvement by patients. More care is also needed to obtain PPI input that is representative of the target group, taking into account their capabilities, opportunities and motivational aspects. It may be useful to give the PPI group a small condition management task that emulates what trial participants will be required to do. Moreover, we can confirm a recent review suggesting  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  that health apps should be co-designed with users (7), rather than presenting them with a pre-existing app. These implications for our study are also generalisable to other technology studies.

Author contributions
EB led the study as the CI. EB and CR were the main authors of the grant application for this study, and co-lead authors of the current paper. FR, SJCT, JD, JB, SN and LS contributed to the study design and initial protocol. All authors provided support throughout the trial and contributed towards the final paper. CR led on the PPI and CR and LS led the interview and focus group field work and analysis reported here. CR, LS, SN, CCC, JD and JB were all involved in the field work.

Role of the funding source
The UK National Institute of Health Research, Research for Patient Benefit (RfPB No. PB-PG1013-32025) funded the MEMPHIS study. The funder had no role in the study design, in the collection, analysis, and interpretation of the data, in the writing of this report, or in the decision to submit the paper for publication. The first and last authors vouch for the integrity, completeness and accuracy of the data and analyses, and for the fidelity of this report to the protocol and statistical analysis plan. The views and opinions expressed herein are those of the authors and do not necessarily reflect those of the RfPB, NIHR, NHS or the Department of Health.

Competing interests
We have read and understood BMJ policy on declaration of interests. We have no competing interests to declare.

Data sharing
The data are collected from a small number of people which could compromise their identity if shared with others. Therefore we are not making them available except under exceptional circumstances which will be determined by the custodian of the data (Elizabeth Ball) on an individual basis.  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  The lead author confirms that the manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned and registered have been explained.

Graphics
The pictures were often not appropriate as they focused on the problem (e.g. tooth pain logo) rather than something positive or soothing. Unanticipated problems could arise according to one participant; she found it hard to lose weight and felt the food icons a) did not represent success and b) reminded her of eating.
One participant found the pictures hard to see.
Psychological theory supports the view that positive images would be advisable Another participant, with cyclical pain and hence cyclical use, did not aim to progress but simply chose to use the baseline video each time.
Three participants were frustrated and even 'angry' and 'patronised' at having to do the same (basic) meditation several times before moving up a stage.
There was no indication of goals, duration of each level or gamification rewards.
Goal setting is recommended by psychological theory and could be as simple as using "day 1 of 30" rather than simply Day 1.
Changing emojis could show progress, for example from a sad face to a smiley face to a heart; the inbuilt progress function was not felt sufficiently motivating.

Introduction
Experienced app users suggested the introduction could be improved for initiates overview of the entire app and its levels suggested for the start.
One participant had done mindfulness before but thought an introduction to mindfulness might be helpful for others.

Enjoyment
considered fun by some participants.

Instructions to authors
Complete this checklist by entering the page numbers from your manuscript where readers will find each of the items listed below.
Your article may not currently address all the items on the checklist. Please modify your text to include the missing information. If you are certain that an item does not apply, please write "n/a" and provide a short explanation.
Upload your completed checklist as an extra file when you submit to a journal.