Article Text
PDF
Abstract
Objectives The Acne Priority Setting Partnership (PSP) was set up to identify and rank treatment uncertainties by bringing together people with acne, and professionals providing care within and beyond the National Health Service (NHS).
Setting The UK with international participation.
Participants Teenagers and adults with acne, parents, partners, nurses, clinicians, pharmacists, private practitioners.
Methods Treatment uncertainties were collected via separate online harvesting surveys, embedded within the PSP website, for patients and professionals. A wide variety of approaches were used to promote the surveys to stakeholder groups with a particular emphasis on teenagers and young adults. Survey submissions were collated using keywords and verified as uncertainties by appraising existing evidence. The 30 most popular themes were ranked via weighted scores from an online vote. At a priority setting workshop, patients and professionals discussed the 18 highest-scoring questions from the vote, and reached consensus on the top 10.
Results In the harvesting survey, 2310 people, including 652 professionals and 1456 patients (58% aged 24 y or younger), made submissions containing at least one research question. After checking for relevance and rephrasing, a total of 6255 questions were collated into themes. Valid votes ranking the 30 most common themes were obtained from 2807 participants. The top 10 uncertainties prioritised at the workshop were largely focused on management strategies, optimum use of common prescription medications and the role of non-drug based interventions. More female than male patients took part in the harvesting surveys and vote. A wider range of uncertainties were provided by patients compared to professionals.
Conclusions Engaging teenagers and young adults in priority setting is achievable using a variety of promotional methods. The top 10 uncertainties reveal an extensive knowledge gap about widely used interventions and the relative merits of drug versus non-drug based treatments in acne management.
- DERMATOLOGY
- QUALITATIVE RESEARCH
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This study has demonstrated that teenagers and young adults can be engaged in priority setting, but considerable resources went into promotional activities that were subsequently found to be ineffective.
To ensure all views were captured, much effort went into collecting responses from a wide spectrum of people with acne and different types of care professionals; despite this, males with acne were under-represented.
The volume of unsorted questions was so large that grouping into broad themes was the only way of generating manageable numbers to take to the prioritisation stages without overwhelming participants.
Saturation was reached in that no new uncertainties were contained within the final submissions to the harvesting survey.
Methods were developed for sorting and sharing large volumes of submissions.
Introduction
During the last decade, the inclusive methods pioneered and validated by the James Lind Alliance (JLA) for treatment-related research priority setting have become well established in the UK (http://www.jla.nihr.ac.uk). JLA Priority Setting Partnership (PSPs) bring together, on a level playing field, people with a disease and those who treat them, first to identify, and then to prioritise, unanswered questions (uncertainties) about existing treatments. Twenty-six PSPs have been completed to date, including this and a further three on diseases of the skin.1–3 Conditions covered include those primarily affecting the elderly (eg, dementia and stroke), infants and young children (cleft lip and palate, eczema) or people at any time of life (Lyme disease, asthma). So far, no PSP has targeted a disease with peak prevalence during adolescence in mainly healthy subjects. Although acne is starting earlier4 and lasting longer,5–7 possibly as a result of lifestyle changes, peak prevalence is between the ages of 16 and 20 years.8 ,9 The age range of acne now spans five decades; few teenagers in Westernised societies are able to avoid acne in one form or other.10 For reasons that remain poorly understood, postadolescent acne is more common in women than in men.5–7 Acne was one of three skin conditions in the top 10 most prevalent diseases worldwide in 2010.11 However, the most widely used treatments have changed little in the past 30 years. Systematic reviews have consistently shown a paucity of robust evidence from adequately powered randomised controlled trials.12 ,13 When compared with disease burden as estimated by disability-adjusted life years from the Global Burden of Disease 2010 project, acne is under-represented in the Cochrane Database of Systematic Reviews.14
Most clinical trials of new and existing therapies in acne have been conducted by the pharmaceutical industry; few independent trials are conducted anywhere in the world. In consequence, key issues of importance to patients and clinicians remain to be adequately investigated and addressed.
The Acne PSP was formally initiated in November 2012. In order to collect a representative range of opinions, we sought to involve not only National Health Service (NHS) patients and staff but also the large numbers of people with acne who never present to a doctor.15–19 Of importance, we considered that a successful priority setting exercise, by highlighting significant evidence gaps, would stimulate new high quality treatment-related research within and beyond the UK.
Methods
A steering group to oversee the PSP was established in accordance with JLA guidelines and held its first meeting in January 2013. The steering group, which was chaired by a representative of the JLA, consisted of people with acne, and healthcare and allied professionals involved in treatment delivery, as well as experts in patient and public involvement in research and information management. At the initial meeting, the protocol and terms of reference for the steering group were formally adopted. The protocol was developed with reference to the JLA guidebook with modifications necessitated by targeting a predominantly adolescent population.
Stage 1: set-up
Organisations that represent people who treat acne in any setting were contacted and invited to become partners. In the UK, there is no longer a patient group representing people with acne. However, we invited VERITY, the support group for women with polycystic ovarian syndrome, to become a partner as persistent acne is often a feature of this condition. Organisations that volunteered to help deliver the priority setting exercise were also invited to become partners. Partner organisations were expected to help promote the PSP to their members or to the public and encourage participation in the harvesting survey and vote. They were required to affiliate to the JLA. Our partners are listed in the Acknowledgements section.
As the target population was expected to comprise mainly of teenagers and young adults, it was decided that a dedicated website should be used to collect and disseminate information. The website was launched in April 2013 and included background information about the PSP, partners and steering group members, with a brief summary of acne treatments and ways of obtaining them.
In order to better understand what would motivate people to take part in the harvesting survey, an informal meeting was held in February 2013 in Harrogate, to which approximately equal numbers of people with acne, and healthcare professionals, were invited together with two dermatology patients without acne but with marketing experience. Attendees reviewed and changed the publicity flyer and harvesting survey, which had been drafted to closely match the traditional open JLA layout. To make the harvesting survey form more visually attractive and clearer for young people, two versions were produced. The patient version was more structured, including images and specific questions about each treatment type, and contained additional questions to capture details about current and past sources of professional help and treatment. The professional version of the survey adhered to a more conventional layout with a limit of five questions per respondent. Publicity materials were simple in design, carried the banner “Join the fight against acne” to reflect the campaigning dimension requested by meeting participants, and included a QR code linking directly to the survey. To increase participation by teenagers and young adults, a small financial incentive was offered to respondents to the patients’ version. This was £25 or local currency equivalent in vouchers for a global online retailer, awarded to 50 people who submitted at least one uncertainty and whose names were selected at random at the close of the survey. No financial incentive to take part was offered to the professionals.
Stage 2: harvesting uncertainties
Both versions of the survey to collect treatment uncertainties were constructed in Survey Monkey and embedded into the PSP website. The survey was open between the 22 May and 31 August 2013. It was kept open longer than the planned 8 weeks after analyses of demographic data showed low levels of participation by some target groups.
A variety of traditional as well as novel methods were utilised to publicise the survey. Organisational stakeholders were asked to promote the PSP and survey to members via email, via their own web site or in any other way they wished. Posters and flyers were sent to local and specialist centres within the UK. In addition, a national chain of community pharmacies distributed flyers via their branches. A dedicated Twitter account was set up and at least one original tweet per day was issued until the final workshop had been completed. Steering group members were asked to email colleagues and contacts and/or put adverts in local newsletters. Two national health-related organisations, Talk Health and Embarrassing Bodies, promoted the survey via a variety of mechanisms including their own web sites. A celebrity agreed to endorse the PSP on the home page of the web site and another generated a promotional video. The local National Institute of Health Research network (North East Yorkshire and North Lincolnshire) took the lead in disseminating information to colleagues in other networks, including Primary Care Research Networks, and also approached local educational establishments and NHS trusts.
To complement the harvesting survey, a search of research recommendations within recent, relevant and reliable systematic reviews or treatment guidelines, was undertaken with a view to including any novel uncertainties within the prioritisation exercise.
Stage 3: processing the uncertainties
The survey generated very many responses, so consideration was required of the most efficient mechanism for analysing the data and distributing workload across the project team. Submissions from Survey Monkey were downloaded into Microsoft Excel to facilitate refining and collating uncertainties. Submitted text was often rephrased for clarity and to separate out individual questions.
To maximise flexibility and to enable the data to be sliced in various ways, controlled vocabulary terms were assigned to the uncertainties. First, uncertainties were collated into themes, many of which were necessarily broad (covering aspects such as adverse effects, long-term management and skin care). The themes were supplemented with terms relating to relevant intervention categories (eg, phototherapy, antibiotics and topical therapies). These controlled lists were defined using the clinical expertise within the Steering Group. The key concepts were then indexed by assigning National Library of Medicine Medical Subject Headings (MeSH), commonly used in searching literature, to each of the included uncertainties. Not all concepts were available in the MeSH vocabulary. To identify controlled terms for these omitted concepts, the Excel file was loaded into Google Sheets and parsed with the ISA-TAB Ontomaton tool.20 As a result, the medical subject headings were extended to include terms from other vocabularies such as Clinical Terms V.3 (Read Codes; National Health Service National Coding and Classification Centre).
Most of the data were validated on input to minimise typographical errors and ensure consistency across the subsets of data; the exception being one of the MeSH descriptor fields that enabled more obscure concepts to be included. There was no limit to the number of key words that could be assigned to any uncertainty. Therefore, each uncertainty could be indexed to a high level of detail if needed, facilitating segmentation of the data and enabling similar submissions to be merged.
In order to manage the above process and the distribution of the data across the project team, the datasets were stored in a central online document repository. Initially based on the National Institute for Health Research (NIHR) SharePoint portal, the files were later transferred to a dedicated site on the NIHR Hub when the NIHR moved its infrastructure onto the Google platform. The list of uncertainties generated by this process was reviewed at a meeting of the Steering Group to decide which to take forward to the ranking stage. The group was provided with information on the number of related submissions so that the shortlist was decided largely by popularity of the uncertainty.
As a final check, the International Clinical Trials Registry (http://apps.who.int/trialsearch/) and the Database of Abstracts of Reviews of Effects (DARE) were searched to ensure that no uncertainty was already being addressed by an ongoing clinical study or had been the subject of an existing or planned high quality systematic review.
Stage 4: voting and ranking
Because large numbers of young people were expected to take part in the vote, the number of questions on the shortlist was limited to 30. As in the harvesting survey, the voting form was constructed in Survey Monkey and embedded in the PSP web site. Each respondent was asked to choose the three questions they felt were most important and to rank them. The order of questions on the voting form was random. The vote was open from 22 December 2013 to 10 February 2014. The same methods used to promote the harvesting survey were used to promote the vote.
A weighted ranking system was used to generate scores from the vote; the first choice question scored three points, second choice two points and third choice one point. Summed scores from patients and professionals were calculated separately; summed scores from professionals were adjusted to take account of the lower number who voted, so that ranks could be validly compared from both groups. Votes from respondents who identified themselves as researchers or ‘other’ were excluded. The Steering Group appraised the ranked scores from patients and professionals and selected 18 uncertainties to take forward to the priority setting workshop based on the highest ranking by both groups.
Stage 5: priority setting workshop
The final stage of the PSP was the workshop. This was held at the London headquarters of the British Association of Dermatologists (BAD), on 04 March 2014. Equal numbers of people with acne and healthcare professionals working within and outside the NHS were invited to attend. Representatives of partner organisations and Steering Group members not regularly involved in patient care attended as observers. To make informed decisions, participants were provided with ranks from the vote and the number of related submissions from the harvesting survey.
Sample size and composition
Sample sizes were available from two previously completed PSPs on eczema and vitiligo; 493 and 461 participants, respectively, submitted uncertainties to each PSP. Since acne is a more common condition, the protocol for the Acne PSP set an aspiration of 600 respondents for the harvesting survey. While no formal target was set for the vote, numbers and sample composition were closely monitored so that the voting period could be extended if necessary. For both the harvesting survey and the vote, considerable efforts were made to collect responses from representative samples of people with acne and the professionals who care for them. To assist with this, extensive demographic information was collected for the survey and the vote.
Results
Harvesting and collating uncertainties
A total of 4363 people submitted the harvesting survey form. However, many patients and professionals did so without including a question. A majority of patients and parents who did not ask a question completed other parts of the survey form, whereas healthcare professionals and researchers who did not include a question left the entire form blank. Usable submissions containing one or more uncertainties were received from 1636 patients or family members and 652 professionals. Following rewording and refining, there were 8276 questions. After removing those that were out of scope (ie, relating to diagnosis, pathogenesis, genetic predisposition, new drug development or access to treatment), not answerable by research, already answered or not about acne, a total of 6255 questions remained.
Few respondents submitted questions about specific interventions; the exception was oral isotretinoin. In consequence, the Steering Group decided, at a special face-to-face meeting, not to generate a long list of specific uncertainties. To produce a manageable short-list for voting, it was agreed to sort questions into broad themes with the specific aim of including as many of the original questions as possible within these. Rigorous efforts were made to avoid bias by collating questions in different ways and counting the number of questions in each set before the final themes were selected. Additionally, members of the steering group were asked to review the tentative groupings and could ask for new themes to be evaluated. An example of a broad uncertainty with samples of illustrative questions included within this theme is given in online supplementary table S1.
The number of people submitting questions relating to each theme was calculated separately for respondents to each version of the survey and compared. The most common themes are shown in table 1. Over a quarter of all respondents (27%) and over a third (37%) of respondents to the patient survey asked about the safety and/or efficacy of physical therapies. This was the intervention type about which most questions were asked. Some uncertainties were submitted by similar proportions of patients and professionals, whereas there were marked differences for others.
Uncertainties identified by 200 or more respondents to the harvesting survey
The 29 most popular themes were taken forward to the vote together with an additional uncertainty about the prevention and management of post-inflammatory pigmentation, which was identified by many people with acne in pigmented skin. Together, the top 30 themes encompassed more than 87% of the submitted questions. The question “What are the best ways to support self-management of acne?” was constructed to reflect the large number of submissions, impossible to count accurately using keywords, that reflected people's struggles to manage their acne without professional help. Popularity of a theme in the harvesting survey did not predict popularity in the vote or ranking in the top 10 (see below).
Characteristics of survey respondents
Extensive demographic information was collected to determine whether a representative sample had been obtained. Interim analyses showed that women were over-represented in the patient sample. Keeping the survey open longer and specifically targeting men via Twitter did not improve participation by males with acne. At the close, the professional sample comprised of 64.2% women (see online supplementary table S2), and the patient/family member sample comprised of 83.1% women (see online supplementary table S3). Among the professionals, most submissions came from doctors, nurses and pharmacists, working in primary or secondary care or in the community, but treatment providers working outside the NHS were also represented (see online supplementary table S2). Among respondents to the patient version were 1125 individuals with acne at the time of completing the survey, 331 who had acne in the past as well as 132 family members. The age range was as expected: 40% of respondents to the patient version were aged 16–24 years (see online supplementary table S3). The patient sample was predominantly white (80% vs 87% for the UK in 2011 census) but a good spread of minority ethnic groups was represented. Almost a quarter of the patient sample (23%) lived outside the UK compared with 15% of the professional sample. Within the UK, all but five postcodes were represented in the sample of patients and family members with a large number of responses (33%), as might be expected, from Yorkshire, where the PSP management team and the local NIHR research network were based. A majority of professionals (73%) did not disclose their postcodes.
People who had acne at the time that they completed the survey were asked to provide information about sources of professional help to ensure that the sample composition included individuals seeking advice and/or treatment from a variety of sources and not just from within the NHS. Forty-one per cent were not getting any professional help, although 88% had sought help in the past (table 2). The most common source of help was a general practitioner. Interestingly, 26% of respondents had seen a beauty therapist in the past and 34% had sought the advice of a pharmacist. Other sources of professional help not shown in table 2 included paediatricians, gynaecologists, nurses, health counsellors and the British Association of Skin Camouflage. A minority of respondents said they obtained help from family and friends, the internet and/or social media such as YouTube (an online video sharing website), or became their own expert.
Sources of professional help used by respondents (n=1125) who had acne when they completed the harvesting survey
The most common source of acne treatment was from a pharmacy with a prescription (see online supplementary table S4). At the time of completing the survey, 40% of people with acne were not using any treatment but 82% had used treatment in the past.
How people heard about the harvesting survey
Respondents were asked to identify how they heard about the survey, and the answers are shown in table 3. The commonest mechanism identified by both patients and professionals was email. Among the professionals, many (21%) heard about the survey from a colleague. Many patients and family members had heard about the survey via social media (notably Facebook and YouTube), with several specifically mentioning a celebrity video. Methods with negligible impact were the Acne PSP website, Twitter account and the printed word. Three times as many patients heard about the survey from the Embarrassing Bodies or Talk Health web sites than from the PSP's site. There was some evidence of propagation with several people mentioning organisations we had not directly targeted.
How respondents heard about the harvesting survey
Ranking exercise
A total of 3084 people took part in the online vote. Of these, 277 votes were excluded because the respondents identified themselves as a researcher or ‘other’. Valid votes were received from 1573 people with acne, 237 family members and 1012 professionals. Again, males were under-represented; only 25% of patient voters were male compared to 35% of professionals. A demographic analysis of votes is shown in table 4; 44% of voters were aged between 16 and 34 years. A large majority of voters (85%) lived in the UK and 81% were white. More than half the patient voters lived in Yorkshire (53.6%). This time, 78% of professionals provided a postcode; of these, 21% practiced in Yorkshire. All but seven postcodes were represented in the patient sample and all but 15 in the professional sample. Google Analytics showed that most people landed directly on the voting survey without visiting other pages before leaving the PSP web site.
Demographic analysis of the vote
For several uncertainties, there were marked differences in weighted ranks between patients and professionals (figure 1). For others, there was good agreement, especially among the less popular questions. Popular uncertainties with the biggest differences in weighted scores (patients vs professionals) were about use of oral isotretinoin (191 vs 693), ways of preventing acne (1006 vs 461) and the use of antibiotics (282 vs 972).
Flow diagram showing the number of participants and submissions at each stage.
The Steering Group used ranked weighted scores to decide which uncertainties to take forward to the final workshop. The distribution of scores (figure 2) showed a long tail and, with one exception, uncertainties in the tail were not taken forward. The exception was the uncertainty about the best way to use oral isotretinoin, which ranked fifth among professionals but 19th among patients and family members. This meant a total of 18 uncertainties were considered at the final workshop.
Comparison of summed weighted scores from people with acne versus care professionals. Summed weighted scores from patients, partners and other family members are shown in blue; summed weighted scores from professionals are shown in orange.
Final priority setting workshop
A total of 43 people, including 13 patients, 12 professionals and 13 observers, attended the final workshop. In the first session, three groups comprising equal numbers of patients, professionals and non-participatory observers, each with an independent moderator, were asked to prioritise all 18 uncertainties using a nominal group technique. The results were collated and discussed in one combined afternoon session, moderated by the chairman of the steering group. The final top 10 set is shown in box 1. Seven uncertainties were ranked in the top 10 without alteration. There were three instances in which two related questions were merged; all three merged questions were also ranked in the top 10. The top 10 set was announced the following day via the Acne PSP website, Twitter and several partner web sites. Subsequently, thank you postcards promoting the top 10 were disseminated in Harrogate dermatology clinics, at European and Global Alliance acne meetings and through a meeting supported by the British Association of Dermatology. Given the number of more specific questions, work is still ongoing to optimise methods of disseminating these to patients and professionals. In due course, all verified uncertainties will be entered into the UK Database of Uncertainties about the Effects of Treatment (http://www.library.nhs.uk/duets/).
The top 10 research priorities for the treatment of acne
1. What management strategy should be adopted for the treatment of acne in order to optimise short and long-term outcomes?
2. What is the correct way to use antibiotics in acne to achieve the best outcomes with least risk?
3. What is the best treatment for acne scars?
4. What is the best way of preventing acne?
5. What is the correct way to use oral isotretinoin (Roaccutane) in acne in order to achieve the best outcomes with least risk of potentially serious adverse effects?
6. Which lifestyle factors affect acne susceptibility or acne severity the most and could diet be one of them?
7. What is the best way of managing acne in mature women who may/may not have underlying hormonal abnormalities?
8. What is the best topical product for treating acne?
9. Which physical therapies, including lasers and other light based treatments, are safe and effective in treating acne?
10. How long do acne treatments take to work and which ones are fastest acting?
Discussion
The top 10 acne research priorities, reached by consensus between patients and professionals, reveal concern about the paucity of evidence on the relative efficacy and safety of commonly used treatments, and their place in both short and long-term management. This evidence gap may reflect the low number of robust, industry-independent trials of acne therapies but also that efficacy within RCTs does not reliably predict effectiveness or patient satisfaction in the real world. In the past 5 years, no published acne RCTs have been conducted in whole or in part within the UK. Within the Cochrane library, there are only two published Cochrane reviews wholly dedicated to acne therapies, although there are six ongoing Cochrane reviews that will provide new insights into some of the uncertainties in the top 10, specifically the efficacy and/or safety of oral isotretinoin, light therapies and topical treatments. Identifying the top 10 priorities of patients and healthcare professionals highlights the need for more pragmatic trials that put widely used interventions head-to-head, and also more focused research to identify the safest and most effective alternatives to drug based therapies. While awareness of these alternatives among patient responders was high, so was scepticism about paying for such treatments from unregulated practitioners outside the NHS. Several uncertainties that were very frequently identified by respondents to the harvesting survey, including two of the three most asked about intervention types (complementary and alternative medicines (CAM), cosmetic remedies), did not make it into the top 10. Although there was clearly a lot of interest in them among people with acne, they did not rank in most voters’ top 3 when presented alongside the other shortlisted questions.
Among submissions to the harvesting survey, we received many treatment-related questions not answerable by research and so not included in the prioritisation exercise. They asked about such things as availability of non-drug based treatments within the NHS, and how to identify reliable sources of advice about their safety and efficacy. There were also many questions about affordability and accessibility of treatments not provided by the NHS. These questions reflected valid concerns among people with acne and should be addressed by information providers, policymakers and regulatory bodies.
Almost half the people who submitted the harvesting survey did not include a question, although most patients and parents did provide demographic information and contact details. This may reflect the offer of a small financial inducement even though the survey form clearly specified that inclusion in the prize draw was dependent on submitting at least one question. Feedback from local patients indicated that they were definitely interested in the survey (sufficient to want to open it) and curious about its content. The issue was that they had nothing specific they wanted to ask or did not know how to frame a research question. It remains unclear why 302 professionals and 30 researchers did not submit a question.
At the time the Acne PSP was set-up, the JLA process of priority setting through partnership and consensus had already become established. Several PSPs have revised the basic procedures laid down in the JLA guidebook and adopted new strategies to overcome difficulties associated with specific diseases or hard-to-reach groups. For example, the stroke PSP devised a new model for assisted participation and targeted engagement of stroke survivors with communication difficulties.21 No previous priority setting exercise has been completed for a very common condition that primarily affects teenagers and young adults. We recognised that they might be hard to engage and motivate in an exercise such as this. There are at least three million people with acne in the UK at any one time. A sample of 0.05% would represent a minimum of 1500 people. While this sounds easily achievable, in fact it was extremely challenging. A huge amount of work by large numbers of people went on behind the scenes to achieve this. Initial discussions with young people suggested that it would be necessary to break with JLA tradition in several ways, including: a very different layout for the harvesting survey, the use of a small financial inducement and emphasis on the use of social media to promote participation. Whether due to the layout of the harvesting survey or not, we obtained a much broader range of uncertainties from patients than from professionals, almost certainly as a consequence of reminding people what treatments are available. Perhaps as a result of opening people's eyes to treatments they were not aware of, we also received many questions from patients about therapies not available within the NHS, most commonly physical treatments as well as CAM. In contrast, two types of drug treatment dominated questions from professionals: antibiotics and oral isotretinoin. Among numerous questions relating to the conduct of a consultation for acne, many asked about doctors’ reluctance to engage in conversations about alternatives to commonly used drug-based therapies.
Feedback from respondents showed that some promotional activities were much more successful than others and this information may be of use to those conducting PSPs in future. It was almost certainly not necessary to have a dedicated PSP website or Twitter account and leafleting via a national chain of pharmacies was not as successful as we had hoped. The role of partner organisations was vital in publicising the PSP and encouraging participation. With such a prevalent condition, we felt it was necessary to demonstrate that respondents to the harvesting survey and vote were representative of all people with acne and all professionals who offer care.
The demographic information we collected showed a pronounced local concentration of survey respondents in Yorkshire, where both the PSP management team and local NIHR network, which helped to promote the survey, are based. While the effort: reward ratio, in terms of respondent numbers to hours spent on promoting the survey, might be low, it guaranteed that we achieved a representative sample in terms of age, location and ethnic mix. With patient gender, we were not so successful. Ours is not the first or only PSP to find that only a minority of respondents were male,3 ,22 ,23 although many PSPs do not report the gender of participants. It is perhaps particularly surprising for acne, as there are fewer treatment options for men. The smaller gender bias among healthcare professionals is not of concern as it simply reflects the 60:40 ratio of females to males within dermatology, pharmacy and general practice in the UK. Weighted ranks show that uncertainties voted for by men and women were broadly similar with the exception of a question about managing acne in mature women (data not shown). This suggests that gender bias in the sample may not have affected the findings of the PSP to a significant extent. However, we perhaps could have made greater attempts to target males, for example, via sports clubs, magazines, men's lifestyle websites or schools for boys.
Because of the large number of questions submitted, our approach, as in several PSPs before us, was to merge very specific questions on related themes into broad questions that could be voted on. Any PSP has to balance the sample size required to reach saturation (ie, few or no new uncertainties identified by increasing participation) with the practicalities of processing the large volumes of data generated. The novel use of key words based on MeSH descriptors in this PSP was essential to facilitate sorting and grouping. Following the PSP, preliminary discussions have been held with National Institute for Health Research (NIHR) Evaluation Trials and Studies Co-ordinating Centre (NETSCC) about translating the top 10 uncertainties into researchable questions that faithfully represent the original submissions. A forthcoming challenge is to capture and disseminate, via UK DUETs and other mechanisms, the insightful and more detailed questions that have been lost as a result of merging.
Acknowledgments
The authors thank to everyone who submitted uncertainties, participated in the vote and/or attended the prioritisation workshop. The authors also thank the following partner organisations for promoting the survey and vote to members: American Acne and Rosacea Society, British Association of Beauty Therapy and Cosmetology, British Association of Dermatologists, British Dermatological Nursing Group, British Skin Foundation, British Society for Paediatric Dermatology, Canadian Acne and Rosacea Society, Cochrane Skin Group, Lloyds Pharmacies, Primary Care Dermatology Society, Society for Academic Primary Care, South Asian Health Foundation, Talk Health Partnership, UK Dermatology Clinical Trials Network, VERITY. The authors were ably supported by our Steering Group and would especially like to acknowledge the enthusiasm of our patient representatives, Julia Jordan, Karen Thomas and Charlotte Jones. The authors are grateful to Martin Welch, Dipika Patel and Mick Mullane, of the North East Yorkshire and North Lincolnshire Clinical Research Network in York, and Tom Newbound, NIHR Learning and Development Manager, who helped harness the power of the NIHR networks to promote the study, and assisted with publicity and data management. The authors also wish to express our gratitude to Kim Thomas, Tessa Clarke and Carron Layfield of the Centre of Evidence Based Dermatology (CEBD), University of Nottingham, for their encouragement to undertake this PSP and their ongoing guidance. Thanks are also due to the clinical and nursing staff in the Department of Dermatology at Harrogate and District NHS Foundation Trust and Ketaki Bhate of the CEBD who helped rephrase, sort and collate submissions to the harvesting survey, and to participants in the discussion group who informed the design of the harvesting survey and publicity materials. Finally, the authors thank Anton Alexandroff for setting up and managing our Twitter account and to Samina Walimohamed, Claudia Donkor and Katerina Steventon, who energetically promoted the PSP to patients and colleagues.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
Footnotes
Contributors All the authors made substantive contributions to the conduct of the PSP, including promoting the survey and vote to patients and colleagues, as well as the collection, analysis and/or interpretation of data. The manuscript was drafted by EAE and HW. All authors except HW were members of the Steering Group and all including HW were involved in review and revision, and approved the final version. The project was managed by EAE and AL, with administrative support from Elizabeth Oldham.
Funding This work was supported by grants from the UK Dermatology Clinical Trials Network and the Society for Academic Primary Care.
Competing interests None declared.
Ethics approval Ethical approval was obtained from West of Scotland REC4 (reference number 13/WS/0015). NHS governance approval was obtained from Harrogate and District NHS Foundation Trust. Both approvals were required in order to embed a research question about outcomes important to patients within the harvesting survey. The findings from that will be published separately.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Additional data can be accessed via the Dryad data repository at http://datadryad.org/ with the DOI:10.5061/dryad.gf1k0.