The modified Delphi process
| Criteria | Expert workshops | Pilot testing | Round 1 survey | Round 2 survey |
|---|---|---|---|---|
| Panel size | Northwest Invited n=25 Attended n=15 Southwest Invited n=25 Attended n=19 Public advisory group Invited n=11 Attended n=8 | Invited n=11 Responded n=10 | Invited n=740 Opted-out n=23 Responded at round 1 n=318 | Eligible n=318 Opted-out of round 2 n=3 Invited to participate in round 2 n=315 Responded at round 2 n=231 |
| Reminders | NA | Yes×1 | Yes×2 | Yes×2 |
| Response rate | NA | 91% | 43% | 73% (of 43%) |
| Area of expertise | Members of the public User/academic/clinical researchers Research managers Research commissioners | Members of the public User/academic/clinical researchers Research managers Research commissioners | Members of the public User/academic/clinical researchers Research managers Research commissioners | Members of the public User/academic/clinical researchers Research managers Research commissioners |
| Problem exploration | Round-table discussions/group activities to explore normative debates around the value/potential impacts of PI | Questionnaire—questions derived from literature review and Expert Workshop outcomes with five-point and seven-point Likert scales for close-ended questions Open question options | Questionnaire—as for pilot testing with revisions to unclear questions and formatting Additional open questions added to provide further opportunities for comment | Questionnaire—questions derived from analysis of round 1 responses with five-point Likert scale for close-ended questions |
| Consensus | NA | NA | 70% endorsement with at least 55% in the extreme category=critical consensus 60% endorsement=clear consensus | 70% endorsement with at least 55% in the extreme category=critical consensus 60% endorsement=clear consensus |
| Feedback | Expert Workshop outcomes fed back to participants and members of the Public Advisory Group | Consultation process | Expert panel members fed back responses with response %age of their own sub-group and those of other sub-groups Summaries of comments made by respondents also fed back | Wide-spread project dissemination of findings: Study report(s) Workshops Conference presentation(s); peer-reviewed journal publication(s) |
| Access route(s) to data collection | Email Group discussions Video-conference | Email Face-to-face Tele-conference | Email Online questionnaire | Email Online questionnaire |