Article Text
PDF
Abstract
Objectives Because healthcare and community organisations and treatment methods are always changing, continuous changes might also be needed in questionnaires that register patient-reported outcomes (PRO) and patient-reported experiences (PRE) of healthcare interventions and community support. Thus, the aim of this study was to test the content and face validity, including the readability, of two questionnaires used by the Swedish national stroke register Riksstroke to register PRO and PRE at 3 and 12 months poststroke.
Design Clinicians’ and patients’ knowledge and experiences of current care, rehabilitation, community support and functioning after stroke as well as comments noted regarding the content and layout of the questionnaires were retrieved in focus-groups with expert clinicians and in patient interviews analysed with content analysis. A workgroup of experts with experience in stroke care, rehabilitation and research repeatedly revised the questionnaires regarding content, layout and consistency throughout the validation process.
Participants The participants included allied healthcare professionals, nurses and physicians with extensive experience of working with stroke care and rehabilitation (expert clinicians) as well as patients who had suffered a stroke 3 or 12 months earlier and who were purposefully selected among those who had completed and returned the 3-month questionnaire.
Setting Expert clinicians met at their work place in focus-groups. Patients were interviewed where they resided, that is, in their home or nursing home, including rural, town and city areas in Sweden.
Results Based on clinical expertise and comments from the patients (n=47), the questionnaires were revised and then found to be valid in terms of content validity and face validity, including readability.
Conclusions The present evaluation emphasises the need for testing aspects of validity, including readability, of questionnaires addressing PRO and PRE and for the recurrent revision of such questionnaires in order to maintain their validity in a society undergoing constant change.
- stroke
- questionnaire
- validity
- patient reported
- outcome
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
The strength of the present study is that it includes a thorough description of a methodology that can be used to identify patient-reported outcome and patient-reported experience that are relevant to patients, clinicians and researchers.
The method used in this study provides evidence to support the choice of items and includes input from patient and expert clinicians and documented results supporting the modifications of items.
A limitation of the present study is that the validation method used might be considered time-consuming.
Background
The Swedish stroke register, Riksstroke, was launched in 1994. One main aim of Riksstroke is to evaluate the hospitals’ adherence to the National Board of Health and Welfare’s recommendations for emergency treatment at stroke onset.1 Since 1998, all emergency hospitals in Sweden are enrolled, and with a coverage of >90%, approximately 23 000–25 000 patients treated for an acute stroke diagnosis are registered annually.2 Over the years, Riksstroke has expanded its registrations to incorporate data regarding care during the acute phase and rehabilitation phase, secondary prevention, community support and patient perceptions of their level of functioning and disability and their perceptions of the healthcare interventions that are provided to them.1
The Riksstroke steering committee has developed quality indicators, for example, care in the stroke unit, and outcome indicators, for example, survival rates, that are reported by each hospital through web-based reporting.1 National quality and outcome indicators are currently being developed in other European countries, but for stroke audits, there is still no European-wide consensus on content, data documentation, definition or development process for quality indicators. Commonly, the main focus is on acute stroke care, and an overview of six European stroke audits shows that among the 123 identified quality indicators, only two are used in all audits (anticoagulants in patients with atrial fibrillation and brain imaging) and another five are used in all but one audit (stroke unit care, swallowing test, antiplatelet/antithrombotic therapy, lipid lowering therapy and thrombotic therapy at discharge).3 Notably, to our knowledge, Riksstroke is the only instance in Europe that collects both patient-reported outcomes (PRO) and patient-reported experiences (PRE) of healthcare interventions after stroke.3 4 However, the use of PRO and PRE of healthcare interventions to evaluate and develop the healthcare system is of growing interest, and the necessity of involving the patients in the development of PRO and PRE measures has been stressed.5
Two questionnaires have been developed by the Riksstroke steering committee in order to evaluate PRO and PRE, including perceived functioning and disability, for example, dependence in activities of daily living (ADL) and depression, and how the healthcare interventions that were provided were perceived. The first follow-up questionnaire is administered by the hospital and posted to the all registered patient who had a stroke 3 months after stroke onset.4 At 12 months, an additional follow-up questionnaire, administered by Riksstroke, is posted to all of the registered patients who had a stroke.4 The questionnaires are usually completed by the patient, but they can be completed by a significant other or by healthcare personnel. In 2013, 86% responded to the questionnaire administered at 3 months after stroke onset, and 79% responded to the questionnaire administered at 12 months after stroke onset.2 6 Non-respondents have more commonly been found to be (1) women, (2) of an older age, (3) living alone and (4) ADL-dependent prior to stroke onset.7
The data collected by Riksstroke are used to evaluate the quality of stroke care and provide feedback on how the development work and clinical quality improvements are progressing within healthcare at a local, regional and national level. At a national level, data from Riksstroke are used to evaluate the adherence to the National Guidelines for Stroke Care.1 8 Data from Riksstroke provide a unique opportunity to pursue research based on national data, and they are frequently used for that purpose.9 Consequently, it is plausible that these research results might have an impact on decisions made by the responsible authority in the healthcare organisation10 as well as on the agenda of patients’ organisations and the opinions of the general public.
A basic precondition that needs to be met before results from any questionnaire used to assess PRO and PRE can be considered to be credible is that the collected data are valid, that is, the data collected can be expected to capture the essence of what the target group considers to be of importance. To ensure the questionnaire’s ‘content validity’, a group of experts educated in the target field can be consulted. A questionnaire expected to evaluate an individual’s perceptions and self-rated functioning will also need to be tested on the target group, for example, individuals who have suffered a stroke, in order to ensure ‘face validity’. This test needs to be performed in order to ensure that the questions included are relevant and sufficient to describe the individual’s perception of, for example, their functioning and disability. Notably, another precondition that needs to be met is that the questionnaire can be interpreted and completed correctly, that is, it has good ‘readability’.11 12
When developing PRO and PRE questionnaires to be used in the stroke population, the effects on functioning and disability need to be considered. The effects of stroke are diverse in expression and degree and might involve both mental and physical impairments that restrict activity and participation. This diversity puts great demands on the content of a questionnaire intended to give a true picture of relevant aspects of perceived functioning and disability and interventions provided after stroke. Moreover, the effects of stroke that might affect an individual’s ability to complete the questionnaire need to be considered. Impaired initiative and concentration, tiredness and limitations in reading and writing put great demands on the questionnaire’s readability. Furthermore, limitations in fine hand use might negatively affect an individual’s ability to complete a questionnaire independently.
Today, there are psychometrically tested assessment tools available for assessing PRO related to body function, activity and participation after stroke (eg, Stroke Impact Scale,13 HADS,14 EQ-5D,15 VAS-scale for pain,16 Fatigue Severity Scale17). However, to send a battery of these forms along with added questions on PRE is likely to put too much strain on the patient and will most definitely affect response rates negatively.
In addition, continuous changes in the questionnaires used to assess PRO and PRE such as Riksstroke’s might be necessary considering that healthcare and community support organisations and diagnostic, prognostic and treatment methods are constantly developing and changing over time. The original versions of Riksstroke’s questionnaires were developed by its steering committee, including professionals with clinical and research expertise within the field. The questionnaire used in the 3-month follow-up has undergone several major revisions since 1994, some in cooperation with patient organisations. From the original questionnaire in 1994, six questions still remain, although in an extensively modified form. Twenty-three questions from the questionnaire in 2004 remained in 2013; however, the phrasing of 13 of these questions has been modified over the years. The original version of the questionnaire used at the 12-month follow-up has been in use since 2009 and has been revised on one occasion. Twenty-nine questions from the original questionnaire in 2009 remained unmodified in 2013. The number of questions in the questionnaires has gradually increased without negative effects on the response rate.1
Nevertheless, in the every-day healthcare context, clinicians working in the patients’ homes report that a fair number of patients are in need of support with reading, interpreting and completing the questionnaire used at the 3-month follow-up. Among researchers, a need was identified to revise the recurrence of questions in the 3-month and 12-month questionnaires to enable evaluations of changes over time and to test the questionnaires’ ability to capture topics that are relevant to patients after stroke. Thus, the aim of the current study was to continue the work initiated by the Riksstroke steering committee in developing the questionnaires used at 3 and 12 months after stroke by testing these for content validity and face validity, including readability. The work carried out in this current study was initiated by the register manager of Riksstroke.
Research questions
Are the questions in the questionnaires used by Riksstroke at 3 and 12 months after stroke onset relevant to patients with stroke in terms of functional outcome and life situation after stroke, and do they capture experiences of healthcare interventions?
Are there unclear or ambiguous questions or wordings that might prevent a correct interpretation and the ability to correctly complete the questionnaires used by Riksstroke at 3 and 12 months after stroke onset?
Methods
Data collection
In figure 1, a flowchart of the validation process is presented.
Flowchart of the validation process of the 3-month and 12-month questionnaires.
The first author (SP), a female registered physiotherapist specialised in neurology and with a PhD and extensive clinical research experience within the field of stroke rehabilitation and evaluation and the methods used, coordinated the data collection and collated all data from the participants, that is, the patients and the expert clinicians. Participants were informed about SP’s health professional and research background, the purpose of the study and their rights as participants. There was no relationship between SP and the participants prior to the study start. The data collection procedures were based on the methodology proposed by Streiner and Norman11 and were pretested in a smaller study by the authors (SP and DS).12 In the smaller study, a workgroup developed a questionnaire aiming to map the long-term effects of stroke in a younger stroke population based on meetings with a group of expert clinicians and younger patients living with stroke who were asked to comment on the questionnaire while completing it. New younger patients living with stroke were included until no new comments regarding the content or format of the questionnaire were identified.12
Expert clinicians partaking in the testing of the content validity, including readability, of the original questionnaires
During the identification of clinicians in the present study, a representative sample of the different professions active in clinical stroke care and rehabilitation settings was sought. Moreover, a representative sample of clinicians working throughout the care trajectory after the referral from the emergency hospital was sought as well as a representative sample of experts working in different geographical areas. Thus, experts with many years of experience of stroke care and rehabilitation of patients at 3 and 12 months after stroke onset were approached. Two groups of experts were working in a city (Stockholm, the capital of Sweden) and one group was working in a town and rural area (Umeå, a town with rural surroundings). These clinicians are henceforth referred to as ‘expert clinicians’. According to written instructions by email and based on their clinical knowledge and experience, the expert clinicians were asked to prepare relevant questions to ask patients at 3 and 12 months after stroke onset. Thereafter, they were instructed to analyse the original questionnaires so as to assess how the construction and content of the questionnaires captured the patients’ life situation and behaviours after stroke and how the questionnaires could provide sufficient information to draw conclusions about the target group, that is, patients who are living with the effects of stroke (content validity). In addition, the expert clinicians were asked to analyse the format of the questionnaires so as to ensure the correct interpretation of the questions (readability) by patients who had a stroke. Thereafter, the expert clinicians met in their respective focus-groups18 at the participants’ workplace. The meetings were booked for 2 hours. During each meeting, the 3-month and 12-month questionnaires were analysed, and single questions and the questionnaires as a whole were discussed. The leader of the focus-group (SP) took notes that were written out in a fair copy and emailed to the respective focus-group members for their approval.
Revisions of the questionnaires made by workgroup members
Another group of clinicians and researchers with expertise within the stroke field took part during the planning and realisation of the study. This group is henceforth called ‘the workgroup’ and included allied healthcare professionals and physicians with extensive experience of working with stroke care and rehabilitation during the first year after stroke onset. Except for one person, the register manager of Riksstroke, no workgroup member had taken part in previous revisions of the original Riksstroke questionnaires. Therefore, an external testing of the questionnaires could be performed in co-operation with a representative from the Riksstroke steering group who could follow and contribute in the working process. Using the comments from the expert clinicians taking part in the revision, and the workgroup’s joint expertise within the field, a first revision of the original questionnaires was made. Further revisions made by the workgroup were based on comments by patients and the workgroup’s joint expertise. No members of the workgroup took part in the focus-groups of expert clinicians described above.
Inclusion of patients and the interview process—testing of face validity, including readability, after the first revision of the original questionnaire
In order to identify patients who complete and return the questionnaires2 and to achieve a geographical representation of patients living in rural, town (Umeå and Västerås, two towns with rural surroundings) and city (Stockholm) areas, purposefully selected patients were identified. Eligible patients were identified in Riksstroke’s data register (emergency hospital data collected at stroke onset and at 3 months poststroke) among patients who had returned the 3-month questionnaire. The selection was made based on registered data on sex, age, functioning and disability. The patients who were approached had completed the questionnaire independently or with the assistance of a significant other. Eligible patients were contacted by a letter including study information, and this was followed by a telephone conversation. Patients who gave their consent to take part in the study were booked for an interview in the patient’s home.
The interview was booked for 1 hour and was carried out by the first author (SP). The process started with a recorded interview that took place where the patient resided, that is, in the patient’s home or nursing home. To identify question areas that were relevant to patients living with effects of stroke (face validity), the participants were asked to express their perception of the most significant effects of stroke at 3 and 12 months (respectively) after stroke. Thus, the patients were asked to appraise if and how their life situation had changed after stroke onset and whether they had any comments regarding the care and rehabilitation interventions they had received. Thereafter, each patient completed and commented on the questionnaire that had been revised by the workgroup based on the comments of the expert clinicians. The patients’ comments regarding the relevance of the questions and their ideas for questions that should be added were noted as well as comments regarding the format of the questions, including perceived difficulties in interpreting and completing the response alternatives (readability). Notes were made in the questionnaires and were approved by the patients during the interview. New purposefully selected patients were included and interviewed until no new comments regarding the content or format of the questionnaires were identified.11
The recorded interviews were transcribed verbatim and were analysed during the ongoing interview process using content analysis.19 With a deductive approach,19 the International Classification of Functioning, Disability and Health (ICF)20 was used to identify areas described by patients in the interviews. The ICF components and domains related to physical and mental functions, activity and participation and personal and environmental factors in which healthcare and community support are included were used as a matrix in the analysis.19 In this way, areas perceived as relevant by the patients could be identified without directing the patients towards areas already included in the questionnaires. The analysis was performed by the first author (SP), and the results were presented and discussed with the workgroup members.
Results
Expert clinicians partaking in the testing of the validity, including readability, of the original questionnaires
The participating expert clinicians represented inpatient and outpatient care and rehabilitation as well as a nursing home for older patients and patients of working age (table 1).
Participating expert clinicians’ profession, clinical representation and experience of patients at 3 and 12 months after stroke onset
Examples of given comments:
A need to group the questions under headings, for example, ‘mental and physical function’ and ‘care and rehabilitation’ because it might be hard for these patients to change focus.
A need for uniformity between the 3-month and 12-month questionnaires in order to enable evaluation of change over time.
A need for additional questions regarding secondary prevention.
A need for additional response alternatives regarding impaired mental and physical function, activity limitations and participation restrictions after stroke onset.
During the review of the questions, the expert clinicians provided suggestions on the content and formulation of each question so as to make it easier for patients to interpret and complete the questionnaire correctly. Comments that were given in the focus-group were compiled and discussed in the workgroup. Thereafter, the questionnaires were revised before the interviews with the patients started.
Test of face validity, including readability, of the 3-month and 12-month questionnaires that had been through a first revision based on the comments from the expert clinicians
Participating patients
The patients who participated in the interview process used to test the face validity, including readability, of the questionnaires after the first revision based on the comments made by the expert clinicians are presented in table 2. The patients who took part in the testing of the 3-month questionnaire had suffered a stroke 3–5 months earlier. The patients who took part in the testing of the 12-month questionnaire had suffered a stroke 13–15 months earlier. No one other than the patients and the researcher were present during the interviews.
Presentation of patients who participated in the interview process to test the face validity and readability of the questionnaires after the first revision
Among the identified patients, 6 could not be reached by telephone and 24 declined to participate during the conversation. New strategic selections of patients were made in order to substitute for those who could not be reached or who declined participation.
Interview process
Phase 1
The results of the analyses of the 11 patients who had been interviewed and who had completed and commented on the 3-month questionnaire and the 9 patients who had been interviewed and who had completed and commented on the 12-month questionnaire revealed a common need for a larger revision of the content and layout of the questionnaires. In the interviews, the patients reported impairments related to, for example, continence and balance, additional diagnoses, limitations in transportation, a need for mobility aids, assistance with private finances and information about stroke to be important areas that were not covered in the original 3-month and 12-month questionnaires.
Next, while the patients were completing the questionnaires, difficulties related to the ability to differentiate care and rehabilitation interventions from community support were noted. Furthermore, patients with additional diagnoses tried to differentiate the effects of stroke from the effects of their other diagnoses. As a result, they only reported disability if they considered their disability to be related to their stroke instead of choosing response alternatives that matched their actual functioning and disability. In addition, these patients did not choose response alternatives that matched the care and community support they were actually receiving. A lack of suitable response alternatives when a patient was not in need of an intervention that was focused on in a question resulted in patients reporting that their needs were met or not met at all.
These results were presented and discussed in the workgroup, and the questionnaires were revised accordingly. To enable analyses of change over time, the 3-month and 12-month questionnaires were analysed and revised simultaneously.
Phase 2
The interview process continued in accordance with the method described. During the selection of patients, a spread in age, sex and functioning and disability was sought (as presented in table 2). When five selected patients had commented on the 3-month questionnaire and another six selected patients had commented on the 12-month questionnaire, some questions were considered to be in need of further revision. Based on difficulties identified related to reading, comprehension and completing the questions correctly as well as a lack of appropriate response alternatives, these questions were considered to be in need of a new layout and rephrasing of response alternatives. These questions were related to healthcare and community support as well as, for example, ambulation. Suggested revisions were commented on by the workgroup, and the questionnaires were revised accordingly.
Phase 3
In the last phase, a selected sample of nine patients commented on the 3-month questionnaire and another seven patients commented on the 12-month questionnaire. The interview process was ended when no new difficulties related to completing the questionnaire or lack of response alternatives were identified. At this point, the need of further minor revisions was identified by the workgroup. This need was based on the difficulties identified related to comprehension of and filling in one question correctly and some ambiguities found in a few response alternatives. The patients who reported difficulties were asked to give their view of the proposed alterations. These proposed alterations were in line with alterations previously made to similar questions where no new difficulties had been identified in the subsequent interview process. The proposed alterations were all approved by the patients who had reported difficulties, and a final revision was made with the support of the workgroup.
Workgroup members’ contributions
Based on the comments from the expert clinicians and the patients, possible solutions to enhance the content and face validity and readability of the 3-month and 12-month questionnaires were discussed among the members of the workgroup. The length of the questionnaires was a recurring topic. Because mental impairments are commonly reported after stroke,21 a questionnaire that is too extensive might impair the patient’s ability to complete the questionnaires. Given this background, it was considered vital that the format of the questionnaires be as simple as possible in terms of readability, but still include the areas identified as relevant by expert clinicians and patients (content and face validity). The workgroup also found that, based on the comments from the expert clinicians and patients, the number of questions and response alternatives needed to be extended.
Because the Riksstroke data are used in research, the workgroup, with its representation of researchers, also discussed the content of the questionnaires from a research perspective. As a result, a uniform format of the 3-month and 12-month questionnaires was sought in order to enable comparisons of outcomes at 3 and 12 months poststroke.
Results of the validation process
The final 3-month questionnaire was expanded from 29 to 41 questions (of which five were follow-up questions). Questions included in the original questionnaire, questions that were revised (in terms of formulation and response alternatives) and added question areas are presented in the online supplementary appendix. During the interviews conducted while the patients were completing the questionnaires, the patients were asked to comment on the length of the questionnaire and whether they found any questions to be unnecessary or missing. Of the 25 patients who completed a revised version of the questionnaire, 22 patients had no comments regarding the number of questions in the questionnaire (not too many or too few) and found the questions to be relevant. Three patients considered that the questionnaire consisted of too many questions, and these three patients had all needed assistance from a significant other to complete the original questionnaire.
Supplementary file 1
The final 12-month questionnaire was expanded from 30 questions (of which one was a follow-up question) to 43 questions (of which five were follow-up questions). Questions included in the original questionnaire, questions that were revised (in terms of formulation and response alternatives) and added question areas are presented in the online supplementary appendix. None of the 22 patients who participated had any comments regarding the number of questions in the questionnaire (not too many or too few), and all but two found the questions to be relevant. These two patients, who were independent in ADL, thought that some of the questions were more suited to patients who had suffered a more severe stroke.
Discussion
Riksstroke’s 3-month and 12-month questionnaires were revised based on clinical expertise and a purposefully selected sample of patients. After the revision, the questionnaires were considered valid in terms of content validity and face validity, including readability. The method used in this study is extensive and might be considered time-consuming. However, the results clearly point to the value and necessity of including clinical expertise and academia and the targeted patient group when tailoring PRO and PRE measures.
Revisions resulting in changed questions, as in the present study, might affect the ability to compare old and new data. However, the data collected after such a revision will be retrieved from questions that have been systematically reviewed and considered relevant by the target group. Furthermore, the revised questionnaires are designed to be interpreted and completed correctly. Thus, a revision as such might lead to greater gains than losses.
Contribution of the expert clinicians
Given the representation of various professions among the expert clinicians, different effects of stroke could be pinpointed in the areas of physical function, mental function, activity, participation and environmental factors. When the original design of the questionnaires was analysed, unclearness and ambiguities were identified, particularly concerning issues in distinguishing interventions within a healthcare setting from services provided through community support. The results of the present study highlight the importance of involving clinically active experts of different professions in the process of validating questionnaires addressing patients’ functioning and disability and the current interventions provided by the healthcare and community support systems.
Patients’ contribution
A wide range of patients was selected, including patients with mild to more severe mental and/or physical impairments and activity limitations. Notably, the proportion of patients with disability might have been affected by the fact that eligible patients were identified among patients who had completed the 3-month questionnaire independently or with the assistance of a significant other and not by proxy. Moreover, at the time of inclusion the participating patients were slightly younger (mean age 73 years) compared with the patients registered in Riksstroke (mean age 76 years).22 This difference was due to the fact that several patients of working age needed to be included so as to validate the work-related questions in the 12-month questionnaire. The proportion of women and men in the selected sample was in line with the total number of patients registered by Riksstroke (48% women).22 A larger proportion of patients living at home with home help service were included (36% compared with 21% among the patients registered by Riksstroke) as several patients needed to be included to revise questions regarding support from the community. A smaller proportion of patients in the selected group were living in a nursing home (8% compared with 16% reported by Riksstroke).22 In Sweden, residents in nursing homes live with severe disability and are thereby less likely to fulfil the inclusion criteria of the present study regarding the ability to complete the questionnaire independently or with the assistance of a significant other and not by proxy. This is, however, a subgroup of patients that should be focused on in future studies to explore their level of functioning and need of support from the society. In the present study, patients with reported limitations in reading and writing were included and the selection resulted in 36% who reported limitations in reading and 20% who reported limitations in writing compared with 21% and 28%, respectively reported by Riksstroke.22 A majority of the questions underwent some kind of revision regarding content, wording and/or layout. This demonstrates the need to test a questionnaire on the target group for which it is intended.
PRO and PRE in a national and international perspective
Riksstroke in Sweden is the longest-running national stroke registry in the world.23 Other countries with national stroke registers for monitoring and improving the quality of hospital care include, for example, Australia, Canada, England, Finland, Scotland and USA. The most commonly reported outcomes in these registers include survival status, functional status, recurrent stroke and place of residence, but PRO are rarely recorded in these registers.23 Because PRO provide additional valuable information in the assessment of health status in clinical practice compared with clinician-reported measures alone,24 it seems important to add PRO in stroke registers. Though PRO are scarce in stroke registers, they do occur in other contexts, and the countries that are most advanced in implementing PRO at a national or jurisdictional level are England, the Netherlands, Sweden and USA.25
On a national level, the Swedish Riksstroke steering committee is working to identify a limited number of indicators that clearly reflect the quality of care provided to patients who had a stroke.26 These indicators cover aspects of prestroke functioning and living conditions as well as process and outcome indicators throughout the care trajectory.1 Indicators on process and outcome might provide relevant feedback on quality improvements and lead to new improvement plans. However, an indicator should always be developed in a systematic process tailored to ensure its validity. The present study presents a systematic validation process for two important aspects of evaluating process and outcome, that is, ensuring the validity of questions targeting PRO and PRE in a stroke population.
Comparisons between European countries would also help to identify appropriate components of stroke services and provide new insights into how best to configure and run stroke services given that the comparisons include validated data. This paper presents a method that might be used to identify and formulate common indicators for PRO and PRE in Europe.
The method used in this study is supported by the US ISPOR Patient-Reported Outcomes Task Force, where the expert panel has identified four threats to validity when choosing and modifying PRO measures.27 These were found to be (1) a lack of evidence to support the choice of items or measures, (2) a lack of direct patient input, (3) a lack of documented results supporting the modifications of items and (4) a mismatch between what is evaluated and the intended claim.27 The first three threats are addressed in the present study. The fourth, concerning the intended claim and the use of the PRO and PRE items, has been addressed by the Riksstroke steering committee, and the outcome of the validation process presented in the present study has been considered by the committee in their revisions of the questionnaires. The questionnaires may now be tested further for reliability in terms of stability, responsiveness and concurrent validity.
Conclusion
Riksstroke’s 3-month and 12-month questionnaires were revised based on clinical expertise and testing on purposefully selected patients. After revision, the questionnaires were considered to be valid in terms of content validity, face validity and readability, and they may now be used by the Swedish Stroke Register to map PRO and PRE of healthcare interventions found to be relevant to patients after stroke. The present evaluation emphasises the need for testing aspects of validity, including readability, of questionnaires addressing PRO and PRE and for their recurrent revisions in order to be valid in a society undergoing constant change.
Acknowledgments
We wish to thank the workgroup members who collaborated with the authors in the validation process: Magnus von Arbin, MD, associate professor specialized in acute stroke; Kjell Asplund, MD, Professor Emeritus, former register manager for Riksstroke; Lena Henricson, speech and language therapist and stroke coordinator; Carina Hortans, occupational therapist specialized in stroke rehabilitation, and Anna Söderholm, MSc, project leader. We also wish to extend our gratitude to the expert clinicians and the patients who had a stroke for contributing with their experiences and knowledge and for taking the time to comment on the questionnaires.
References
Footnotes
Contributors SP and DS both contributed with the methodology. SP collected all of the data. SP and DS contributed with critical analysis of the methodology and wrote the manuscript.
Funding The present project was funded by the Swedish national quality registries (nationella kvalitetsregister).
Competing interests None declared.
Patient consent Detail has been removed from this case description/these case descriptions to ensure anonymity. The editors and reviewers have seen the detailed information available and are satisfied that the information backs up the case the authors are making.
Ethics approval Regional Ethical Review Board in Umeå, Sweden (2012-294-31M).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional unpublished data from the study are available.