Article Text
PDF
Abstract
Objectives It is not known which of the many asthma-specific quality of life (QoL) questionnaires best capture the lived experience of people with asthma. The objective of this study was to explore patients' views of three commonly used asthma-specific QoL questionnaires.
Design Qualitative study using semistructured interviews.
Setting Primary and secondary care in Brighton and Hove, UK.
Participants 30 adult people with a physician-diagnosis of asthma who were asked to complete the Juniper Asthma Quality of Life Questionnaire (AQLQ-J), the Sydney Asthma Quality of Life Questionnaire (AQLQ-S) and the Living with Asthma Questionnaire (LWAQ) to elicit their views on the content validity of these.
Results Thematic content analysis revealed a lack of congruence between the concerns of people with asthma and the questionnaire content in terms of missing (eg, allergies) and irrelevant (eg, smoky restaurants) content. The AQLQ-J was perceived as a ‘narrow’, ‘medical’ questionnaire focused on symptoms, the environment and functional ability. In contrast, the LWAQ and the AQLQ-S were perceived to be ‘non-medical’. The LWAQ was described as a ‘test’ and as a wide-ranging, embracing and holistic questionnaire. Its strong emotional focus was irritating to some. The AQLQ-S was described as a simple, quick and easy questionnaire, although there was a perception that it was lacking in depth.
Conclusions Patient interviews highlighted strengths and shortcomings in the content validity of these three asthma-specific questionnaires. For patients, the AQLQ-S content seemed to be the most pertinent in its adequacy of coverage of medical, social and emotional aspects of health-related QoL in asthma.
- QUALITATIVE RESEARCH
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
There is a body of research comparing the psychometric performance of asthma-specific quality of life (QoL) measures, but the assessment of the validity of available measures requires an assessment of their content validity.
As no attempt has yet been made to assess the content validity of available measures from the patients' perspective, we used a qualitative interview technique to elicit how the experience of patients with asthma related to the content of available questionnaires.
Patient involvement in comparing and contrasting the validity of asthma-specific QoL measures has been limited to date, but is of key importance in guiding the choice of instrument in clinical and research settings.
We were able to recruit a diverse sample; there was a female preponderance and a lack of ethnic diversity.
Data were analysed by a diverse research team.
Data generated were rich, revealing the coexistence of missing, irritating, redundant and irrelevant content.
Introduction
To capture patient's perception of the burden of disease on their functional status and well-being, the use of patient-reported outcome measures (PROMs) in clinical trials has been recommended by regulatory bodies such as the US Food and Drug Administration.1 PROMs are also used in clinical audit, patient registries, quality management and routine healthcare.2 Health-related quality of life (HrQoL) is an outcome domain which is widely measured by PROMs in questionnaire form. A structured literature review of six asthma-specific HrQoL questionnaires concluded that they differed in almost all the review criteria (conceptual and measurement model, reliability, validity, interpretability, burden as measured by time required to complete the questionnaire, administration format and number of linguistic validations/translations).3
While head-to-head comparisons are often conducted to investigate the comparative psychometric performance of questionnaires, the patient's perspective of comparable content validity has not been explored. This study aimed to capture the views of people affected by asthma of three commonly used asthma-specific QoL questionnaires; the Sydney Asthma Quality of Life Questionnaire (AQLQ-S), the Living with Asthma Questionnaire (LWAQ) and the Juniper Asthma Quality of Life Questionnaire (AQLQ-J), to understand how they perceived the relevance of the questionnaires in relationship to their own experience of living with asthma.
Methods
Design and recruitment
This qualitative study was conducted between August 2011 and November 2012. Patients were purposively sampled if they had a physician-diagnosis of asthma, good spoken and written English and no severe mental health difficulties. Particular efforts were made to sample men with asthma because in the first cycle of interviews (n=8) few men (n=2) had been included. Posters describing the study in lay language and how to contact the research team were displayed in the outpatient departments of two local general hospitals and in the waiting rooms of general practices in Brighton and Hove, East Sussex, UK. A relationship was present with some of the participants, but otherwise relationships were not established prior to interviews.
Participants established contact via telephone and/or email. Interviews took place in private at the medical school, hospital or participant's home. Besides the participants and researchers, nobody else was present at the interviews. Written informed consent was sought from each study participant.
Recruitment continued until enough data were obtained to formulate meaningful comparisons about the three questionnaires and theoretical saturation was reached, that is, when no new themes emerged in three consecutive interviews.
Questionnaires
The choice of questionnaires used for this study was based on the structured literature review mentioned in the introduction. This review was informed by discussions with experts in the field who identified six QoL measures frequently used in asthma. From these, we chose the three that were specifically developed and validated for patients with asthma (and not for instance, for people with chronic respiratory illness in general or people with asthma and rhinitis).
The three questionnaires used were:
Sydney Asthma Quality of Life Questionnaire (AQLQ-S)
A 20-item self-administered questionnaire with a five-point Likert scale.4 Responses are based on experiences in the preceding 4 weeks (4-week recall period). The AQLQ-S has four subscales: breathlessness, mood disturbance, social disruption and concerns for health.
Living with Asthma Questionnaire (LWAQ)
This comprises of 68 items in 11 domains (social/leisure, sport, holidays, sleep, work and other activities, colds, mobility, effects on others, medication usage, sex, dysphoric states and attitudes) and has no specified recall period.5 It uses a three-point Likert scale. A mixture of positive and negative items compensates for acquiescence bias. Unlike the other two questionnaires, LWAQ has a ‘not applicable’ response option.
Juniper Asthma Quality of Life Questionnaire (AQLQ-J)
AQLQ-J's idiographic component allows patients to choose from a list of five activities important to them.6 It has 32 items in four domains (activity limitation, symptoms, emotional functioning and exposure to environmental stimuli) and uses a seven-point Likert scale and a 2-week recall period.
Data collection
At the beginning of each encounter with participants, the reasons for conducting this research were explained. First, participants completed three different, commonly used asthma-specific QoL questionnaires. In-depth interviews then explored individuals' subjective narratives of how the content of the questionnaires related to their experience of living with asthma. Using a topic guide, derived from the literature3 and discussions within the research team (box 1), the two interviewers (CA and CJ) encouraged participants to talk freely about bothersome aspects of their asthma and whether the questionnaire items covered these aspects. CA and CJ hold PhDs and had prior training in qualitative research methodology. Basic demographics (age, gender, duration of asthma, years in full time education) were noted. Repeat interviews were not carried out.
Topics covered in the interview topic guide
General impression of the questionnaires: What were your feelings/thoughts when completing the questionnaires?
Length of the questionnaires:
How did you feel about the length of the questionnaires?
Layout/visual clarity: How did you feel about the looks/the layout of the questionnaires?
Format of the questions/response options: How did you feel about the options that were given to choose from when responding to each question?
Comprehensibility: How understandable were the questions to you?
Burden: How strenuous was it for you to answer the questions?
Redundancy of questions: Are there any questions which are repeated or very similar?
Need for specification of question wordings: Are there any questions which should be phrased in a more specific manner?
Adequacy/validity of questionnaires in relation to situation of living with asthma:
How much did you feel the questions in the questionnaires covered the issues you are concerned with because of your asthma?
Do important aspects of living with asthma lack in the questionnaires?
Suggestions for improvement: Do you have any suggestions to improve the questionnaires?
Preference: Did you like one of the questionnaires better than the other/others? Could you tell a preference? If questionnaires should be judged as insufficient: which questionnaire is still the most adequate?
Preference: Would you recommend one of the questionnaires?
We are very interested in all of your views and impressions, so in this last section please feel free to add any other comments you feel are relevant to the ways asthma influences quality of life.
Interviews lasted between 20 and 90 min. They were audio-recorded and transcribed verbatim. Field notes were additionally taken. Transcripts were not returned to participants for comment and/or correction.
Data analysis
Thematic content analysis was performed by coding the verbatim transcripts and then grouping the codes into thematic categories.7 Data were coded by CA using ATLAS.ti.8 ,9 The emerging themes were discussed regularly within the research team and credibility of the findings was established by seeking agreement among coresearchers.10
Here, we report on those themes that related to the content validity from the respondents' perspective.
Results
Thirty individuals with asthma participated, of which 12 were men. Their age ranged from 20 to 68, with a median age of 39. Eleven participants reported onset of asthma in infancy (0–2 years), 14 reported onset in childhood (2–14 years) and 5 participants reported adult onset of their asthma.
General perceptions
Generally the AQLQ-S was reported as quick and easy to complete, with unambiguous questions. Its focus on the social, attitudinal and emotional, rather than the medical, aspects of asthma was noted. Participants valued its broad questions relating to everyday life. One participant suggested one needed to be very emotive to relate to the AQLQ-S and that it may only be relevant to people with extreme asthma. Feedback was not consistent. Some interviewees mentioned that the brevity of the questionnaire resulted in it being ‘light’, lacking breadth of coverage.
The AQLQ-J was considered a ‘medical’ questionnaire. Its foci on environmental triggers and activity restriction were perceived as too narrow by some, but pertinent to others. Choosing five relevant activities was viewed positively, individualising the questionnaire and making people think about areas of importance to themselves, but some participants found it difficult to choose five specific activities from the list provided. Some concern was expressed that people would not necessarily choose the activities truly important to them in order to create a good impression with the clinician or researcher.
There was a strongly held view that the LWAQ was a ‘holistic’ and ‘non-medical’ questionnaire with a focus on social and emotional aspects. Some participants said that the content coverage of the questionnaire was wide ranging and that it was a thorough, in-depth questionnaire. On the other hand, the LWAQ was described as irritating because it was felt to generate problems with its emotional focus and its intrusive questioning.
When comparing the different questionnaires, interviewees described the AQLQ-S as being located between the AQLQ-J and LWAQ because it was partly a ‘social’ and partly a ‘medical’ questionnaire. They emphasised that the LWAQ dealt least well with symptomatology and that it was the questionnaire which was most reflective of the impact of asthma on QoL. The AQLQ-S was considered to be concise and ‘short and sweet’. In comparison with the AQLQ-S and the LWAQ, being able to choose activity items was seen as an advantage of the AQLQ-J.
Missing content
Participants identified a number of areas that they considered missing from the questionnaires, such as the need to seek healthcare (medication, consultation or admission) for their asthma (table 1). Participants commented on a lack of content relating to asthma control (such as peak flow measurement), and noted their responses to the items would vary reflecting how well their asthma was controlled.
Well I have to say, like I said, the one thing that is missing, is the asthma management of an individual, because that will have a bearing on quality of life—if it is very badly managed. (P15)
Patients' perspective on content missing in the questionnaires
Participants also lamented that allergies and asthma triggers were not reflected in the questionnaires' content, thereby almost missing the very nature of asthma as a fluctuating disease.(…) it was almost like it is for someone who has got asthma just all the time, you know … but my asthma mainly occurs when I come in contact with animals or with tree pollen or the things that set me off, you see what I mean? (P11)
Redundant or similar content
All three questionnaires attracted comment about redundant content. For example the AQLQ-S asks about frustration (item 9), anger (item 10) and worry (item 19), which were perceived as being so similar as to be repetitive. Examples from all three questionnaires are shown in table 2.
Patients’ perspective on redundant questionnaire content
The use of positive and negative items in the LWAQ was also contextualised in a discourse of redundancy (eg, item 3 ‘Having asthma restricts the sort of holiday I can take’ and item 25 ‘I can go on the same kind of holiday as everybody else’).
Irrelevant content (table 3)
No irrelevant content was identified in the AQLQ-S. In the AQLQ-J and LWAQ, items relating to cigarette smoke (AQLQ-J item 9: experiencing asthma symptoms as a result of being exposed to cigarette smoke, item 11: feeling having had to avoid a situation or environment because of cigarette smoke) were considered irrelevant and outdated because the UK has had a smoking ban in all enclosed public space and work places since 2007.
Patients’ perspective on irrelevant questionnaire content
In the AQLQ-J, having to avoid situations because of exposure to, for instance perfume, was not thought to be pertinent. The items relating to having sexual intercourse (an activity option), air pollution, fighting for air, experiencing a feeling of chest heaviness as well as feeling bothered by heavy breathing were also highlighted as irrelevant.Umm, I feel bothered by breathing difficulties, you know. Or constricted breathing or something like that, you know, rather than heavy breathing. So yes, that was that. (P19)
The list of activity items from which to choose five items in the AQLQ-J was perceived as not offering the right choice.Yes, but they are not the choices that I would have (…) they are not the things that I would have put down as choices. So I wasn't really able to choose the right things, if that makes sense. (P22)
With respect to the LWAQ, carrying shopping, colds, taking tablets for asthma, the questions on holidays, walking up stairs and getting depressed about asthma were mentioned as examples of irrelevance. These views were frequently expressed but without people being able to offer alternative suggestions.
Confusing and challenging content
In the AQLQ-S, the items ‘I have felt that asthma has prevented me from achieving what I want in life’ (item 11), ‘I have felt asthma is controlling my life’ (item 17) and ‘I have been restricted in walking up hills and doing heavy housework because of my asthma’ (item 5) were reported to be confusing or meaningless. One participant (P20) mentioned that frequent use of the word ‘troubled’ in the AQLQ-S was confusing because it could relate to either emotional or physical problems.
In the AQLQ-J, it was felt that item 23, ‘experiencing asthma symptoms as a result of the weather or air pollution outside’, was a difficult item as most people are unaware of levels of air pollution.
The final two AQLQ-J questions, one asking whether one was limited in the overall range of activities that one would have liked to have done (item 31) the other asking whether one was limited in all the activities that one has done (item 32) were felt to be confusing. Asking these questions when questions about activities had already been asked at the very beginning fuelled the confusion. The item asking whether one felt afraid of not having one's asthma medication available (item 21) was problematic because it was unclear what type of medication was being referred to, be it reliever or preventer medication. The item feeling afraid of getting out of breath (item 27) was perceived as difficult to understand. Experiencing a wheeze in one's chest (item 10) was perceived as confusing because not everyone with asthma experiences wheeze.
With respect to the LWAQ, feeling angry with one's body (item 9) lacks clarity of meaning. ‘Getting emotionally upset when puffy’ (item 32) was described as ‘not English’ terminology and of limited meaningfulness. Never feeling fed up because one has asthma (item 20) was considered challenging because the precise reason for feeling fed up lacked definition (eg, someone wondered whether it related to activity limitation or medication regime). Patients were unsure about the item on taking good care to avoid doing things which make one's asthma worse (item 5) as well as the item on having a good future ahead of oneself (item 50). Being able to walk up a flight of stairs without stopping (item 42) was perceived as problematic because ability depended on the characteristics of the stairs.Well I suppose when they ask you (…) if you can walk up a flight of stairs. And then you realise that you did have a situation where you couldn't walk up some stairs, but the reason was because they were extremely long and extremely high. (P19)
Irritating content
Content could sometimes cause respondents to feel patronised, for instance questions on depression (AQLQ-S and LWAQ). The item asking whether one felt angry with one's body (item 9) was reported to have an irritating effect. The item on sexual frustration in the LWAQ (item 56) was described as irritating and invasive.In the ‘Living with Asthma’ questionnaire is a really difficult one to … to … it is quite a … one could feel angry with one's body for many different reasons and again, I felt that was too limiting and wasn't quite the right … I was a bit put off by that. I found it a bit, you know, presumptuous. (P22)
Discussion
Statement of principal findings
It was the aim of this study to explore how people affected by asthma perceive the relevance of three commonly used asthma-specific QoL questionnaires (the AQLQ-S, the LWAQ and the AQLQ-J) in relation to their own experience of living with asthma. Participants expressed a wide variety of views about the content of the three questionnaires. The emotional focus of the LWAQ was perceived as irritating by some participants. Completion was described as burdensome and likened to a ‘test’ or ‘quiz’. A recurrent theme was that the LWAQ was a wide-ranging and holistic questionnaire. In contrast, the AQLQ-J was perceived as a ‘narrow’ ‘medical’ questionnaire with a focus on symptoms, environment and functional ability. The selection of relevant activities was perceived to be positive by some and difficult by others. The AQLQ-S was described as a simple, quick and easy questionnaire, but there was also a perception that it simultaneously lacked depth. Overall, the AQLQ-S was felt to be located ‘between’ the AQLQ-J and the LWAQ on the ‘medical’–‘emotional’ spectrum.
Strengths and weaknesses of this research
This novel study elicits patients' views on the content validity of different questionnaires that purport to measure the same construct (HrQoL) for asthma. We interviewed adults with a wide age range and range of disease duration. The interviewers were non-clinicians which may have facilitated an open discussion that might have been impeded if the interviewers had been clinicians. The research team included a male health services researcher and a female psychologist holding PhDs as well as a female academic general practitioner and a male respiratory specialist holding MDs. A further strength of this study is that it built on a previous small-scale pilot study conducted in Germany.11 Interestingly, all the themes that had emerged in the previous study were also found in this study, but our data were much richer and the thematic framework was expanded to include the new themes of ‘confusing/difficult content’ and ‘irritating content’.
Interviews were our chosen method because they are more suitable for sensitive issues and, unlike focus groups, can be arranged for the convenience of each participant. However, group discussion may have further enriched the information generated.12–14 We had expected a certain degree of respondent fatigue, as completing three questionnaires may be perceived burdensome. However, in reality, participants appreciated completing the questionnaires and being able to talk at length about their experience. Recall bias (ie, bias introduced by participants focusing on the last completed questionnaire) was dealt with by actively encouraging respondents to provide feedback on all questionnaires. The sample has a female preponderance and, with one exception, is of white British origin. Although common themes were identified in this study and the preceding German study, caution must be exercised when generalising findings. Generalisability, as conceptualised in empirical quantitative research, is not usually sought in qualitative research which seeks theoretical generalisability.10 Participant checking (participants providing feedback on the findings) was not undertaken in this study.
Similarity to other published work
Two previous studies have addressed the respondents' perspectives on self-reported health status questionnaires. The first study explored how older people with chronic health problems interpreted questions in the most widely used health status questionnaire, the Short-Form (SF)–36.15 Participants found some questions vague, for example, ‘How about lifting or carrying groceries?’ was unclear as there was no detail about the weight of the bag. Such findings were reflected by the participants in our study who found lack of specificity confusing and challenging.
A similar study assessed the validity of the Oxford hip score (OHS), a joint-specific measure to assess patients' disability following total hip replacement.16 There is resonance in several areas between the OHS study findings and our study. Using the OHS, patients were unsure whether they should report their actual disabilities or their level of disability using aids (ie, actual or relative disability). Similarly, patients with asthma were unclear whether they were being asked about impairments pre or post control. Patients with hip replacement found it difficult to report an average level of pain as their pain was dynamic rather than static. Patients with asthma also spoke frequently about its dynamic, fluctuating nature. OHS study participants reported difficulties separating out the impact of their hip problems from other significant comorbidities. In the context of asthma, allergies, coughs and infections were mentioned as comorbidities influencing responses, but lacking in the questionnaires. Difficulties with activities not being important to all individuals and activities with changing importance over time have been noted with many validated patient-centred outcome measures.16 This highlights the tension between the subjectivity of the PROMs and their apparent claim to be objective measures.
Implications of the findings
Improving content validity
All three questionnaires involved patients in the identification of important issues in their early development, but as the development process progresses, the need for robust objective measurement over-rides attentiveness to the subjective lay perspective. Inevitably, items are lost to achieve a practical questionnaire with an internally consistent dimensional structure from a large item pool. Our work highlights how the existing and conventional process can result in patient concerns about missing items as well as items considered as difficult or confusing. This suggests that it may be advantageous to check content validity after consideration of the metrics of the item set.
While patient involvement highlights shortcomings in the content validity of existing questionnaires, the patient's view needs to be balanced with a scientific perspective. The suggestion from respondents that asking about depression or sadness was irrelevant fails to recognise that for others, these emotions can affect asthma control and QoL.17 This example highlights that some tension between the lay perspective and the professional perspective is inevitable, but this should not stop us from striving to minimise non-congruence.
Choice of questionnaire
Based on our findings, the AQLQ-S seems to be the most pertinent questionnaire for people with asthma. However, there was a diversity of views expressed and some participants also liked the focus on activities or the psychosocial domain in the AQLQ-J or LWAQ, respectively. Future research needs to explore patient responses in other cultural or linguistic contexts.
Acknowledgments
We are grateful to all patients who volunteered to participate in this study and Elizabeth Godfrey, the professional transcriber who transcribed the interviews from audio records.
References
Footnotes
Contributors CA conceived and designed the study, conducted interviews, analysed and interpreted the data, wrote the manuscript and reviewed it for important intellectual content. CJ conducted interviews, interpreted the data, helped with writing the manuscript and reviewed it for important intellectual content. AF helped with patient recruitment and writing the manuscript and also reviewed it for important intellectual content. HS supervised the concept and design of the study, interpreted the data, wrote the manuscript and reviewed it for important intellectual content. All authors have read and approved the final version of manuscript.
Funding The work was supported by a PhD studentship from the University of Brighton, UK.
Competing interests None declared.
Ethics approval Brighton East Research Ethics Committee, reference number: 10/H1107/38.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The data set is available from the authors on request.