Article Text
PDF
Abstract
Objectives Patient-identified need is key to delivering holistic, supportive, person-centred care, but we lack tools enabling patients to express what they need to manage life with a long-term condition. The Support Needs Approach for Patients (SNAP) tool was developed to enable patients with advanced chronic obstructive pulmonary disease (COPD) identify and express their unmet support needs to healthcare professionals (HCPs), but its validity is unknown. This study aimed to establish face, content and criterion validity of the SNAP tool.
Design Two-stage mixed-methods study involving patients with advanced COPD and their carers. Stage 1: Face and content validity assessed though focus groups involving patients and carers considering appropriateness, relevance and completeness of the SNAP tool. Data were analysed using conventional content analysis. Stage 2: Content and criterion validity assessed in a postal survey through patient self-completion of the SNAP tool and disease impact measures (Chronic Respiratory Questionnaire, COPD Assessment Test, and Hospital Anxiety and Depression Scale). Content validity assessed using summary statistics; criterion validity via correlations between tool items and impact measures.
Settings and participants Two hundred and forty patients and carers participated. Stage 1 patient and informal carer participants were recruited from two primary care practices and Stage 2 patients from 28 practices. Participating practices located in the East of England were recruited via the NIHR Clinical Research Network: Eastern.
Results Patients and carers found the tool patient-friendly and comprehensive, with potential clinical utility. No tool items were redundant. Clear correlations were found between tool items and the majority of items in the impact measures.
Conclusions The SNAP tool has good face, content and criterion validity. It has potential to support the delivery of holistic, supportive, person-centred care by enabling patients to identify and express their unmet support needs to HCPs.
- respiratory medicine (see thoracic medicine)
- chronic airways disease
- qualitative research
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Recruitment from primary care ensured that study participants with advanced chronic obstructive pulmonary disease were not restricted to those attending specialist services.
Patient participants were central to each stage of the validation process ensuring the results reflected a patient perspective.
A potential limitation of this study was that data came only from patients from the East of England.
A further limitation may be that only two focus groups were conducted to assess face and initial content validity, but reflected anticipated recruitment targets.
Background
The need to deliver holistic, supportive, person-centred care to patients with long-term conditions is widely recommended1–4 and increasingly recognised, for example, the Twitter movement “#WhatMattersToYou”. Understanding the patient’s view on their needs and preferences is seen as key to delivering this approach. Implementation guidelines have highlighted the potential for using assessment tools to support patients with long-term conditions identify those areas with which they need more support.5 6
A range of tools have been recommended for use with patients in clinical practice7–11; however, most focus primarily on measuring disease burden, patient functionality or patient concerns. These tools can be valuable indicators of need but do not directly identify areas where patients need more support to manage life with their condition (their unmet support needs). Where tools include patient support needs, for example, the Sheffield Profile for Assessment and Referral for Care9, they focus only on a narrow range of support domains or prescribed responses to need (eg, information), limiting the patient’s consideration of wider support needs. Furthermore, other tools such as the PEPSI COLA Aide Memoire7 are practitioner-led, contrasting with patient-completed tools that actively support a person-centred approach through ensuring that areas of support need identified and discussed are those prioritised by the patient. In addition, many of the tools contain long lists of questions making them too lengthy to be feasible in busy generalist settings where most people with long-term conditions are seen.
Thus, there is a lack of suitable tools that support patients with long-term conditions to directly identify and express what they need to help them manage their condition. The Support Needs Approach for Patients (SNAP) tool was developed to fill this gap. It was developed for patients with chronic obstructive pulmonary disease (COPD), a progressive long-term condition in which there are known barriers to patient articulation of need including beliefs held by many patients that they are undeserving of support12 13
The development of the evidence-based SNAP tool is described in detail elsewhere.14 In brief, this was a two-stage process: (1) establishing a typology of evidence-based patient support needs and (2) formulation of items and format, review and refinement (summarised in figure 1). Unlike many tools designed for use in a healthcare context, the development of the SNAP tool was not informed by psychometric theory: the aim was not to develop a scale of single items in order measure individual patient need in relation to a theoretical construct of support need. Instead, the methodology used in the development process was chosen to support the creation of broad tool items, informed by the identified domains of support need, with the aim of enabling patients to reflect on and identify their particular support needs within those domains.
Two stages of SNAP tool development.(PPI, patient and public involvement)
The developed tool consists of 15 items based on the broad domains of support need identified within the typology of evidence-based patient support needs. The first 14 items use the following format “Do you need more support with…” for example, “Do you need more support with overcoming boredom and loneliness?” The 15th item enables patients to identify any concerns they may have about those who support them. An additional item is also included to capture “anything else” that the patient feels does not fit under the formulated items. Box 1 provides a list of all tool items.
SNAP tool support domains
Do you need more support with…
…understanding your illness
…managing your symptoms (including medication and oxygen)
…dealing with your feelings and worries
…looking after any other physical health problem you may have
…having a healthier lifestyle (eg, keeping active or eating well)
…getting out and about
…overcoming boredom or loneliness
…financial, legal, work or housing issues
…practical help in the home or garden
…your personal care (eg, dressing, washing)
…aids or equipment to help you
…family relationships (including talking to your relatives about your illness)
…knowing what to expect in the future
…accessing or using services
…anything else
Does a family member or friend who helps you need more support?
The SNAP tool items are set in a grid layout with three response categories (No/A little more/Quite a bit more) to encourage any expression of need. There are brief, clear instructions for completion. Patent engagement is crucial therefore the SNAP tool is titled with the question “How are you?” to invite completion. The tool is currently available in two versions: (1) a coloured (or black and white) version designed by an NHS Trust media studio in an A5 booklet format recommended for use in clinical practice and (2) an alternative black and white version on one side of A4 which may suit the needs of researchers seeking to identify patients’ unmet support needs (ie, not for clinical practice use). An inspection copy of the SNAP tool is available online (https://thesnap.org.uk/).
The SNAP tool was developed to be patient friendly, cover the range of support need domains experienced by patients with COPD and enable patients to identify issues relevant to their condition. In order to facilitate uptake of the SNAP tool in clinical practice, there is a need to evidence these properties.15 16 As the SNAP tool is not a psychometric tool, the use of reliability and validity tests designed to specifically assess the characteristics of items derived within a psychometric framework would not be appropriate (eg, factor analysis). However, the pragmatic application of other aspects of validity testing would demonstrate suitability for purpose and enhance clinical confidence in the tool. More specifically, establishment of face validity could demonstrate acceptability to patients, content validity could establish that tool items reflect patient support needs, and criterion validity could provide evidence of a relationship between items on the SNAP tool and established clinical indicators of patient need.17 The aim of this study was therefore to assess the face, content and criterion validity of the SNAP tool.
Methods
The design was a sequential mixed-methods study, comprising two stages:
Stage 1. Assessment of face and initial content validity via two focus groups with patients living with advanced COPD, and their informal carers.
Stage 2. Assessment of content and criterion validity via a patient-completed postal survey administering the SNAP tool alongside standard measures of disease impact to patients with advanced COPD.
Stage 1: assessment of face and initial content validity
Focus groups were used to obtain patient and informal carer feedback on face and content validity of the SNAP tool.
Recruitment of focus group participants
Two primary care practices, recruited via the NIHR Clinical Research Network: Eastern, invited patients to participate in focus groups if they met two or more of six well-established clinician-developed inclusion criteria for patients with advanced COPD (and none of the exclusion criteria), shown in table 1. Eligible patients were sent a recruitment pack by their practice comprising a letter of invitation, participant information sheet, reply slip and freepost envelope for direct return to the study team. Patients were also invited to bring along a friend or family member who supported them (their informal carer). On receipt of a reply form from patients wishing to take part, the study researcher (ACG) made contact to answer any questions they had about the study and to make arrangements for the focus groups.
Study inclusion and exclusion criteria
Data collection
Two focus groups were held in local hotels chosen for ease of access and comfort, and refreshments provided. Participants completed a consent form and a brief questionnaire to capture background details prior to the group discussion. Each group was facilitated by two members of the study team (ACG and MF), lasted approximately 90 minutes and was audio-recorded with participant permission.
In the first part of the focus group, participants were shown the black and white version of the professionally designed A5 booklet format and asked to consider the range of items, clarity of the instructions, ease of completion and suitability for purpose. In the second part, they were asked to review the coloured version of the A5 booklet format in two different colour schemes.
Data analysis
Audio-recordings were transcribed by a professional transcription company, checked for accuracy and anonymised by the study researcher (ACG). Data were analysed using conventional content analysis and identified codes and categories were discussed and refined by the study team (ACG, GE, MF).18 19
Stage 2: assessment of content and criterion validity
A postal survey was used to assess the content and criterion validity of the SNAP tool.
Survey recruitment
A further 28 primary care practices, recruited via the NIHR Clinical Research Network: Eastern, invited patients to participate in the postal survey using the same inclusion and exclusion criteria shown in table 1, until a minimum sample size of 200 patients was achieved. The justification for this sample size was that it had sufficient power (>80%) to detect a Spearman’s correlation of 0.3 between SNAP tool items and the disease impact measures.
Study recruitment packs were sent by each practice to eligible patients and comprised a letter of invitation, participant information sheet, postal survey booklet (including consent form) and freepost envelope for direct return to the study team. A follow-up procedure was employed of a phone call from the practice to non-responding participants within two weeks of their recruitment pack being sent. This enabled practice staff to establish if the patient had received and understood the pack, and whether they would like to be sent a new pack, so giving every eligible patient the opportunity to participate if they wished. This was mentioned in the letters of invitation to potential participants.
Data collection
The survey booklet contained the original A4 black and white version of the SNAP tool, basic demographic questions to assess sample representativeness and three standard measures of disease impact: the Chronic Respiratory Questionnaire (CRQ),20 the COPD Assessment Test (CAT)21 and the Hospital Depression and Anxiety Scale (HADS).22 The CRQ measures quality of life in chronic lung disease: the 20-question self-report version (CRQ-SR) covers dyspnoea, fatigue, emotional functioning and mastery which form two subscales for physical and emotional functioning (CRQ–Emotional and CRQ–Physical). The CAT (eight questions) assesses COPD impact, for example, shortness of breath and ease of living at home. The HADS (14 questions) consist of two subscales to screen for anxiety (HADS-A) and depression (HADS-D). These standard measures are commonly used in clinical practice to assess patients with advanced COPD and are thus considered as clinical indicators of patient need, providing appropriate comparators to the SNAP tool. Data were collected between July and December 2017.
Data management
Completed survey booklets were anonymised on receipt by removal of consent forms. Data from returned postal surveys were entered into SPSS V.24. Double data entry was performed on a 10% sample (ACG and MF) and showed a high level of data entry accuracy. Missing data for the HADS were replaced by the mean value of the patient’s responses within the respective subscale, provided that the patient had answered at least four of the questions in that subscale.23 Missing data for the CRQ was handled in line with the CRQ scoring system outlined in the accompanying manual.24 Where data were missing in the CAT, the final CAT score for the patient was not calculated. Missing data on the SNAP tool were left as missing.
Statistical analysis
Content validity was assessed by the percentage of patients indicating a need for more support with each tool item, to identify any redundant items. Where patients had commented in the “Anything else—please write in” section, these comments were reviewed to assess whether they could be allocated to existing tool items. To do this, each member of the study team (ACG, GE, MF) independently mapped the ‘anything else’ comments to existing tool items; this mapping was then compared and differences of opinion resolved through team discussion until consensus was achieved.
Criterion validity was tested by investigating the relationship between the SNAP tool items and each of the standard measures of disease impact. We hypothesised that greater need for support (indicated by a greater need for support on a SNAP domain and represented numerically purely for the purpose of this test) would correlate with lower health-related quality of life, lower levels of patient functioning, and higher indicators of anxiety and depression. As the items on the standard measures of disease impact act as indicators of patient need, but not patient perceptions of carer need, the final item on the SNAP tool “Does a family member or friend who helps you need more support?” was omitted from this stage of validity testing. The nature of the tool items indicated the need for non-parametric testing; Spearman’s rank (rs) correlation was therefore used. Correlations of 0.10 were considered weak, 0.3 moderate, 0.5 strong and 0.7 very strong.25 Missing values were handled by pairwise deletion in the correlation table.
Patient and public involvement
This validation study is part of an ongoing programme of research that is supported by a patient and public involvement (PPI) advisory group. In this study, PPI members (1) provided input into reviewing participant recruitment documents for appropriateness and clarity prior to submitting an application for ethics approval and (2) reviewed, and endorsed, findings from the face and content analysis.
Results
Study sample
Focus group
In total, 27 eligible patients were invited to participate; eight patients and four carers agreed to take part. Seven participants were women (58.3%) and they ranged in age from 51 to 90 years. Participating carers were two spouses, a daughter and a friend. The level of recruitment reflected protocol targets (ie, two focus groups comprising 12–16 patients and carers).
Postal Survey
Recruiting practices identified 503 eligible patients who were invited to take part. Of these, 167 (33%) responded to the first mail out and a further 61 (12%) responded to the re-mail resulting from the follow-up phone call from the practice; in total, 228 patients participated (45%). There were 126 male participants (58%) and participants’ ages ranged from 49 to 94 years.
Face and content validity
Focus groups
The patient and carer focus groups were unanimous in identifying the SNAP tool as easy to understand, straightforward to complete and patient-friendly. For example, one participant commented that the SNAP tool was “all quite clear and straightforward” (FG1-S4).
The focus group participants felt that the range of items on the tool was both comprehensive and relevant for people with advanced COPD. When considering the list of items, one carer commented on how it reflected particular concerns she had:
" I find there’s quite a few things on here that we don’t know… about what’s going to happen in the future and, you know, what equipment we might need and, you know, just in general. So, we would like to know more about that, but they don’t seem to tell you at the doctors " (FG2-S5)
None of the focus group participants identified additional items they felt were missing from the SNAP tool.
Feedback was also provided on the wording of the questions, endorsing in particular the use of plain English and non-medical language. One patient noted positively: “It’s in layman’s terms as opposed to medical terms” (FG1-S4). Another participant described how use of the term ‘feelings and worries’ within one of the items was far less threatening than standard medical terminology. Participants also commented favourably on the tool’s format: “I think the layout is particularly good…. It’s the clarity of it… it’s not threatening in any way” (FG1-S6). They appreciated, in particular, the concise nature of the tool, the clarity of the instructions and overall ease of completion. They also liked the title ‘How Are You?’
There was a further positive response to the use of colour, and the potential for clinical service providers to customise the tool’s front and back covers (eg, by adding service logos, notes sections etc). It was felt that both the use of colour and the adaptive properties of the cover would enhance patient and healthcare professional engagement with the tool. Only one participant felt that the current design would not attract their attention.
Participants also highlighted a number of ways in which patients might find the SNAP tool useful:
" at a glance you can see where the problem is and then you can think about it rather than asking questions to find out " (FG-S2)
They also discussed the experience of arriving at a consultation with questions but then forgetting to raise these issues and felt that the completed tool might act as a reminder with one participant suggesting that “if you write it down at least someone will look at it” (FG1-S2).
There were, however, concerns about whether healthcare professionals had the time, resources or interest to discuss and address issues that might be raised by the tool. A couple of participants discussed how the level and nature of need changes over time and suggested the SNAP tool was potentially useful at different stages of the disease trajectory.
" In the early stages … they might say, ‘ Oh you’ve got this ’ … it [the SNAP tool] might make them think … ‘ Actually, yeah, I do want some more help ’ , but [they] might not want to say it " (FG2-S4)
Postal survey
Data from the postal survey established that the tool items reflect the domains of support needs of patients with advanced COPD. Figure 2 shows the 15 SNAP tool items ordered according to the percentage of patients who expressed a need for more support with each (ie, any indication of need for more support other than ‘No’). Over 50% of patients wanted more support with ‘knowing what to expect in the future’ and ‘looking after any other physical health problems’. In addition, over 40% identified a need for support with ‘practical help in the home or garden’, ‘getting out and about’, ‘dealing with your feelings and worries’, ‘accessing or using services’ and ‘having a healthier lifestyle’. Only one item was identified by less than 20% of patients (‘family relationships’), but this was nevertheless relevant to 39 people who took part. In total, 3% identified other needs in the ‘anything else’ section, but all could be consensus-mapped to existing tool items, for example, ‘Access to a GP that does not judge’ could be mapped to the item ‘accessing and using services’. Across the sample, patients responded to all tool items indicating that none was redundant for this population.
Percentage of patients with advanced chronic obstructive pulmonary disease expressing need for more support with each SNAP tool item. Sample size ranged from 218 to 225 patients across items except for “Does a family member or friend who helps you need support?” (n=157).
Content and criterion validity
Clear correlations were found between tool items and the majority of items on the impact measures. We will focus on correlations showing statistical significance with p value ≤0.05. All SNAP tool items were positively correlated with both HADS-D and HADS-A scores. That is, the greater the unmet need for support, the higher the likelihood of depression or anxiety. All the SNAP tool items were negatively correlated with the CRQ–Emotional subscale and all but one item were negatively correlated with the CRQ–Physical subscale. That is, the greater the unmet need for support, the lower health-related quality of life reported. All but two of the SNAP tool items were positively correlated with the CAT scores: that is, the greater the unmet need for support, the greater the impact of COPD as reported.
Stronger associations between items on the SNAP tool and the measures of disease impact are highlighted on table 2: the strength of associations ranged between rs=0.15 and rs=0.44 for items that were positively correlated and between rs=−0.13 and rs=−0.48 for items that were negatively correlated (see table 2). While the overall picture is mixed, moderate correlations were primarily identified between psychological indicators (CRQ–Emotional, HADS-D) and SNAP tool items, whereas there were weaker correlations with indicators assessing physical aspects of the disease (CRQ–Physical and CAT). With the exception of three items, each of the SNAP tool items was moderately correlated with at least one of the standard measures of disease impact. The exceptions were ‘aids and equipment to help you’, ‘financial, legal, work and housing issues’ and ‘family relationships’.
Correlations between SNAP tool items and standard measures of disease burden in COPD
Discussion
This paper reports the validation of the SNAP tool in patients with advanced COPD. The findings suggest that the SNAP tool has face, content and criterion validity for patients within this population.
Face validity has been identified as particularly important for tools as it enables acceptability of the tool to potential users.26 Qualitative feedback on the format and acceptability of the tool came directly from patients with advanced COPD and their carers recruited from primary care.
Content validity was demonstrated via two sources: qualitatively through focus groups which established patient and carer acceptability of the range of items and quantitatively through the postal survey findings. Cross-sample identification of need for more support through using the tool showed that none of the items was redundant and that the existing items comprehensively reflected patient support needs.
If the SNAP tool is identifying something meaningful and relevant within the context of advanced COPD, it should also be significantly related to standard measures of disease impact in advanced COPD.17 Criterion validity was used to test this and identified significant correlations in the right direction between the tool items and the standard measures, with low to moderate strengths of association. If the SNAP tool was a psychometric measure, stronger associations would be more desirable: traditional psychometric testing of criterion validity seeks to assess how far a measurement tool can be used to predict outcomes on a ‘gold standard’, with stronger levels of association indicating higher levels of predictive ability, suggesting in turn that the two tools are performing the same function.26 However, as the SNAP tool is intended to facilitate a conversation, rather than a ‘measure’ in the psychometric sense, our approach was fundamentally different. We were not looking to assess the predictive ability of the SNAP tool against a gold standard; no such gold standard of need exists and the SNAP tool is not a measurement instrument. Instead, our aim was to look for evidence of an underlying relationship between the tool and the standard measures, in order to give clinicians confidence in the tool. We would argue that the strengths of association found suggest the SNAP tool is related to, but different from, the standard measures. These results support the view that as a tool that has been specifically designed to directly identify patients’ unmet support needs, the SNAP tool is distinct from the standard measures of disease impact currently used in the assessment of patient needs.
The SNAP tool was not subject to construct validity or reliability (internal consistency) testing because the tool was not developed to define, or measure, any internal psychometric construct.27 Rather, the tool consists of a set of questions intended to help patients identify and express individual unmet support needs within a range of broad, evidence-based domains of support need relevant to their situation. The domains taken together are comprehensive, in terms of the range of areas they address, yet they remain ‘standalone’ individual areas of support need rather than contributing to an overarching construct of support need. Assessing reliability or construct validity is therefore neither feasible nor warranted. Reliability testing establishes the stability of scores over time, but when completing the SNAP tool, a patient may highlight the same domain of need at two different time points but for different reasons at each time point, for example, should a patient with COPD tick that they feel they need more support with ‘managing symptoms’, their precise need within that broad domain at timepoint 1 might relate to breathlessness, whereas at timepoint 2 it might relate to fatigue.
A key strength of this study is the central role of patients. Patients with advanced COPD were involved in each stage of the study, and recruitment via primary care was undertaken to ensure representation of all patients with advanced COPD (rather than just those referred to specialist or secondary care). PPI members also reviewed the study findings. While it is standard practice to include patients in the assessment of face and criterion validity,17 Osse15 and Richardson16 both noted that not all studies involve patients in validation of tool content. Osse argues that failure to use patient data in this respect can result in tools that give precedence to needs that are seen as important from a professional, rather than patient, perspective. The use of qualitative patient data from focus groups combined with quantitative patient data from completed SNAP tools within the postal survey ensures that each item on the tool has been identified as relevant by patients with advanced COPD.
A potential limitation of this study was that data came only from patients from the East of England. However, the 30 GP practices involved in the study were spread across the region serving a diverse population in terms of low, moderate and high deprivation across both rural and urban areas. In addition, the age and sex of the study population reflects known national patterns for patients with COPD.28
A further potential limitation may be that only two focus groups were undertaken in the assessment of face and initial content validity, involving 12 patients and carers in total. However, this reflects the peer-reviewed recruitment targets in the protocol that were in turn established in response to known difficulties of recruiting patients with advanced disease.29 Furthermore the findings suggest that the number of focus groups was sufficient and the recruitment of further participants would have been unwarranted, and thus unethical.
In conclusion, this study establishes the SNAP tool as valid for identifying the unmet support needs of patients with advanced COPD. As the SNAP tool is not a psychometric tool, further psychometric testing is not appropriate.
As a validated tool the SNAP tool has clear utility as a research tool in studies examining patients’ experience of living with COPD. However, crucially, it also has potential utility in clinical practice to identify and enable person-centred discussion of patients’ unmet support needs. This study has shown the tool’s suitability and acceptability to patients with its relevant content and simple and attractive format. It could therefore provide healthcare professionals with a way of engaging with patients, enabling patients themselves to identify their unmet support needs, thus ensuring more tailored support to manage their condition.
The next stage of the SNAP programme of work will explore clinical practice use in terms of how the tool should be introduced and used within consultations. As the SNAP tool has been developed to underpin the Support Needs Approach for Patients (SNAP), an intervention designed to enable person-centred care in clinical practice, future testing of the tool will be an integral component of exploring the feasibility and effectiveness of the SNAP intervention. For further information about the SNAP intervention (including how to obtain an inspection copy of the SNAP tool), please visit their website (https://thesnap.org.uk/). You can follow SNAP on Twitter: @SNAPstudyteam.
Accessing the SNAP tool: The SNAP tool is a copyrighted tool available free of charge to the NHS and other not-for-profit organisations. A license is required for use. For further details about obtaining an inspection copy and licence please visit their website (https://thesnap.org.uk).
Acknowledgments
The authors thank the patients and carers who participated in this study, PPI and Study Advisory Group members, NIHR CRN: Eastern and the study funder Marie Curie. Use of the Chronic Respiratory Disease Questionnaire Self-Administered Standardized (CRQ-SAS), authored by Drs Gordon Guyatt and Holger Schünemann, was made under license from McMaster University, Hamilton, Canada. Use of the Hospital Anxiety and Depression Scale, authored by R P Snaith and A S Zigmond, was made under license from GL Assessment Ltd (www.gl-assessment.co.uk).
References
Footnotes
Twitter @MoragCFarquhar
Contributors ACG, MF and GE all contributed towards data analysis, drafting and critically revising the paper. SM provided statistical expertise. All authors gave final approval of the version to be published.
Funding This paper presents independent research funded by Marie Curie Research Grants Scheme (MCRGS-07-16-10).
Competing interests None declared.
Patient consent for publication Obtained.
Ethics approval Ethics approval was obtained from the East of England Essex Research Ethics Committee (REC reference 17/EE/1092).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request.