Article Text
PDF
Abstract
Objectives Good social functioning is important for people living with dementia and their families. The Social Functioning in Dementia Scale (SF-DEM) is a valid and reliable instrument measuring social functioning in dementia. However the minimum clinically important difference (MCID) has not yet been derived for SF-DEM. This study aims to define the MCID for the SF-DEM.
Design We used triangulation, incorporating data from a cross-sectional study to calculate the MCID using distribution-based and anchor-based methods, and a Delphi survey.
Setting and participants The cross-sectional survey comprised 299 family carers of people with dementia. Twenty dementia experts (researchers, clinicians, family carers) rated whether changes on clinical vignettes represented a meaningful change in the Delphi survey.
Primary outcome measures We calculated the distribution-based MCID as 0.5 of an SD for each of the three SF-DEM domains (1—spending time with others, 2—communicating with others, 3—sensitivity to others). We used the carers’ rating of social functioning to calculate the anchor-based MCID. For the Delphi survey, we defined consensus as ≥75% agreement. Where there was lack of consensus, experts were asked to complete a further survey round.
Results We found that 0.5 SD of SF-DEM was 1.9 points, 2.2 and 1.4 points in domains 1, 2 and 3, respectively. Using the anchoring analysis, the MCIDs were 1.7 points, 1.7 points, and 0.9 points in domains 1, 2 and 3, respectively. The Delphi method required two rounds. In the second round, a consensus was reached that a 2-point change was considered significant in all three domains, but no consensus was reached on a 1-point change.
Conclusions By triangulating all three methods, the SF-DEM’s MCIDs were 1.9, 2.0 and 1.4 points for domains 1, 2 and 3, respectively. For individuals, these values should be rounded to a 2-point change for each domain.
- dementia
- delirium & cognitive disorders
- epidemiology
- old age psychiatry
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study which aims to derive the minimum clinically important difference (MCID) for the Social Functioning in Dementia Scale; this has useful implications for future research aiming to improve social functioning in dementia.
This study triangulates the MCID from three established methods, which are the distribution-based, anchor-based and Delphi-based methods.
The cross-sectional study comprised a large research sample, which was diverse in terms of gender, ethnicity, background and severity of dementia.
A diverse range of dementia experts were recruited onto the Delphi survey, including family carers, doctors, researchers and therapists.
The study took place in the UK so it is not clear if the findings are applicable to populations outside of the UK.
Introduction
Social functioning is important to human experience including for those with dementia. Decline in social functioning—how individuals interact in society and their own personal environment1—is a diagnostic criterion for dementia2 and social functioning impairments are distressing to people with dementia and their families.3 As well as increasing the risk of dementia onset, loneliness and social isolation have lasting psychosocial effects on those living with dementia and their caregivers.4–7 Conversely, increase in social interaction has been associated with improvement in cognition,8 9 and improved quality of life for individuals with dementia.10
Accurate measurement of social functioning in dementia is essential for research aiming to understand the causes of decline in social functioning in dementia, effects of different lifestyles and to assess whether interventions are effective in maintaining or improving social functioning. There are few instruments designed to assess social functioning in dementia.11 We, therefore, developed and psychometrically tested the Social Functioning in Dementia scale (SF-DEM) scale showing it to be acceptable, reliable and valid12 and establishing its factor structure.13 SF-DEM has, therefore, been recommended for research into social functioning in dementia.11
A statistically significant difference between or within groups on an instrument may not equate to a clinically important difference.14 15 An important metric of a scale is therefore the minimum clinically important difference (MCID), defined as ‘the smallest change or difference in an outcome measure that is perceived as beneficial and would lead to a change in the patient’s medical management, assuming an absence of excessive side effects and costs’.16 To our knowledge, no MCID has been reported for the SF-DEM or any other instrument that measures social functioning in dementia. The MCID is subjective and there are several ways to calculate it, including statistically based approaches based on distribution of data or anchoring score changes to another measure, and seeking expert opinion via a Delphi survey.17 In this study, we aimed to derive the MCID of the SF-DEM using all of these approaches. By utilising three established methods to calculate the MCID and subsequently triangulating the results, we aim to mitigate any potential biases which may arise from the different approaches.
Methods
We conducted a study using data from a cross-sectional survey of family carers of people with dementia and a Delphi survey of dementia professionals and experts by experience.
Cross-sectional survey
Setting and participants
This study used data collected from a previous research project13 to calculate the MCID for the SF-DEM. Two hundred and ninety-nine family carers of people with mild, moderate and severe dementia were recruited across three UK National Health Service mental health trusts in Sussex and North London and gave informed consent to participate.
Procedures
Trained researchers conducted a single interview with the family carer particpants to obtain demographic information about the person with dementia and scores for the SF-DEM. The carers also provided information for other validated carer-rated measures used to determine dementia severity, using the Clinical Dementia Rating Scale (CDR).18
Social functioning was assessed using the SF-DEM, a 20-item questionnaire administered by an interviewer. There are seventeen items covering different aspects of social functioning which are divided into three domains (‘spending time with other people’ (dDomain 1), ‘communicating with other people’ (domain 2) and ‘sensitivity to other people’ (domain 3)). These items are scored using a Likert scale (0–4 indicating frequency: ‘never’ to ‘very often’). A score is calculated for each domain; the scale scores range for each domain from 0 to 21, 0–18 and 0–12, respectively. High scores indicate better social functioning. There are also three unscored summary questions which assess overall impression of social functioning, recent change and willingness to make future changes.
Analysis
Distribution-based methods are based on the statistical characteristics of obtained samples. They determine what magnitude of change in an outcome measure, here the SF-DEM score, is greater than what would be expected from chance alone.19 Previous studies have suggested that 0.5 SD may be clinically significant.20–22 We, therefore, calculated the SD of the SF-DEM score for each of the three domains, and 0.5 of the SD
Anchor-based approach determines the MCID by associating the change in the numerical scale to a subjective and independent assessment of improvement.19 This allows a numerical measurement to become ‘anchored’ to a qualitative assessment which is likely meaningful to patients. In this study we used the carers’ overall impression of social functioning (four points on a Likert scale: excellent, good, fair, poor) to anchor to the SF-DEM score. We first calculated whether there was correlation between the score in each of the three SF-DEM domains and the overall impression of social functioning using Spearman’s correlation coefficient. We then calculated, for each domain, the mean difference in SF-DEM domain score, per different overall rating of impression in social functioning (excellent, good, fair, poor), as the anchor-based MCID.
All analyses were conducted using IBM SPSS V.25.
Delphi survey
Setting and participants
Twenty dementia experts were recruited as participants, providing informed consent to participate in the survey. We defined experts as researchers who specialise in dementia care, healthcare professionals with experience working with people with dementia, or family members, close friends, or carers of a person with dementia with at least weekly contact.
Procedures
We created eight brief fictionalised anonymous vignettes for each of the three SF-DEM domains based on previous studies using the SF-DEM scale (see online supplemental appendix 1).12 13 These changes corresponded to improvement and decline in SF-DEM score of 1, 2, 3 and 4 points. The study participants were asked for each vignette ‘do you consider the change described as important to the health or quality of life of the person’, and asked to answer ‘yes’, ‘no’ or ‘not Sure’. They were invited to explain more about their answer to help move to consensus by clarifying and specifying. The survey was delivered using Opinio, a web-based survey tool.
Supplemental material
Participants were also asked to define their gender (male, female, other, prefer not to say), ethnicity (using UK census categories), expertise (dementia carer, psychiatrist, neurologist, geriatrician, clinical psychologist, nurse, social worker, occupational therapist, researcher, other), country of residence (UK, other European country, North America, other) and years of experience in caring for or working with people with dementia (less than 5, 5–10, more than 10). The survey was piloted by two researchers prior to being circulated to the participants.
Analysis
The Delphi method aims to obtain a consensus regarding what would constitute a meaningful change from a panel of experts in the field, using a questionnaire.23 There is a range of recommendations for Delphi study sample sizes,24 including that 10–15 participants would be sufficient to reach consensus.25 Several rounds may be required before the process ends, in order to reach overall agreement.26 We, therefore, aimed for 15 respondents21 and anticipated an attrition rate of approximately 20%–30% over two or three rounds27 ; thus we recruited 20 experts for the Delphi study. There are no existing guidelines for establishing consensus within a Delphi study, however, many Delphi healthcare studies define consensus as a 75%–80% agreement.28 We, therefore, defined a consensus as an agreement of ≥75% (75% agreement that the change is either considered meaningful or not meaningful).
We analysed the Delphi results by calculating the percentage consensus for each vignette, which corresponded to a change (improvement or decline) of 1, 2, 3 or 4 points in each SF-DEM domain. We then calculated the overall consensus for each point of change by calculating the proportion of study participants who judged that the vignettes reflected a meaningful difference (averaged across the vignettes which described improvement and decline). We judged any level of SF-DEM change as having reached consensus if 75% of participants judged it as being either clinically meaningful or not clinically meaningful. For levels of SF-DEM change where consensus was not reached in the first Delphi round, we wrote new vignettes using feedback and comments provided by the participants in the first round and presented these to study participants for round 2. We determined the MCID using the Delphi method as the minimum agreed meaningful points of change, as decided by a consensus among the Delphi participants. For example, if an overall consensus is reached that a 1-point change is not meaningful or no consensus can be reached, and a 2-point change is meaningful, the MCID would be calculated as 2 points.
Triangulation
We present MCIDs derived from the three methods separately and, as it is common practice to triangulate values from the methods used in order to determine an overall MCID,20 21 29 30 we present a triangulated MCID for each SF-DEM domain by calculating the simple mean (the values summed and divided by three) from the three methods.
Patient and public involvement
No patients involved.
Results
Cross-sectional survey
Demographics
Of the 299 carers, the mean age was 63 (SD 14, min 21, max 90) years and 218 (73%) were female. The mean age of the people with dementia was 81 (SD 8, min 55, max 98) years and 179 (60%) were female. Half (148) of the family carers were spouses or long-term partners of the person with dementia and the majority of those remaining 128 (43%) were children of the person with dementia. There was a range of dementia severity as scored on the CDR (very mild=31 (10%), mild=108 (36%), moderate=99 (33%), severe=61 (20%)). Alzheimer’s disease was the the most common dementia subtype (159, 53%).
Distribution method
The mean SF-DEM score was 6.8, 5.9 and 8.3 for domains 1, 2 and 3, respectively. Table 1 details the values of the mean, SD and 0.5 SD for each of the three domains. The MCIDs, defined as 0.5 SD, were calculated as 1.9, 2.2 and 1.4 points for domains 1, 2 and 3, respectively.
Distribution-based minimum clinically important difference on SF-DEM scale
Anchor method
As is reported in table 2, we found a moderate correlation of SF-DEM score vs overall impression in social functioning in domain 1, and a weak correlation in domains 2 and 3. We then calculated the MCID as the mean difference in score per different level of overall impression of social functioning. The MCIDs were 1.7, 1.7 and 0.9 points for domains 1, 2 and 3, respectively.
Anchor-based minimum clinically important difference on SF-DEM scale
Delphi survey
Demographics
In the first round of the Delphi study, there were twenty participants. Table 3 details the demographic information of the participants in rounds one and two. Approximately two-thirds of the participants were female. The majority of the participants were either psychiatrists (50%) or researchers (30%). They had a varied level of experience, and the majority (95%) were based in the UK. There were 14 participants in the second round of the Delphi survey. Seventy-one per cent of the participants in round 2 were female, and the majority were psychiatrists (43%) or researchers (36%), as in round 1. 93% of participants were based in the UK.
Delphi study participants’ demographic information
Round 1
Table 4 reports the results of both Delphi rounds. In the first round of the Delphi process, a consensus was reached for eight out of the twelve levels of change represented by the vignettes. For domains 1 and 3, there was consensus that 2-point, 3-point and 4-point changes were considered significant but no consensus with regard to a 1-point change. For domain 2, a consensus was reached that 2-point and 4-point changes were considered significant, but no consensus for 1- or 3-point changes.
Delphi survey results
Round 2
We presented amended vignettes for the levels of SF-DEM change which did not reach consensus in round 1. The second round led to a consensus that a 3-point change in domain 2 was clinically significant. However, no consensus was reached with regards to a 1-point change in all three domains with 71%, 43% and 39% viewing 1-point change as meaningful for the three domains (see table 4). Therefore, the Delphi consensus MCIDs are defined as 2 points in all three domains.
Triangulation results
The mean MCIDs from the three methods are 1.9 points for domain 1 (range 1.7–2), 2.0 points for domain 2 (range 1.7–2.2) and 1.4 points for domain 3 (range 0.9–2).
Discussion
We used three different methods, the distribution, anchor and Delphi methods, in order to establish the MCID for the SF-DEM, which measures social functioning in people living with dementia.
The mean MCIDs from the three methods are 1.9 points for domain 1, 2.0 points for domain 2 and 1.4 points for domain 3. If using the MCID for an individual patient, it would be appropriate to consider 2 points as the MCID in all three domains. However, in a research study such as a clinical trial, the triangulated values may be more useful. The MCIDs calculated from the three methods have also been reported separately in this study, and the most applicable value could be used with the researcher’s judgement and prespecified in the study protocol.
For the distribution method, we chose to use the value of 0.5 SD to define the MCID. An influential systematic review demonstrated a consistency of 0.5 SD among the MCIDs reported in health-related quality of life measures in chronic diseases.22 Several studies have since used 0.5 SD as the value to calculate the MCID using the distribution method.20 31–33 The data in the cross-sectional survey13 find a significant correlation between SF-DEM scores and carers’ impression of social functioning. This allowed us to successfully anchor the SF-DEM to the subjective opinions of the family carers and calculate MCIDs using this method. The results of the Delphi survey led to a consensus on 2, 3 and 4-point changes, but to no consensus with regards to a 1-point change in all three domains with some experts judging it as important and others not. Interestingly, in round 2, there was a 71.4% agreement (close to the required 75% consensus) that a 1-point change in domain 1 was significant, in contrast to domains 2 and 3, which had 42.9% and 39.3% agreement for a 1-point change respectively. Nonetheless, we conclude that it would be appropriate to consider a 2-point change in domain 1 as clinically important, particularly when using the MCID to compare the scores of individual patients, given it is the most conservative estimate and so most appropriate in this context.
In other long-term conditions, similar methods have been used to determine an MCID to measure social functioning.34 35 One study used the distribution and anchor method to establish the MCID for a scale measures quality of life (including social functioning) in children with cerebral palsy. Another used the anchor method to determine the MCID for a scale which measures quality of life (including social functioning) after total knee replacement.
Strengths and limitations
Stengths include the fact that no previous study has established an MCID for the SF-DEM. This innovation is therefore useful for future research which uses the SF-DEM as an outcome measure. We also used three different methods in order to calculate the MCID, each of which have their specific benefits and limitations. The distribution and anchor methods were determined using data from the cross-sectional survey which took place in 2019.13 This study used a large research sample, which was diverse in terms of gender, ethnicity, background and severity of dementia. The distribution method is a standardised method of statistical analysis which has been demonstrated to be consistent.22 However, this method is not recommended as a first line means for determining MCID due to the lack of an anchor value which links the scores to a value that is meaningful to patients.19 36 We have mitigated this potential drawback by also using the anchor method, which anchors the score to the subjective views of the family carers, and the Delphi method, thus taking expert opinion into account (in this study encompassing the views of clinicians, researchers, social workers, psychologists and family carers).
A limitation of the anchor-based method is that the results will differ depending on the choice of the anchor.15 The anchor used in this study enabled us to factor in the subjective views and experiences of family carers for people with dementia. These are the individuals who spend the most time with people with dementia, and arguably may have the most insight into a clinically important change. There were 20 participants in first round of the Delphi survey, and 14 participants in the second round. We had a diverse research sample with an appropriate sample size for a Delphi survey. However, the participants were mostly UK-based which renders the results less applicable to populations outside of the UK.
Conclusions
The results of this study specify that the MCID values for the SF-DEM are 1.9 points for domain 1, 2.0 points for domain 2 and 1.4 points for domain 3. These values are derived from the distribution-based, anchor-based and Delphi methods. Given the lack of consensus regarding a 1-point change for all three domains in the Delphi survey, it would be appropriate to round the MCID to a 2-point change as a more conservative value. As the SF-DEM is the only validated scale for measuring social functioning in dementia,13 these results are of potential value for future research in this field. The calculation of the MCID will allow future researchers to identify a change which is of clinical benefit to patients, when using the SF-DEM as an outcome measure and therefore enable research into the important person-centred domain of social functioning for people with dementia.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
The cross-sectional study was approved by the South Central-Hampshire A Research Ethics Committee (15/SC/0605). The Delphi survey was approved by the University College London (UCL) Research Ethics Committee (6048/002).
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @gill_livingston
Contributors TL, GL and AS conceived the idea for this study and designed the analysis plan with input from all authors. TL conducted the literature search and conducted the data analysis plan, with input from AS. SB acquired funding for the cross-sectional study. The article and figures were drafted by TL and GL, SB and AS edited the manuscript. All authors read and approved the final article. AS is the guarantor, who accepts full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Funding The cross-sectional study was funded by an Alzheimer’s Society Project Grant (234 AS-PG-14–017). TL, GL and AS are supported by the University College London Hospitals NIHR Biomedical Research Centre.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.