Article Text
PDF
Abstract
Objectives The My Positive Health (MPH) dialogue tool is increasingly adopted by healthcare professionals in the Netherlands as well as abroad to support people in their health. Given this trend, the need arises to measure effects of interventions on the Positive Health dimensions. However, the dialogue tool was not developed for this purpose. Therefore, this study aims to work towards a suitable measurement scale using the MPH dialogue tool as starting point.
Design A cross-sectional study design.
Participants and settings A total of 708 respondents, who were all members of the municipal health service panel in the eastern part of the Netherlands, completed the MPH dialogue tool.
Methods The factor structure of the MPH dialogue tool was explored through exploratory factor analysis using maximum likelihood extraction. Next, the fit of the extracted factor structure was tested through confirmatory factor analysis. Reliability and discriminant validity of both a new model and the MPH scales were assessed through Cronbach’s alpha tests.
Results Similar to the MPH dialogue tool, the extracted 17-item model has a six-factor structure but named differently, comprising the factors physical fitness, mental functions, future perspectives, contentment, social relations and health management. The reliability tests suggest good to very good reliability of the aimed measurement tool and MPH model (Cronbach’s alpha values ranging from, respectively, 0.820 to 0.920 and 0.882 to 0.933). The measurement model shows acceptable discriminant validity, whereas the MPH model suggests overlap between domains.
Conclusion The results suggest that the current MPH dialogue tool seems reliable as a dialogue, but it is not suitable as a measurement scale. We therefore propose a 17-item model with improved, acceptable psychometric properties which can serve as a basis for further development of a measurement scale.
- quality in healthcare
- health economics
- preventive medicine
- primary care
- public health
- statistics & research methods
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- quality in healthcare
- health economics
- preventive medicine
- primary care
- public health
- statistics & research methods
Strengths and limitations of this study
The main strength of this study was the thorough psychometric analysis to develop a Positive Health measurement scale.
Data from a large group of respondents (n=708) with suitable characteristics for factor analysis were used.
Development of the Positive Health measurement tool was based on the items of the Positive Health dialogue tool, which is widely used in the Netherlands.
This study had a relatively low response rate (22%), which may have created a sampling bias.
Given the relatively small geographic area in which the data are collected, wider generalisation of the present results may be restricted.
Introduction
Over the last years, the European healthcare system is increasingly shifting its focus from cure and disease towards health and healthy behaviour.1 2 In the Netherlands, more attention is given to health-oriented approaches, which focus on health promotion, vitality and possibilities rather than on imperfections.3 This process was accelerated by the demographic situation of an increasing number of elderly people with one or more chronic disease, increasing healthcare expenditures and an increasing wish of citizens for an active and autonomic role during medical consultation.1 4 5 Within the shift towards health-oriented approaches, a focus on health that is broader than only biomedical aspects, and which contributes to achieving a more meaningful life, has gained more interest.6–10 Furthermore, it is increasingly recognised that understanding patients’ experiences about living with a disease is of vital importance in the management of chronic diseases.11
Taking this broader focus into mind, Huber et al 12 proposed a new concept of health, which describes health ‘as the ability to adapt and self-manage, in the face of physical, mental and social challenges’. According to this concept, being healthy reflects the capacity to deal with internal and external stressors, despite possible limitations—and the tendency to adapt to changing conditions. This opposes to the more static current definition of the WHO, which regards health ‘as a complete physical, mental and social well-being and not merely the absence of disease or infirmity’. Although this definition was not intended as such, critics express that the very high ambition has resulted in a major focus on the diagnosis and treatment of symptoms and disease.13–16 However, people with chronic diseases do not automatically see themselves as being unhealthy.17 Similarly, many elderly people consider themselves to be healthy, even if their physical abilities are significantly reduced. To them, being healthy often means to have the ability to fulfil one’s life.10 18 19 This is fully in line with the content of the new concept of 2011.
The Dutch government considered the dynamic concept of health by Huber et al 12 to serve well as a framework for the new strategy within the Dutch healthcare system and required an operationalisation study to make the concept useful for practice.20 During this study, the concept has been further elaborated into the concept of Positive Health, nowadays written with capitals so as to express the specific content. The content of the concept is derived from interviews and focus groups with different stakeholder groups (eg, patients, healthy citizens, healthcare professionals). This inductive, bottom-up approach enabled the researchers to gain a thorough insight into the perceptions about the health of patients and other stakeholders themselves. Positive Health represents a broad perception of health, expressed by six dimensions with 32 underlying aspects, representing indicators for health. The dimensions were by then named bodily functions (BF), mental functions and perception, spiritual existential dimension, quality of life (QL), social and societal participation (SP) and daily functioning (DF). The six dimensions were visualised in a spider web with six axes, representing these six dimensions and ranging from value 0 (in the centre for poor) to 10 (on the periphery, for excellent). Soon, people in practice started to use the spider web in dialogue with patients.
Also, soon after the spider web became available, an attempt was started to transform the dimensions and 32 aspects into a questionnaire that could serve as a validated outcome measurement instrument to measure Positive Health. However, this attempt failed.21 The results of the validity study suggested that the tool was not suitable as a measurement instrument.21 It appeared that the goal of a measurement tool to express outcomes in health in a fixed number was too far from the experienced reality of an individual. Moreover, the 32 items appeared to be rather long for measurement purposes, but also interviews made clear that the language of the present spider web was too complex for many people. Because of this feedback, the six dimensions and the underlying 32 aspects of Positive Health were elaborated by an expert panel group into a more simple language and into 42 aspects, including now (as exceptions) the determinants of living conditions and having enough money. The dimensions were renamed as BF, mental well-being (MW), meaningfulness, QL, participation and DF. The result was called the My Positive Health (MPH) dialogue tool (see https://mijnpositievegezondheid.nl/).
This MPH tool aims to provide individuals insight into their own health and stimulate self-reflection. The target population are Dutch citizens, with or without a chronic disease. By completing 42 statements, mean scores for each of the dimensions are graphically displayed in a spider web (see online supplemental file 1). This spider web can be used during consultations with (for example) healthcare professionals to discuss one’s perceived health and to reveal one’s needs, desires and abilities. Thereby, it could lead to identifiable statements that would find connection with the people concerned. This recognition was confirmed by a user evaluation among populations of healthy citizens, elderly and chronically ill. The vast majority of the respondents rated the MPH dialogue tool good to excellent.22 The tool is widely used in the Netherlands, with more than 100 000 unique users for the digital version since its introduction in 2016 (see MijnPositieveGezondheid.nl; ‘MyPositiveHealth.nl’). In addition, a paper version of the MPH tool is used across a wide range of care centres.
Supplemental material
Along with the extensive use of the dialogue tool, a growing interest is once again observed for a measurement instrument that measures changes in a person’s Positive Health. Such a measurement tool could provide stakeholders in various domains and levels (eg, healthcare professionals, national and local policymakers and insurers) with valid information on the effectiveness of a Positive Health approach. Such insight could support them during the decision-making process and thereby enable a more structural implementation of interventions that improve people’s (positive) health. As far as we were aware, no other validated questionnaire that covers the broad concept of Positive Health is available, and since Prinsen and Terwee,21 no new attempt was made to develop such measurement instrument.23 24 The objective of this study was to set first steps in a new attempt towards a suitable measurement instrument with adequate psychometric properties and scale brevity. This instrument could meet the needs of professionals wishing to evaluate their Positive Health interventions. To reach this objective, we aimed to extract an improved model to measure (positive) health by performing an exploratory factor analysis (EFA) and to test it through confirmatory factor analysis (CFA). Performing a factor analysis will not only help to identify items to measure (positive) health, it will also provide insight into the adequacy of the current arrangement of dimensions and aspects of the MPH dialogue tool. Furthermore, we also aimed to examine the reliability of the MPH dialogue tool.
Methods
Design
This study comprised a cross-sectional quantitative survey study. Data from the quantitative survey were used to develop a new model to measure health by performing an EFA following a CFA.
The instrument
We use the digital version of the MPH dialogue tool (MijnPositieveGezondheid.nl; ‘MyPositiveHealth.nl’) in this study. In this digital version, 42 statements are proposed (7 for each dimension) on an 11-point scale, where 0 means totally disagree and 10 means totally agree. To avoid any missing data, it is not allowed to skip any question.
Participants and data collection
Members of the citizens’ panel of GGD Twente (regional municipal health service) were asked to fill out the 42 questions of the MPH dialogue tool and were asked for their age, gender, educational level, poverty (difficulty getting around) and health literacy (difficulty understanding health information such as leaflets). This panel comprises adults (19+) who took part in the national health survey of GGD Twente in the east of the Netherlands. This survey is carried out every 4 years to monitor the general state of health of Dutch citizens. At the end of that health survey, the participants were asked whether they were willing to participate in other future studies by GGD Twente. Participants were invited by email to complete the questionnaire. Data collection took place from January to February 2018.
Analytical plan
To examine the construct validity of the MPH tool, we used a split-half validation method in which we randomly divided the participants into two groups. We used one of these groups to explore the factor structure through EFA and the other group to test the goodness of fit of the extracted factor structure through CFA. In conducting the factor analyses, we followed best practices described by Brown,25 Costello and Osborne26 and Cabrera-Nguyen.27 Suitability of the data for EFA was examined based on the Kaiser-Meyer-Olkin (KMO) statistics of sampling adequacy and Bartlett’s test of sphericity.28 29
Through EFA, we explored the factor structure using maximum likelihood (ML) extraction. Because we expected our factors to be interdependent aspects of Positive Health, we used ML extraction with direct oblimin rotation. Kaiser criterion, scree plot analysis and parallel analysis were used to determine the number of factors to extract.30 31 We considered items with cross-loading values of ≤0.32 on at least two factors as weak (and thereby as candidates for deletion).26 And we considered items with factor loading of ≥0.60 as strong.32
We strove for a model with improved psychometric properties and scale brevity while maintaining enough items to create stable factors. We therefore aimed to reduce each factor to three items with the highest factor loadings.
Through CFA, we evaluated the goodness of fit of the factor structure extracted during EFA. With the CFA evaluation, we compared the extracted factor structure with two baseline models containing all 42 items of the MPH dialogue tool: the original six-factor structure that includes the six dimensions of health and a one-factor structure that considers all items belonging to one health domain. This comparison helps to understand the degree to which our extracted factor structure fits unseen data better than, respectively, the original six-factor structure and the one-factor structure. We evaluated the goodness of fit using several indices: Pearson’s χ2 test, comparative fit index (CFI; >0.95 is acceptable), Tucker-Lewis index (TLI; >0.95 is acceptable), the root mean square error of approximation (RMSEA; <0.06 is acceptable) and standardised root mean square residual (SRMR; <0.08 is acceptable). These indices reflect model fit (Pearson’s χ2 test), incremental fit (CFI, TLI) and absolute fit (RMSEA, SRMR). The threshold values we applied are cut-off values recommended by Hu and Bentler33 and endorsed by both Brown25 and Cabrera-Nguyen.27 For fitting the models, we used lavaan V.0.6-334 in R V.3.5.1.35 We used ML estimation and standardised the latent factors to allow free estimation of all factor loadings.
Finally, we examined the reliability and discriminant validity of the factors of both the original six-factor model (MPH dialogue tool) and the new experimental model. We examined the reliability by evaluating the Cronbach’s alpha coefficients and tested the discriminant validity by evaluating the factor correlations.
Patient and public involvement statement
Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.
Results
Characteristics of respondents
In total, 3218 participants were invited to enrol in the study. Of those, 708 participants completed the questionnaire (response rate is 22%). The mean age of the respondents was 62 years (SD=15), and 46% of them were women. Most respondents had a high educational level—44% (medium, 34%; low, 22%). And 9% of the respondents indicated some to severe difficulty in getting around (poverty). Low health literacy (difficulty understanding health information such as leaflets) was indicated by 5% of the respondents. Comparing with the general 19+ population in the Twente region, the respondents were older (Twente: mean age=51 years), were higher educated (Twente: high educational level=30%), had higher health literacy (Twente: 9%) and had less difficulty in getting around (Twente: 17%).
Exploratory factor analysis
Our exploration showed that the data are suitable for EFA. Our sample had both an acceptable size of 35632 33 and a very common participant-to-item ratio (8.5:1). The KMO test yielded a statistic of 0.97, implying that the data set contains a significant number of factors, and the Bartlett’s test of sphericity yielded significant results, χ2(41)=318.368, p<0.001, implying that the correlations among variables are greater than one would expect by chance.
Common approaches for determining the number of factors to extract showed support for a six-factor structure. First, the Kaiser criterion method showed that the data contain six factors with eigenvalues greater than 1, suggesting that the data cluster in six factors. Second, the scree plot suggested one, three or six factors because the eigenvalues level off after these amount of factors (see figure 1). Third, parallel analysis (see figure 2) suggested a structure of six factors—the crossing point of the actual scree plot with the possible scree plot based on randomly resampled data. A six-factor structure accounts for 67.5% of the total item variance.
Scree plot. Kaiser criterion is shown by the black dashed line.
Parallel analysis. The six-factor structure is shown by the black dashed line.
Factor loadings are shown in table 1. In this table, factor names for the experimental model are displayed in the second horizontal row. Overall, the items that group together in our EFA mostly group together in the original model as well.
Factor pattern/structure rotated to the oblimin criterion
Table 1 shows that the first factor has factor loadings above 0.40 for several items originating from the dimension SP. These items are as follows: SP29, social contacts; SP30, being taken seriously; SP31, doing fun things together; SP32, having the support of others; and SP33, belonging. We selected the three highest factor loadings (SP29, SP32 and SP33) and labelled this factor ‘social relations’.
The second factor showed loadings above 0.40 on items from the original dimension DF. These items are as follows: DF36, looking after yourself; DF37, knowing your limitations; DF38, knowledge of health; DF39, managing time; and DF40, managing money. We selected the three highest factor loadings (DF37, DF38 and DF39) and labelled this factor ‘daily life-management’.
The third factor showed that loadings above 0.40 were all strong factor loadings (≥0.60) on items from the original dimension BF. These items are as follows: BF1, feeling healthy; BF2, feeling fit; BF5, eating pattern; BF6, physical condition; and BF7, exercise. We included the three items with the highest factor loadings (BF2, BF6 and BF7) and labelled this factor ‘physical fitness’.
The fourth factor showed loadings above 0.40 on items across three dimensions of the dialogue tool: MW, MF and QL. These items are as follows: MW11, being cheerful; MW12, accepting yourself; MW14, having control; MF16, being high-spirited; MF19, accepting life; QL22, enjoyment; QL23, being happy; QL24, feeling good; and QL25, feeling well-balanced. We selected the three highest factor loadings (QL23, QL24 and QL25) and labelled this factor ‘contentment’.
The fifth factor showed loadings above 0.40 on items from one dimension of the dialogue tool: MW and MF. These items are as follows: MW13, being able to handle changes; MF17, wanting to achieve ideals; MF18, feeling confident about own future; MF21, continue learning; and SP34, doing meaningful things. We selected the three highest factor loadings (MW13, MF17 and MF18) and labelled this factor ‘future perspectives’.
The last factor showed that loadings above 0.40 were all strong factor loadings (≥0.60) on items from one dimension of the dialogue tool: MW. These items are as follows: MW8, being able to remember things; and MW9, being able to concentrate. We selected these two highest factor loadings (MW8 and MW9) and labelled this factor ‘mental functioning’, as these aspects solely focus on cognitive abilities and do not concern any emotional aspects or feelings.
In our exploration for a measurement instrument model, we were successful in reducing the number of items for five factors from seven to three items and for one factor to two items. The final factor structure we extracted thus contained 17 items.
Confirmatory factor analysis
The two baseline models, against which we compared the factor structure we extracted during our EFA, had low fits. First, the original six-factor structure had a CFI of 0.846, TLI of 0.835, RMSEA of 0.086 with 90% CI=0.082 to 0.089 and an SRMR of 0.063. Second, the one-factor structure had a CFI of 0.731, TLI of 0.717, RMSEA of 0.112 with 90% CI=0.109 to 0.115 and an SRMR of 0.066. The factor structure we extracted during the EFA, in contrast, had an acceptable fit, with a CFI of 0.964, TLI of 0.953, RMSEA of 0.071 with 90% CI=0.062 to 0.081 and an SRMR of 0.036. This fit was significantly better than the fit of both the original six-factor solution (χ2(700)=2604.48, p<0.001) and the one-factor solution (χ2(715)=4174.19, p<0.001).
The items we selected during the EFA all showed positive factor loadings on their respective domains, with standardised coefficients ranging from 0.71 to 0.97 (see table 2), supporting the factor structure. As we report in table 3, the items within each factor yielded highly consistent response. More specifically, the Cronbach’s alpha values of the factors ranged from 0.82 to 0.92. The six factors correlated significantly positively among each other (see table 3), indicating that individuals who score higher on one domain typically score higher on the other domains. The factor correlations did not exceed 0.80, which suggest acceptable discriminant validity.25 In comparison, the factor correlations of the original model suggest overlap between MF and MW, MW and QL, QL and MF, QL and SP, and SP and DF (see table 4). Cronbach’s alpha values for this original model range from 0.88 to 0.93.
Parameter estimates of confirmatory factor analysis
Means, SD, Cronbach’s alpha (in correlation matrix diagonal) and correlations of the extracted factors (new model)
Means, SD, Cronbach’s alpha (in correlation matrix diagonal) and correlations of the original factors (My Positive Health dialogue tool)
Discussion
The results of our factor analysis support a factor structure of six dimensions. The model we extracted contained 17 items, comprising the factors physical fitness, mental functions, future perspectives, contentment, social relations and daily life-management. The extracted model showed improved construct validity compared with the original model with good fit, high reliability and acceptable discriminant validity. For the MPH tool, our reliability tests suggest good to very good reliability (Cronbach’s alpha values ranging from 0.88 to 0.93). Furthermore, our factor analysis suggests overlap across the dimensions MW, MF, QL and SP, making the MPH tool less suitable as a measurement tool.
This study addresses the difference in aims and thereby the required properties between a dialogue tool and a measurement tool. The results show that a 17-item model has better psychometric properties and can thereby serve as a strong basis for the development of a Positive Health measure. However, the 17-item scale neglects several aspects relevant to address when determining and discussing an individual’s perception of health. For example, for BF, the ignored items about sleeping and eating patterns can inspire ideas and improve dialogue about improving BF. Similarly, it ignores the aspect accepting yourself, whereas this was considered the most important aspect of MW by respondents (18–25 years) in a study that focused on the development of an adolescent version of the tool.36 Importance of this aspect came forward in studies regarding the development of health-related QL and subjective well-being scales as well.6 37 Furthermore, research shows that poor living conditions and financial problems are often accompanied by considerable physical and mental problems.38–40 Inclusion of these aspects in the MPH dialogue tool provides a broader understanding of the multiple needs of an individual. This enables individuals to express their needs during the dialogue about the results and enables to refer them, if required, to appropriate providers. Overall, for two statements within the MPH dialogue tool, it can be argued that they also function as determinants—which influence people’s perceived health—like patients themselves expressed during the original study or consequences of health as well. While such determinants and consequences of health are not appropriately represented in the measurement of health itself, we suggest that they may still be relevant in a dialogue tool.
Difficulties to blend the two goals of both a dialogue tool and a measurement tool were noted earlier. During the development process, an attempt to transform the dimensions and underlying aspects into a questionnaire that could serve as a validated outcome measurement instrument to measure Positive Health has failed.21 At this stage, it appeared that the goal of a measurement tool to express outcome in health in fixed number(s) was too far from the experienced reality of an individual. Based on the results of our study and the previous attempt, we therefore suggest the use of two separate tools: (1) a dialogue tool with the aim to elucidate a broad representation of a person’s perceived health status, comprising a broad range of aspects, and (2) a measurement tool with improved psychometric properties that is able to capture broad health in a valid and reliable way. For this second purpose, our extracted 17-item model may serve as a basis.
When further developing a measurement tool to measure (positive) health, several issues need to be addressed. First, concepts such as salutogenesis, sense of coherence and resilience form a strong basis of Positive Health, focusing on the abilities of individuals to handle changes and experience MF. It should be further investigated whether these concepts are sufficiently addressed in the current 17-item experimental model. For example, the included aspects from the dimension of MF solely focus on future achievements, whereas MF can be regarded as an integration of past, present and future.41 Second, in line with the concept of health as the ability to adapt and to self-manage, one could expect that an individual with poor objective health status can still experience good health when being able to deal well with the situation, whereas another individual with better objective health status can experience lower health. It would be a challenge to develop a measurement tool that does not reflect health by one continuum, but that can capture both realities.
Methodological considerations
This study was based on a survey among a citizen panel in eastern part of the Netherlands (Twente), with a mean age of 62 years (SD=15) and 46% women. This relatively higher age of the study population compared with the whole adult population in this region (mean age=51 years) could reflect a higher interest in the topic of personal positive health status in older adults and may be the result of the relatively low response rate of 22%. To be able to draw more firm conclusions about the psychometric properties of the MPH tool, this should be investigated among other populations as well. This will reveal to which extent difference in age, education level, health literacy and poverty level affects our findings.
In our study, we have focused on the construct validity. Therefore, other psychometric properties such as predictive validity, discriminant validity and responsiveness of the 17- item model remain still unknown. Such issues should be addressed in future studies. Also, we made a first attempt to label the identified factors with factor names. The appropriateness of these names should be further investigated as well.
We based the choice to select three items per factor on both practical and theoretical arguments. From a practical point of view, it is desirable to have a short and easy-to-use scale with acceptable psychometric properties. A scale of 17 items can be completed within a short amount of time. From a theoretical perspective, factors with fewer than three items are often weak and unstable.26 Due to low factor loadings (<0.40), we were not able to select a third item for the factor mental functions. To increase stability within this factor and to improve overall balance of the scale, we suggest to investigate the possibility to develop and include a third item for the factor mental functions in future studies.
Conclusion
In general, we conclude that the overall structure of the MPH dialogue tool seems reliable. While the 42-item model might be suitable as a dialogue tool, this study shows that it is not suitable as a measurement scale. Instead, we propose a 17-item model with a six-factor structure, comprising the factors physical fitness, mental functions, future perspective, contentment, social relations and daily life-management, which can serve as a basis for the development of an additional measurement scale. Given the prevailing healthcare trend towards a focus on health and well-being, expressed by an increased number of practices based on a Positive Health approach, the existence of such measurement scale is of great importance.
Acknowledgments
We thank all respondents from the GGD Twente citizens' panel for completing our questionnaire.
References
Footnotes
Twitter @InstPosHealth, @simoneboerema
Contributors SB collected the data. BMD performed the statistical analysis. MVV interpreted the results and wrote the manuscript in collaboration with BMD and EMvdA-vM. All authors commented and agreed on the final version.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Due to the harmless and non-invasive character of the study, this study was exempt from medical ethical approval. Prior to completion of the questionnaire, informed consent for the use of data for scientific purposes was asked in the invitation email to the participants. All data are handled anonymously.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request. Data set can be requested by sending an email to the corresponding author.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.