Article Text

Download PDFPDF

Inpatients’ assessment of outcome at psychiatric institutions: an analysis of predictors following a national cross-sectional survey in Norway
  1. Oyvind Bjertnaes,
  2. Hilde Hestad Iversen
  1. Department for health services research, Norwegian Institute of Public Health, Oslo, Norway
  1. Correspondence to Dr Oyvind Bjertnaes; oyvindandresen.bjertnaes{at}fhi.no

Abstract

Objectives The objective was to assess the importance of different types of predictors for patient-reported outcome, both background factors at the patient level and healthcare predictors related to structure and processes of healthcare.

Design Cross-sectional patient experience survey.

Setting All 280 secondary care institutions in Norway providing inpatient care for adult psychiatric patients.

Participants 1683 inpatients responded to the questionnaire on-site (73.4%).

Primary outcome measures The outcome scale of the Psychiatric Inpatient Patient Experience Questionnaire–On-Site was the primary dependent variable. The scale consists of five items relating to overall patient satisfaction, benefit of treatment and patient enablement. Regressions were used to assess predictors, for all patients and for five different patient groups reported by the patients including anxiety/depression, drug-related problems and eating disorders.

Results Multilevel linear regression for all patients showed that background factors related to overall current state, self-perceived mental health before admission and admission type were the most important predictors for patient-assessed outcome. Poor current state was associated with poor assessment of outcome (estimate: 8.64, p<0.001), poor health before admission was associated with better outcome (estimate: −6.89, p<0.001) and patients with urgent admission had poorer scores on the outcome scale (estimate: 4.40, p<0.001). A range of structure and healthcare variables were related to patient-assessed outcome, the most important being clinicians/personnel understanding your situation, treatment adjusted to your situation and adequate information about mental health condition.

Conclusions Self-perceived mental health before admission, current overall state and type of admission were the most important background factors for patient-assessed outcome. The most important structure and process variables were related to patient-centred interaction. The background factors should be considered in case-mix adjustments of quality indicators, while the process variables could be used as focus areas in work aiming to improve patients’ assessment of outcome.

  • quality In health care
  • adult psychiatry

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • The study included data from all psychiatric institutions in Norway and a high response rate secure adequate generalisability.

  • The large data set gave the opportunity to conduct both overall analysis and separate analyses for different patient groups.

  • The broadness of factors included in the questionnaire gave the opportunity to assess most potential sources of variation in the patients’ assessment of outcome.

  • The main limitation of our study is the fact that all data were patient-reported.

  • Another important limitation is that patient-assessed outcome was measured at the institutions at one single time point, thus including patients in different phases of their treatment.

Background 

Patient experiences have been linked to clinical effectiveness and patient safety,1 2 and to better adherence to treatment recommendations and less healthcare utilisation.2 Measuring patient experiences and satisfaction is an important part of the evaluation and improvement of health services, including the mental health services.3–5 Such measurements are often used as quality or performance indicators in the public domain,6 7 creating a need to adjust for background factors outside the control of providers including sociodemographic variables. While case-mix of patient-reported hospital experiences are quite standard and knowledge based,8 9 much less research has been done within the field of mental health, and particularly relating to case-mix of patient-assessed outcome.

A main goal for the measurement of patient experiences and satisfaction is quality improvement. This calls for research on predictors of patient satisfaction, especially related to the factors that mental healthcare can influence through quality improvement. Several studies have assessed predictors of patient satisfaction in a mental health setting, including for patients suffering from depression and schizophrenia,10 substance use disorder,11 first-episode psychosis12 13 and major depressive disorders.14 Studies vary widely on most methodological aspects, but common findings include the lack of importance of sociodemographic variables and the importance of disease severity. However, all studies lack several important potential predictors of patient satisfaction. For instance, very few studies included data on coercion, which is an important variable affecting patient experiences and satisfaction,15–17 and only two included patient-centred healthcare processes.11 17 Patient-centred healthcare processes have been measured as staff recovery orientation including involvement of patients and families,11 and strength of therapeutic alliance between patient and clinician.17 Furthermore, most studies on predictors were small, had poorly documented instrument validity and included only one or two specific patient groups.

The objective of the current study was to assess the importance of different types of patient-reported predictors for mental health inpatients’ assessment of outcome, both background factors at the patient level and healthcare predictors related to structure and processes of healthcare. The identification of background factors outside the control of providers is important for case-mix considerations when results are used as quality indicators in the public domain, while healthcare predictors is important for quality improvement work. The study is based on a national survey conducted in Norway in 2016. The instrument has been validated previously15 and includes three scales on patient assessment of inpatient care: structure and facilities, patient-centred interaction and outcome.

The outcome scale functions as the main outcome variable in this study, and consists of five patient-assessed outcome items. One item concerns overall patient satisfaction with the help and treatment received at the institution, another benefit of treatment received at the institution. The three remaining items relates to patient enablement, whether or not patients perceive that the help and treatment they receive help them understand and cope with their mental condition and lead them to believe that their life will improve after discharge. Potential predictors include the scales and items on structure and facilities and patient-centred interaction, several variables on sociodemographics and patient-reported health and other healthcare factors related to coerced/voluntary admission and treatment and offensive or incorrect treatment. The patient-centred interaction scale includes aspects related to how the clinician/personnel inform and involve patients, the content being quite similar to previous research on the effects of healthcare processes.11 17

Based on the current literature, we expected sociodemographic variables to have a small effect on patient-assessed outcome.10–13 We expected self-perceived mental health to be an important predictor for patient-assessed outcome.11 Furthermore, based on two studies that included patient-centred processes,11 17 we expected patient-centred interaction to be one of the most important predictors of patient-assessed outcome.

Methods

The national survey was conducted by the Norwegian Institute of Public Health, commissioned by the Norwegian Directorate of Health. The results from the national survey were published in October 2017.18

Data collection

The population comprised adult (age ≥18 years) inpatients receiving specialised mental healthcare in 2016. Outpatient clinics, day units, specific psychiatric institutions for old patients and interdisciplinary treatment institutions for substance dependence were excluded. All healthcare units in each of the four health regions in Norway were included, yielding a total of 280 units. The sample comprised all patients staying at one of these institutions on an agreed day in week 49 of 2016.

The Norwegian Institute of Public Health established regional contact persons to help compile the institution lists and establish contact persons at a health enterprise level. Two contact persons were established for each participating institution: a project manager and a substitute. The contact person at the institution notified staff about the survey, ensured that the institution complied with the recommended survey guidelines, sent out information regarding administrative data, including a departmental overview, and ensured that a suitable day for survey completion was selected. The institution contacts were responsible for establishing a member of staff for each department who would be responsible for conducting the survey in that department. Tasks included disseminating information to the patients and employees, distributing and collecting questionnaires and reporting to the institute regarding the survey’s progress.

Standardised guidelines for data collection were developed. Patients’ clinicians were not to be involved in the data collection. Patients were requested to complete the questionnaires on their own, without discussing them or being influenced by other patients or employees. Department employees were permitted to read to and help the patients understand the questions but should not influence their response. The departmental responsible professional distributed closed envelopes containing the questionnaire, information letter regarding the survey and reply envelopes to the patients. The reply envelopes containing the questionnaire were then collected once the patients had responded.

Questionnaire

The development and validation of the Psychiatric Inpatient Patient Experience Questionnaire–On-Site (PIPEQ-OS) has been described elsewhere15 and builds on previous research in Norway.19 20 The core part of the PIPEQ-OS comprises three patient-assessed scales, related to structure and facilities (six items), patient-centred interaction (six items) and outcome (five items). To obtain a scale score, patients had to complete at least half of the items in the scale. The experience items had a five-point response format ranging from 1 (‘not at all’) to 5 (‘to a very large extent’), except the item on benefit of treatment (ranged from ‘no benefit’, to ‘very large benefit’). Descriptives for the experience items and scales are presented in table 1. Sociodemographic variables included gender, age and civil status, while other background factors were previous admissions, self-perceived mental health (current and during the week prior to admission), overall current state (five response categories, from ‘Very poor’, to ‘Very good’), duration of stay and type of admission. Other healthcare variables related to the patients’ perception of coerced/voluntary admission and treatment, necessity of admission (five response categories, from ‘Very necessary’, to ‘Very unnecessary’), offensive treatment by personnel (four response categories, from ‘Never’ to ‘Yes, many times’) and incorrect treatment (five response categories, from ‘Not at all’ to ‘To a very large extent’). One question was about the main reason for admission, instructing patients to choose between the following options: eating disorder, drug-related problem, anxiety and/or depression, psychosis or schizophrenia, other diagnosis. The latter plus a few other items were added to the 2016 survey because the Directorate of Health required additional data for an evaluation of a national policy initiative.

Table 1

Descriptives for patient-assessed scales and items*

Statistical analysis

Descriptives for items and scales of the PIPEQ-OS included n, % missing, mean, SD and % of patients choosing the top score/value (ceiling). Bivariate associations with all three scales as dependent variables were tested using one-way analysis of variance for continuous variables and independent samples t-tests for categorical variables. Multilevel linear regression analysis was used to assess the partial effect of each background factor on patient-assessed outcome, controlled for the provider level. Six multilevel regressions were conducted, one for all patients and one for each of the patient groups, including patients with eating disorders, drug-related problems, anxiety and/or depression, psychosis or schizophrenia and other diagnosis. Multilevel modelling divides the total variance in patient-reported experiences into variance at the health enterprise (macro) versus the patient (micro) level.21 The health enterprises were included as random intercepts in the multilevel analysis, with individual-level variables outside the control of the enterprise as fixed effects at the patient level. There were 22 health enterprises in the study, each constituting a number of departments. Linear regressions were conducted to assess the association between structure and process variables and patient-assessed outcome, one for all patients and one for each patient group.

SPSS V.24.0 was used for the statistical analyses.

Approval

The national survey was conducted as an anonymous quality assurance project. According to the joint body of the Norwegian Regional Committees for Medical and Health Research Ethics, research approval is not required for quality assurance projects. The Norwegian Social Science Data Services states that anonymous projects are not subject to notification. Patients were informed that participation was voluntary and their anonymity was assured. Vulnerable patients were protected by permitting the responsible professional at the institution to exclude individual patients for special ethical reasons. Returning the completed questionnaire constituted patient consent, which is the standard procedure in all national patient-experience surveys conducted by the Norwegian Institute of Public Health.

Patient involvement

The survey was about patients’ experiences with involvement and other healthcare issues. Patients were included in the development process of the PIPEQ,19 to secure that the questionnaire included the most important topics for patients.

Results

One thousand six hundred and eighty-three inpatients responded to the questionnaire (73.4%). The items and scales of the PIPEQ-OS had low missing values, varying from 3.3% for the patient-centred interaction scale to 16.8% for the item about information about mental health condition (table 1). Most items and scales were skewed towards positive assessments, but two of three scales scored just above the middle point: the outcome scale scored 58.2 and the patient-centred interaction scale 58.3, on a scale from 0 to 100, where 100 represents the best assessment (table 1). The percentage of patients in the top box score (ceiling) varied from 8.9% to 35.0% on the item level, and were lower than 5% on all scales.

The respondent sample comprised more women than men, and more than 60% of respondents were 44 years of age or younger (table 2). Anxiety/depression was the most common reason for admission (45.2%), followed by psychosis or schizophrenia (21.5%). 35.9% of respondents described their health as very poor or poor, while 27.9% reported more than five previous admissions.

Table 2

Description of sample and associations between background variables and patient-assessed scales*

Bivariate analysis showed that all background variables were significantly related to the three patient-assessed scales, with the exception of age and the main reason for being admitted (table 2). While the main reason for being admitted were unrelated to the outcome scale, patients admitted for a drug-related problem scored somewhat lower than other groups across the three scales. Women had slightly better assessments on the three scales than men and married/cohabitant respondents had better assessments than those living alone. Patients with planned admission were more positive than patients with urgent admission. Overall current state was strongly associated with all scales (p<0.001), with poor state associated with poor scale scores. Self-perceived mental health the last week before admission was also significantly related to all scales (p<0.001), with highest scores for those with poorest health before admission.

Multilevel linear regressions for all patients and each patient group showed that background factors related to overall current state, self-perceived mental health before admission and admission type were the most important predictors for patient-assessed outcome (table 3). Overall current state and self-perceived health were significant in all six regressions, while admission type was significant in four of six regressions. Adjusted for all other background variables in the regression with all patients, the estimate for overall current state were 8.64 (p<0.001), for self-perceived health before admission −6.89 (p<0.001) and for admission type −4.40 (p<0.001). The direction was the same as in the bivariate analysis, with poor current state associated with poor outcomes, poor health before admission associated with higher outcomes, and patients with urgent admission having poorer scores on the outcome scale. Only a few other variables were significant in the regressions for each patient group, and these were mostly small in size and with p values above 0.001.

Table 3

Multilevel regression for all patients and each diagnosis group: association between background factors and patients’ assessment of outcome

Linear regressions for all patients and each patient group showed that three items related to patient-centred interaction were the most important predictors for patient-assessed outcome (table 4): clinicians/personnel understanding your situation, treatment adjusted to your situation and adequate information about mental health condition. None of the health service factors were significant in all six regressions. All items related to coerced/voluntary admission and treatment, and offensive or incorrect treatment, were significantly related to patient-assessed outcome in the regression with all patients (table 4). Coerced treatment was negatively related to outcome both in the regression with all patients (beta: −0.069, p<0.01), for patients with drug-related problems (beta: −0.205, p<0.05) and for patients with psychosis/schizophrenia (beta: −0.119, p<0.05). Three items related to structure and facilities were significantly related to outcome in the regression with all patients (feeling safe, activities, meals), but most items were insignificant in the regressions for the different patient groups (table 4).

Table 4

Linear regression models: association between health service factors as reported by patients and patients’ assessment of outcome

Discussion

Self-perceived mental health before admission, current overall state and type of admission were the most important background factors for patient-assessed outcome. The most important structure and process variables for patients’ assessment of outcome were related to patient-centred interaction.

A literature review found that age and self-perceived health were the most important individual-level predictors for patient satisfaction.22 The review excluded mental health services, but a previous study11 and the current study documents the association between self-reported health variables and patient-assessed outcome. One of the most important variables was how patients perceive their current overall state. The concept of overall current state might reflect a broad set of dimensions, including family relations, financial state, physical and mental health and more, in which the mental health institution might influence certain dimensions but not all. Previous research has shown the importance of such broader dimensions, including general health assessments,23 meaning they should be included in research and quality measurement work in this setting. Self-reported health before admission was the other major background predictor for patients’ assessment of outcome. It is reassuring for mental health services that the sickest patients before admission report the best outcomes, and that this effect relates to all patient groups in the study. However, the use of health transition questions is debated and more research is needed to validate and use this predictor in case-mix adjustments. Finally, the minimal importance of age and other sociodemographic factors in our study coincide with previous research within the field of psychiatric services.10–14

Patient-centred interaction aspects related to clinicians/personnel understanding your situation, treatment adjusted to your situation and adequate information about mental health condition were by far the most important predictors for patients’ assessment of outcome at the institution. The importance of patient-centred interaction coincides with the study by Blonigen and colleagues.11 An increase in explained variance from 0.09 to 0.47 for satisfaction with care and from 0.09 to 0.27 for helpfulness of care was found by adding items related to staff recovery orientation.11 The most important recovery predictor for patient satisfaction in the study by Blonigen and colleagues was listening to and respecting patients’ decisions about care,11 which is very similar to the content of the most important items in our study. Quality work that aims to improve patient experiences and patient-assessed outcome, including patient satisfaction, could benefit from implementing practices and interventions related to patient-centred interaction generally, and particularly relating to patient involvement and empowerment.

Our study showed that factors related to coerced admission and treatment, and offensive or incorrect treatment, were significantly related to patients’ assessment of outcome. These associations seem quite obvious, but it should be underlined that the multivariate analysis controlled for patient-centred interaction, including items on influence on treatment choice and adapting treatment to the patients’ situation. Consequently, the associations between these variables and patient assessment of outcome exist for patients perceiving the same amount of patient centredness. The importance of coercion for patient satisfaction and patient-perceived outcome has been documented elsewhere,15–17 24 but to our knowledge, no previous studies have adjusted for important characteristics related to both healthcare structure and process as in our study. While patients’ subjective perception of coercion and documented coercion might not be correlated,16 our findings imply that practices and interventions that reduce patients’ perception of coercion might increase patient satisfaction and assessment of outcome.

The practical implications of the study relates to the fields of performance measurement and quality improvement. In Norway, national performance measurements are supposed to be used in quality improvement, for managing healthcare institutions and for public accountability and patient choice. These high-stake purposes mean that the measurement efforts must identify and implement an appropriate case-mix adjustment to improve comparability between healthcare providers. The current research shows that several variables should be included in the case-mix model. Furthermore, from a clinical point of view, case-mix is warranted and expected to increase legitimacy of the indicator. One of the goals of national performance measures is quality improvement locally. The current study shows that a number of patient-reported healthcare structure and process aspects are related to patients’ assessment of outcome. Patient-centred interaction is the most powerful predictor for outcome, implying that knowledge-based or patient-based interventions in this area could be an important start point to improve patient assessed outcome. The questionnaire used in this study includes one page for open-ended comments, of particular interest for healthcare providers aiming to find and implement improvement initiatives of value for patients, thus being truly patient-centred. The identification of initiatives should be sensitive to the fact that the health service predictors varied somewhat between the different patient groups in the sample: for instance, adequate information about the diagnosis was the most important predictor for patients with anxiety/depression, while treatment adjusted to your situation was the most important predictor for patients with psychosis/schizophrenia.

The strengths of our study include data from all psychiatric institutions in Norway and a high response rate. This supports both adequate generalisability in Norway and relevance for countries with similar organisation of secondary inpatient care. Furthermore, the national data set included all types of patients, giving the opportunity to conduct both overall analysis and separate analyses for large patient groups like anxiety/depression and psychosis/schizophrenia. The broadness of factors included in the questionnaire gave the opportunity to assess most potential sources of variation in the patients’ assessment of outcome, including sociodemographics, health and mental health variables, the healthcare process and structure variables and coerced/voluntary admission and treatment. The main limitation of our study is the fact that all data were patient-reported. For instance, we are unable to verify patient-reported diagnosis and severity with clinical data on diagnosis and severity. Furthermore, patients had to choose a main diagnosis, which means that we might have overlooked the effects related to comorbidities.10 Another important limitation is that patient-assessed outcome was measured at the institution, with some patients being far from finished with their treatment. In Norway, we are currently exploring the opportunity of conducting continuous electronic measurements close to the time of discharge. The goal is to measure experiences and outcomes when treatment at the institution is close to finished. This adjusted measurement system could be combined with further research on the predictive value of patients’ assessment of outcome on-site, using longitudinal data.25 Lastly, the lack of information about the healthcare providers was a limitation, such as size, personnel, training background and so on. Further research should include more information about the providers, especially as potential predictors to adjust for and explain variation in patient-reported outcome at the provider level.

Conclusions

Self-perceived mental health before admission, current overall state and type of admission were the most important background factors for patient-assessed outcome. The most important structure and process variables for patients’ assessment of outcome were related to patient-centred interaction. The background factors should be considered in case-mix adjustments of quality indicators in this area, while the process variables could be used as focus areas in work aiming to improve patients’ assessment of outcome. Future research should study the predictive value of patients’ assessment of outcome on-site and the meaning and validity of health transition and overall current state items.

Acknowledgments

We would like to thank Inger Opedal Paulsrud, Kirsten Danielsen, Johanne Kjøllesdal and Mona Haugum for data collection. We are also grateful to the contact persons and other cooperating professionals at the involved departments, institutions and health regions and the patients for participating in the survey.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.

Footnotes

  • Patient consent for publication Not required.

  • Contributors OB planned the paper together with HHI, performed the statistical analysis and drafted the manuscript. HHI participated in the planning process, critically revised the manuscript draft and approved the final version of the manuscript.

  • Funding The national survey was financed by the Norwegian Directorate of Health. The current study was financed by the Norwegian Institute of Public Health.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement The Norwegian Institute of Public Health should be contacted for questions on access to data.