You are here

  • Home
  • Archive
  • Volume 8, Issue 6
  • Patient-reported outcome measurements in clinical routine of trauma, spine and craniomaxillofacial surgeons: between expectations and reality: a survey among 1212 surgeons

Article Text

Article menu

Download PDFPDF

XML
Patient-centred medicine
Research
Patient-reported outcome measurements in clinical routine of trauma, spine and craniomaxillofacial surgeons: between expectations and reality: a survey among 1212 surgeons
  1. Alexander Joeris1,
  2. Christian Knoll1,
  3. Vasiliki Kalampoki1,
  4. Andrea Blumenthal1,
  5. George Gaskell2
  1. 1 AO Clinical Investigation and Documentation, AO Foundation, Dübendorf, Switzerland
  2. 2 Department of Methodology, London School of Economics, London, UK
  1. Correspondence to Dr Alexander Joeris; alexander.joeris{at}aofoundation.org

Abstract

Objective To gain information about the advantages/disadvantages of an implementation of patient-reported outcome measures (PROM) into the clinical routine of trauma/orthopaedic surgeons, and to identify the technical constraints confronting a successful implementation of PROMs.

Design Online survey.

Participants Surgeons who are members of the AO Foundation.

Measures Participants answered questions regarding demographics, their familiarity with specific and generic PROMs and the use of PROMs in clinical routine. Furthermore, reasons for/against using PROMs, why not used more often, prerequisites to implement PROMs into clinical routine and whether PROMs would be implemented if adequate tools/technologies were available, were solicited. Χ2 tests and multivariable logistic regressions were conducted to evaluate the effect of the AO Region, surgeon specialisation, current position, clinical experience, and workplace on the familiarity with disease-specific PROMs, the familiarity with generic PROMs and the current use of PROMs. Exploratory factor analysis was used to identify issues underlying the extent of PROM usage.

Results 1212 surgeons completed the survey (response rate: 6.8%; margin of error: ±2.72%): 54.2% were trauma/orthopaedic surgeons, 16.6% were spine surgeons, 27.9% were craniomaxillofacial surgeons and 16 had no defined specialty. Working in a certain AO Region, surgical specialisation and current workplace were associated with a higher familiarity of disease-specific PROMs and the use of PROMs in daily clinical routine (p≤0.05). Exploratory factor analysis identified four categories important for the use of PROMs and two categories preventing the use of PROMs. In case of the availability of an adequate tool, 66.2% of surgeons would implement PROMs in clinical routine.

Conclusions Our survey results provide an understanding of the use of PROMs in clinical routine. There is consensus on the usefulness of PROMs. User-friendly and efficient tools/technologies would be a prerequisite for the daily use of PROMs. Additionally, educational efforts and/or policies might help.

  • public health
  • adult orthopaedics

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-020629

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • The sample of participating orthopaedic, trauma, spine and craniomaxillofacial surgeons could represent the general situation regarding the use of PROMs in clinical routine, with a margin of error of 2.72%.

    • The large number of participants (1212 surgeons) is a strength of this study. Surgeons’ answers provide an understanding of the familiarity with and the use of PROMs in clinical routine.

    • Reasons for the use of PROMs among surgeons already using PROMs and concerns against PROMs among surgeons not using PROMs were investigated separately; explorative factor analysis was applied.

    • Limitations are the low response rate of 6.8%, and a risk of selection bias associated with it.

    • It was assumed that surgeons interested in the topic responded to the survey. Therefore, the familiarity and current use of PROMs may have been rather overestimated.

    Introduction 

    The collection of patient-reported outcome measures (PROM) is increasingly demanded in many healthcare systems for reasons of quality control, benchmarking and reimbursement.1 It is proposed that the routine use of PROMs in clinical care could have the potential to help reform healthcare, its structure and its delivery. Healthcare providers could compare the healthcare provided with patient outcomes achieved in any patient population and use the gathered information to adjust their systems. Clinicians could use it for making treatment decisions for their patients.2

    Individual physicians and/or hospitals increasingly use PROMs, however, in only a few countries, including England, Sweden and partially the USA PROMs are collected on a regional or national level.2 In England, a limited mandatory collection of PROMs was initiated by the government in 2009.3 4 In Sweden, the initiative to collect disease-specific PROMs nationwide was clinician driven.2

    In 2002, the National Institutes of Health (NIH) in the USA defined the need to develop state-of-the-art PROMs as a high priority scientific project to enhance clinical research in the 21st century.5 Starting in 2004, the NIH developed the Patient-Reported Outcomes Measurement Information System (PROMIS) together with an online assessment centre1 6 to provide generic measures of the most important health outcome domains for many diseases. This freely available system enables improved efficiency and practicability for implementing PROMs in the clinical setting, using computer adaptive testing (CAT).1 6–10

    A few investigations on clinicians’ attitudes to using PROMs in daily routine and/or on implications of using PROMs have been published, including surveys among paediatricians, oncologists, psychiatrists, spine surgeons and systematic literature reviews.11–19 Little information on the familiarity of trauma and orthopaedic surgeons with PROMs is available. Also, there is limited information on the use of PROMs in daily routine. This study aimed to gain detailed information on the perceived advantages and disadvantages of PROM implementation, and the important constraints and reservations for a successful implementation of such procedures.

    Methods

    Survey

    Information was gathered from members of the AO Foundation, a medically guided non-profit organisation led by an international group of surgeons specialising in the treatment of trauma and disorders of the musculoskeletal system (www.aofoundation.org). A questionnaire was sent to 17 931 surgeons (9567 trauma, 6054 spine and 2310 craniomaxillofacial (CMF) surgeons) via SurveyMonkey, together with an invitation email explaining the purpose of the survey and a secure web link. Participation in the survey was anonymous and voluntary, and no remuneration or reward was offered for participation. No reminders were sent. As the questionnaire was developed, based on similar published surveys that evaluated surgeons’ opinions on PROMs among other medical specialties,11 17 20 no pilot survey was conducted.

    Local Institutional Review Board approval has not been obtained as no medical information was collected from the participants, and data were collected and analysed anonymously. However, all participants were informed that the survey results would be published.

    The questionnaire is available in online supplementary file 1. It comprised seven general questions, eliciting demographic information, the region of residence (AO Regions Africa, Asia-Pacific, Europe, Latin America, Middle East and North America), surgical specialisation, current working position, clinical experience and current workplace. Another three questions solicited surgeons’ familiarity with generic or disease-specific PROMs, and surgeons’ current use of PROMs. Detailed statements either about surgeons’ frequent use of PROMs in clinical routine, or not using PROMs in clinical routine were to be made. Therefore, if surgeons responded to currently use PROMs regularly or infrequently in daily clinical work, or both, in daily clinical work and research, their opinion on 10 detailed statements (Question A, online supplementary file 1) regarding their familiarity and usage of generic and disease-specific PROMs in clinical routine was solicited. Whereas if surgeons responded to currently not use PROMs, or to just use them for research, their opinion on seven detailed statements regarding their reasons to not use PROMs was solicited (Question B, online supplementary file 1).

    Supplemental material

    In addition, information on three further aspects of PROMs was collected: the reasons why PROMs are not used more often in clinical routine (four items); what would be the most important prerequisites to implement PROMs in clinical routine (five items); and whether surgeons would use PROMs in their clinical routine (assuming the availability of tools/technologies to overcome reported obstacles) (one item).

    All questions were optional, and surgeons had the possibility to skip questions.

    Patient and public involvement

    There was no patient involvement in the presented survey. The participating surgeons were not involved in the development of this questionnaire. The survey results will be published and made available in this way. Once the survey results are published AOTrauma, AOSpine and AOCMF will be informed so that the results can be distributed among their members.

    Statistical analyses

    The response rate (including completed and partially completed questionnaires) and the margin of error at 95% confidence (expressing the amount of random sampling error) were computed. Percentages for all categorical variables (excluding missing responses) were computed. To examine associations between two categorical variables, Χ2 tests were applied (univariable analyses). Multivariable logistic regression analyses were conducted to evaluate the effect of ‘region of residence’, ‘surgical specialization’, ‘current position’, ‘clinical experience’ and ‘workplace’ on (A) the familiarity with existing generic PROMs, (B) the familiarity with existing disease-specific PROMs, and (C) the current use of PROMs in clinical routine. Listwise deletion applied if a respondent stated ‘other’ for surgical specialisation, current position or workplace. The significance level was set at p<0.05.

    Exploratory factor analyses (EFA) were performed for the questions relating to ‘why to use PROMs’ and for the questions ‘why to not use PROMs’ to identify broader issues underlying decisions.21 Factors were extracted using the principal component method and those factors with eigenvalues greater than 1 (Kaiser criterion) were retained.22 EFA with varimax rotation was used as a data reduction technique to create interpretable results from the retained factors.23

    All statistical analyses were performed using the software SAS V.9.4 (SAS Institute).

    Results

    One thousand two hundred and twelve surgeons of the AO Foundation network (17 931 surgeons; response rate: 6.8%) completed the online survey. The computed margin of error for the results of the survey was ±2.72%. Six hundred fifty-seven (54.2%) were trauma and orthopaedic surgeons, 201 (16.6%) were spine surgeons, 338 (27.9%) were CMF surgeons and 16 (1.3%) had no defined clinical specialty. Demographic data and details about the AO Region, surgical specialisation, current position, clinical experience and workplace are listed in table 1.

    View this table:
    Table 1

    Summary of demographics and working experience of participating surgeons

    Overall, 661 (54.5%) reported being familiar with respective disease-specific PROMs, and 547 surgeons (45.1%) reported being familiar with generic PROMs. The univariable analysis showed that surgeons in Europe and North America were more familiar with specific PROMs (Europe: 68.5%; North America: 72.7%) and generic PROMs (Europe: 60.4%; North America: 66.4%) than those in the remaining regions (p<0.001). Spine surgeons were most familiar with both, specific PROMs (85.6%) and generic PROMs (63.7%), whereas CMF surgeons were least familiar with PROMs in general (specific: 13.6%; generic: 31.7%). ‘Surgeons’ current position’, ‘clinical experience’ and ‘workplace’ were also associated with their familiarity of PROMs (p<0.05) (table 2).

    View this table:
    Table 2

    Univariable analysis evaluating the familiarity, and current use of PROMs in clinical routine and/or clinical research, according to geographical region, surgical specialty, current position, clinical experience and workplace

    The multivariable analysis confirmed the regional differences, differences between surgeon specialties and workplaces in the familiarity with disease-specific PROMs (table 3). Of note, surgeons working either in a private or a public hospital were approximately half as likely to be familiar with disease-specific PROMs than surgeons working in a university hospital (p≤0.05). No evidence for an association between familiarity with disease-specific PROMs and ‘current position’ or ‘clinical experience’ was found (table 3).

    View this table:
    Table 3

    Multivariable logistic regression analyses evaluating the effect of AO Region, specialisation, current position, clinical experience, and workplace on the familiarity with generic PROMs, the familiarity with disease-specific PROMs and the current use of PROMs

    For the familiarity with generic PROMs, above results were also confirmed in the multivariable analysis. In addition, the respondents’ ‘current position’ was found to be a significant variable in this model. Compared with registrars, consultants and senior managers or professors were more familiar with generic PROMs (p<0.05) (table 3).

    Only 97/1210 (8.0%) respondents used PROMs regularly in their clinical routine, 134 (11.1%) irregularly, 94 (7.8%) for both clinical routine and research purposes, and 253 (20.9%) for research purposes only, compared with 632 (52.2%) who did not use PROMs at all. As with ‘familiarity’, the univariable analysis showed a distinct difference in the clinical use of PROMs between the different regions and specialisations: PROMs were most frequently used in Europe (32.5% of surgeons used PROMs in clinical routine or clinical research) and North America (31.3%). Spine surgeons used PROMs the most (45.3%), whereas CMF surgeons used PROMs the least (15.4%). Moreover, ‘surgeons’ clinical experience’ seemed to be associated with the routine use of PROMs (p=0.024) (table 2).

    The multivariable analysis again confirmed the results from the univariable analysis, regarding ‘AO Region’, and ‘surgeons’ specialization’. Contrary to the univariable analysis, ‘workplace’ instead of ‘surgeons’ clinical experience’ was associated with use of PROMs in daily clinical routine (table 3).

    Of 325 surgeons who responded to currently use PROMs in clinical routine (irregularly or regularly) or both in daily clinical work and research, 292 provided their opinions to the 10 detailed statements regarding their familiarity and routine use of PROMs. Five of the 10 statements got more than 80% agreement (either strongly agreed or agreed), and four between 67% and 78% agreement. Only 34% of surgeons agreed that they were required to document quality of life (QoL) data by the government (last statement) (table 4).

    View this table:
    Table 4

    Detailed reasons for and against collecting PROMs in clinical routine, provided by surgeons

    Available (non-missing) data/opinions of 281 surgeons who used PROMs in their daily routine, related to the 10 provided statements, were factor analysed. Four factors were identified with eigenvalues greater than 1 (figure 1) and accounted for 66% of the total variance. We labelled the factors as follows: factor 1: quality of care, factor 2: measurement of outcomes, factor 3: regulations and factor 4: limitations of PROMs. The main factor loadings on the 10 items are shown in figure 2.

    Figure 1
    Figure 1

    Scree plot showing the variance in the data (eigenvalues) of the explorative factor analysis for surgeons who used patient-reported outcome measures (PROM) in their daily routine. The line until factor 4 shows the four of 10 components with an eigenvalue >1 which were included in the explorative factor analysis. The first four factors accounted for 66% of the total cumulative variance.

    Figure 2
    Figure 2

    Factor analysis path diagram displaying the variance of agreement/disagreement for each statement (why patient-reported outcome measures (PROM) are used in clinical routine) explained by the different factors. Factor 1: quality of care, factor 2: measurement of outcomes, factor 3: regulations, factor 4: limitations of PROMs.

    Of 885 surgeons who responded to currently not use PROMs or to only use PROMs in research, 768 provided their opinions to the seven detailed statements regarding their reasons to not use PROMs. Four of the seven statements got less than 31% agreement (either strongly agreed or agreed), one got 47% (PROMs are too time consuming) and two statements got more than 60% agreement (60%: I am lacking sufficient information, knowledge or experience to use PROMs; 70.6%: I am interested but did not yet have the possibility to use PROMs) (table 4). For the second factor analysis, all answers of 739 surgeons to the seven items were used: two factors were identified accounting for 51% of the cumulative variance (figure 3 and 4): factor 1: overall assessment of PROMs; factor 2: institutional responses regarding to PROMs.

    Figure 3
    Figure 3

    Scree plot showing the variance in the data (eigenvalues) of the explorative factor analysis for surgeons who did not use patient-reported outcome measures (PROM) in their daily routine. The line until factor 2 shows the two of seven components with an eigenvalue >1 which were included in the explorative factor analysis. The first two factors accounted for 51% of the total cumulative variance.

    Figure 4
    Figure 4

    Factor analysis path diagram displaying the variance of agreement/disagreement for each statement (why patient-reported outcome measures (PROM) are not used in clinical routine) explained by the different factors. Factor 1: overall assessment of PROMs, factor 2: institutional responses in regard to PROMs.

    When surgeons were asked for further aspects considered important for an implementation of PROM-collecting instruments into the daily clinical routine, user-friendliness was very important or important for 95.2% of surgeons (938/986). But also, other aspects such as costs, time efficiency, comparability with existing software and the interpretability and clinical relevance of the results were considered important or very important (table 5).

    View this table:
    Table 5

    Aspects considered important for an implementation of PROM-collecting instruments into the daily clinical routine

    When surgeons were asked whether they would implement PROMs into their daily routine, if tools/technologies to overcome barriers were available, 656/1212 (66.2%) agreed, whereas 335 (33.8%) were not sure yet. Of those 656 surgeons who agreed, 364 (75.4%) were using PROMs in any form at the time of the survey, and 292 (57.6%) were not using PROMs at the time of the survey (p<0.001). In a multivariable logistic regression model, surgeons from Africa were 2.67 (95% CI 1.05 to 6.78) times more likely to be willing to use a tool that would overcome barriers in collecting PROMs in clinical routine than surgeons from Europe; orthopaedic surgeons (OR: 2.11; 95% CI 1.44 to 3.09), trauma surgeons (OR: 1.90; 95% CI 1.30 to 2.79) and spine surgeons (OR: 1.74; 95% CI 1.14 to 2.66) were more likely than CMF surgeons. Surgeons working in a private hospital were half as likely (OR: 0.49; 95% CI 0.35 to 0.69) as surgeons working in a university hospital (p<0.05 for all).

    Discussion

    Our worldwide study surveyed the familiarity of trauma/orthopaedic, spine and CMF surgeons with PROMs, their current use in clinical routine, reasons for using or not using them and respondents’ perceptions of the important aspects to be considered in implementing PROMs in clinical routine. In a multivariable regression model, familiarity with existing generic and disease-specific PROMs was influenced by ‘geographic region’, ‘surgical specialization’, ‘current workplace’ and for generic PROMs, by ‘current working position’. The use of PROMs in clinical routine was again influenced by ‘geographic region’, ‘surgical specialization’ and ‘current workplace’.

    In a survey published by the Dutch Pediatric Association, 69% of 303 paediatricians reported being familiar with patient-reported QoL data.11 Another survey among the Latin-American AOSpine members reported familiarity with generic PROMs in 79% of 199 participants and familiarity with disease-specific PROMs in 86%.18 In our study, 45% of surgeons were familiar with generic and 55% with disease-specific PROMs. Interestingly, familiarity varied according to surgical specialisation and geographic region and was in general higher for disease-specific PROMs than for generic PROMs. A few European countries, that is, the UK and Sweden,3 24–26 and the USA (http://www.hosonline.org) have started collecting PROMs on a routine basis, which could be a reason for the higher local general awareness. The low overall familiarity of CMF surgeons with PROMs in our study can probably be explained by the lack of well-established PROMs for these patients. In contrast, well-established PROMs for patients with spinal disorders such as the Oswestry Disability Index27 and the Neck Disability Index28 may have contributed to the higher familiarity (of 86% of spine surgeons in our study) with PROMs within this specialty. The study by the Dutch Pediatric Association found that paediatricians working at a university hospital were significantly more familiar with the existence of QoL questionnaires than those working in a community hospital (p=0.007), and paediatric registrars were significantly less familiar with the existence of QoL questionnaires than experienced paediatricians (p=0.019).11 These results were confirmed for the surgeons/respondents participating in our survey (p<0.05).

    PROMs are rather new in the field of trauma surgery and, as such, clinicians’ awareness might well be expected to be lower than in other medical disciplines. This may explain why only 26.9% of surgeons in our study used PROMs in their daily routine and only 8.0% used PROMs regularly. Higher proportions are reported from surveys which involved practitioners dealing with patients with chronic illness.11 17 20 Interestingly, ‘current working position’ and ‘level of clinical experience’ did not have a significant influence on the use of PROMs. We speculate that less experienced clinicians are not familiar with PROMs but are obliged to collect them on behalf of a more senior clinician as first point of contact.

    We found a high degree of agreement among surgeons who routinely use PROMs regarding the reasons for doing so, such as prioritisation of a clinical problem, facilitation of communication between the clinician and the patient, monitoring changes/responses to the treatment, and so on. These reasons are cited in the literature and indicate that with the help of PROMs patient care and treatment can be focused more on the individual patient rather than on the disease. Moreover, the quality of healthcare delivery can be monitored and improved.18 25 28 29 Our factor analysis of the questionnaire showed that surgeons see four broad categories of reasons for using PROMs: to improve the quality of care; to contribute to the measurement of outcomes; compliance with regulation and that PROMs should be limited to the assessment of individual patient outcomes.

    For policymakers, governments, regulatory bodies and funders, awareness of these four categories is important to develop incentives to increase surgeons’ motivation for using PROMs in clinical routine. At the same time, it is as important to realise why PROMs are not used more often in clinical routine. Concerns mentioned by surgeons are: absence of institutional policies to guide the process and concerns about the length of PROM questionnaires, the time to get the results, the impression that PROMs are burdensome and time consuming, and the costs and infrastructural changes related to PROM implementation.2 17–19 30 Other reasons are the lack of knowledge and experience to interpret results and to use them in clinical practice, but also doubts on the compatibility of results with routinely collected clinical measurements.11 20 30–33 Furthermore, surgeons seem to be sceptical about PROMs, because they are thought to be ‘soft and subjective’.4 34

    In our study, 87.0% of all surgeons who did not routinely use PROMs disagreed with the statement that PROMs are not useful in orthopaedics and traumatology, and 61.9% agreed that they do not use PROMs due to insufficient information, knowledge or experience. Another 70.6% of surgeons stated that they would be interested in using PROMs but did not yet have the opportunity to do so. The factor analysis showed two broad constraints to the implementation of PROMs: cost and institutional ignorance and inertia. Developing policies to increase information and educational activities and improved tools/technologies for PROM use could help overcome these obstacles. Important prerequisites to implement and use PROMs were found among surgeons in our study: there was a belief in the usefulness of PROMs and a willingness to use PROMs if user-friendly technologies were available.

    PROMIS instruments (http://www.healthmeasures.net) are one of the most important and promising developments in terms of advancement of PROMs. By using sophisticated algorithms, item selection is automatically matched to the health level of respondents, and the number of questions is reduced to a minimum.1 5–7 35 36 Available PROMIS instruments are useful for patients in the field of orthopaedics and traumatology (eg, the physical function CAT and pain interference CAT instruments), and even superior to established legacy instruments, due to various reasons (eg, taking less time to administer).37–45

    Developments to increasingly implement PROMs into clinical routine, and even into electronic health record systems are apparent. Additionally, stand-alone software to collect PROMs is becoming available for clinical routine. Clinicians immediately see the current health status of their patients and former scores and can monitor changes over time.

    The current study has limitations, mainly the rather low response rate of 6.8%, which gives rise to selection/non-response bias. Comparing our demographic survey data with available internal data derived from a membership statistic of the AO Foundation from 2018, younger surgeons aged between 25 and 34  years were underrepresented in our survey (deviations within specialties and regions: 10%–20%), surgeons aged between 35 and 44, and between 45 and 54 were over-represented (deviations: 5%–11%). CMF surgeons were over-represented in all regions (deviations: 13%–34%) and balanced by spine and trauma surgeons who were under-represented (deviations: 5%–18%). The AO Asian Pacific region was under-represented among trauma and spine surgeons (deviations: 10%–12%), while Europe was over-represented (deviations: 6%–14%). The Middle East was over-represented by 5.5%, while Latin America was under-represented by 8% among CMF surgeons. The gender distribution is considered representative within regions and specialties.

    In addition, response to the survey was voluntary, and it can be assumed that mainly surgeons interested in the topic responded, which in turn may mean that the current use of PROMs is rather overestimated, that is, PROMs are even less known and less used in clinical routine than reflected by the survey.

    On the other hand, the margin of error was only 2.72%, implying that the random sampling error was low. Therefore, survey estimates can be considered precise and representative for the whole population of AO surgeons. Keeping in mind that the AO Foundation has nearly 18 000 members worldwide, results could reflect the general situation regarding the use of PROMs in clinical routine.

    Conclusions

    Our research survey among trauma surgeons provides an understanding of the familiarity with PROMs and their current use in clinical routine. There is consensus on the usefulness of PROMs. A majority of surgeons is willing to implement PROMs, if an adequate tool would be available. The development of user-friendly and efficient tools/technologies both for patients and clinical staff would be an important facilitator to increase the use of PROMs in clinical routine. Additionally, education and/or policies for PROM usage in clinical routine might help.

    Acknowledgments

    The authors thank Ursula Saiti and the AO Clinical Divisions, namely AOTrauma, AOSpine and AOCMF for their support in distributing the online survey among their members.

    References

    1. 1.↵
      2. Broderick JE ,
      3. DeWitt EM ,
      4. Rothrock N , et al
      . Advances in Patient-Reported Outcomes: The NIH PROMIS(®) Measures. EGEMS 2013;1:12.doi:10.13063/2327-9214.1015
      OpenUrl
    2. 2.↵
      2. Black N
      . Patient reported outcome measures could help transform healthcare. BMJ 2013;346:f167.doi:10.1136/bmj.f167
      OpenUrlAbstract/FREE Full Text
    3. 3.↵
      2. Browne J ,
      3. Jamieson L ,
      4. Lewsey J , et al
      . Patient Reported Outcome Measures (PROMs) in Elective Surgery: Health Services Research Unit, London School of Hygiene & Tropical Medicine, Clinical Effectiveness Unit: Royal College of Surgeons of England, 2007.
    4. 4.↵
      Cambridge University Press. Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects (Health Economics, Policy and Management) , 2010.
    5. 5.↵
      2. Ader DN
      . Developing the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care 2007;45:S1–S2.doi:10.1097/01.mlr.0000260537.45076.74
      OpenUrlCrossRefPubMed
    6. 6.↵
      2. Cella D ,
      3. Yount S ,
      4. Rothrock N , et al
      . The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care 2007;45(5 Suppl 1):S3–S11.doi:10.1097/01.mlr.0000258615.42478.55
      OpenUrlCrossRefPubMedWeb of Science
    7. 7.↵
      2. DeWalt DA ,
      3. Rothrock N ,
      4. Yount S , et al
      . Evaluation of item candidates: the PROMIS qualitative item review. Med Care 2007;45(5 Suppl 1):S12–21.doi:10.1097/01.mlr.0000254567.79743.e2
      OpenUrlCrossRefPubMedWeb of Science
    8. 8.↵
      2. Gershon RC ,
      3. Rothrock N ,
      4. Hanrahan R , et al
      . The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. J Appl Meas 2010;11:304–14.
      OpenUrlPubMed
    9. 9.↵
      2. Hays RD ,
      3. Liu H ,
      4. Spritzer K , et al
      . Item response theory analyses of physical functioning items in the medical outcomes study. Med Care 2007;45(5 Suppl 1):S32–8.doi:10.1097/01.mlr.0000246649.43232.82
      OpenUrlCrossRefPubMedWeb of Science
    10. 10.↵
      2. Valderas JM ,
      3. Alonso J
      . Patient reported outcome measures: a model-based classification system for research and clinical practice. Qual Life Res 2008;17:1125–35.doi:10.1007/s11136-008-9396-4
      OpenUrlCrossRefPubMedWeb of Science
    11. 11.↵
      2. Baars RM ,
      3. van der Pal SM ,
      4. Koopman HM , et al
      . Clinicians' perspective on quality of life assessment in paediatric clinical practice. Acta Paediatr 2004;93:1356–60.doi:10.1111/j.1651-2227.2004.tb02937.x
      OpenUrlPubMed
    12. 12.↵
      2. Bezjak A ,
      3. Ng P ,
      4. Skeel R , et al
      . Oncologists' use of quality of life information: results of a survey of Eastern Cooperative Oncology Group physicians. Qual Life Res 2001;10:1–14.doi:10.1023/A:1016692804023
      OpenUrlCrossRefPubMedWeb of Science
    13. 13.↵
      2. Gilbody SM ,
      3. House AO ,
      4. Sheldon TA
      . Psychiatrists in the UK do not use outcomes measures. National survey. Br J Psychiatry 2002;180:101–3.doi:10.1192/bjp.180.2.101
      OpenUrlFREE Full Text
    14. 14.↵
      2. Gilbody SM ,
      3. House AO ,
      4. Sheldon TA
      . Routinely administered questionnaires for depression and anxiety: systematic review. BMJ 2001;322:406–9.doi:10.1136/bmj.322.7283.406
      OpenUrlAbstract/FREE Full Text
    15. 15.↵
      2. Gilbody SM ,
      3. House AO ,
      4. Sheldon T
      . Routine administration of Health Related Quality of Life (HRQoL) and needs assessment instruments to improve psychological outcome--a systematic review. Psychol Med 2002;32:1345–56.doi:10.1017/S0033291702006001
      OpenUrlPubMed
    16. 16.↵
      2. Gilbody SM ,
      3. House AO ,
      4. Sheldon TA
      . Outcomes research in mental health. Systematic review. Br J Psychiatry 2002;181:8–16.
      OpenUrlAbstract/FREE Full Text
    17. 17.↵
      2. Taylor KM ,
      3. Macdonald KG ,
      4. Bezjak A , et al
      . Physicians' perspective on quality of life: an exploratory study of oncologists. Qual Life Res 1996;5:5–14.doi:10.1007/BF00435963
      OpenUrlCrossRefPubMedWeb of Science
    18. 18.↵
      2. Teles AR ,
      3. Righesso O ,
      4. Gullo MC , et al
      . Perspective of Value-Based Management of Spinal Disorders in Brazil. World Neurosurg 2016;87:346–54.doi:10.1016/j.wneu.2015.11.047
      OpenUrl
    19. 19.↵
      2. Nelson EC ,
      3. Eftimovska E ,
      4. Lind C , et al
      . Patient reported outcome measures in practice. BMJ 2015;350:g7818.doi:10.1136/bmj.g7818
      OpenUrlFREE Full Text
    20. 20.↵
      2. Morris J ,
      3. Perez D ,
      4. McNoe B
      . The use of quality of life data in clinical practice. Qual Life Res 1998;7:85–91.doi:10.1023/A:1008893007068
      OpenUrlCrossRefPubMedWeb of Science
    21. 21.↵
      2. Cudeck R ,
      3. Analysis EF
      . In: Tinsely HEA , Brown SD , eds. Handbook of Applied Multivariate Statistics and Mathematical Modeling: Elsevier Inc, 2000:265–96.
    22. 22.↵
      2. Kaiser HF
      . The Application of Electronic Computers to Factor Analysis. Educ Psychol Meas 1960;20:141–51.doi:10.1177/001316446002000116
      OpenUrlCrossRefWeb of Science
    23. 23.↵
      2. Kaiser HF
      . The varimax criterion for analytic rotation in factor analysis. Psychometrika 1958;23:187–200.doi:10.1007/BF02289233
      OpenUrlCrossRefWeb of Science
    24. 24.↵
      2. Feltelius N ,
      3. Fored CM ,
      4. Blomqvist P , et al
      . Results from a nationwide postmarketing cohort study of patients in Sweden treated with etanercept. Ann Rheum Dis 2005;64:246–52.doi:10.1136/ard.2004.023473
      OpenUrlAbstract/FREE Full Text
    25. 25.↵
      2. McNeil JJ ,
      3. Evans SM ,
      4. Johnson NP , et al
      . Clinical-quality registries: their role in quality improvement. Med J Aust 2010;192:244–5.
      OpenUrlPubMedWeb of Science
    26. 26.↵
      2. van Vollenhoven RF ,
      3. Askling J
      . Rheumatoid arthritis registries in Sweden. Clin Exp Rheumatol 2005;23(5 Suppl 39):S195–200.
      OpenUrlPubMedWeb of Science
    27. 27.↵
      2. Fairbank JC ,
      3. Couper J ,
      4. Davies JB , et al
      . The Oswestry low back pain disability questionnaire. Physiotherapy 1980;66:271–3.
      OpenUrlPubMedWeb of Science
    28. 28.↵
      2. Vernon H ,
      3. Mior S
      . The Neck Disability Index: a study of reliability and validity. J Manipulative Physiol Ther 1991;14:409–15.
      OpenUrlPubMedWeb of Science
    29. 29.↵
      2. Higginson IJ ,
      3. Carr AJ
      . Measuring quality of life: Using quality of life measures in the clinical setting. BMJ 2001;322:1297–300.doi:10.1136/bmj.322.7297.1297
      OpenUrlFREE Full Text
    30. 30.↵
      2. Valderas JM ,
      3. Kotzeva A ,
      4. Espallargues M , et al
      . The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res 2008;17:179–93.doi:10.1007/s11136-007-9295-0
      OpenUrlCrossRefPubMedWeb of Science
    31. 31.↵
      2. Fitzpatrick R ,
      3. Fletcher A ,
      4. Gore S , et al
      . Quality of life measures in health care. I: Applications and issues in assessment. BMJ 1992;305:1074–7.doi:10.1136/bmj.305.6861.1074
      OpenUrlAbstract/FREE Full Text
    32. 32.↵
      2. Lohr KN ,
      3. Zebrack BJ
      . Using patient-reported outcomes in clinical practice: challenges and opportunities. Qual Life Res 2009;18:99–107.doi:10.1007/s11136-008-9413-7
      OpenUrlCrossRefPubMedWeb of Science
    33. 33.↵
      2. Meadows KA ,
      3. Rogers D ,
      4. Greene T
      . Attitudes to the use of health outcome questionnaires in the routine care of patients with diabetes: a survey of general practitioners and practice nurses. Br J Gen Pract 1998;48:1555–9.
      OpenUrlAbstract/FREE Full Text
    34. 34.↵
      2. Deyo RA ,
      3. Patrick DL
      . Barriers to the use of health status measures in clinical investigation, patient care, and policy research. Med Care 1989;27(3 Suppl):S254–68.doi:10.1097/00005650-198903001-00020
      OpenUrlCrossRefPubMedWeb of Science
    35. 35.↵
      2. Cella D ,
      3. Riley W ,
      4. Stone A , et al
      . The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol 2010;63:1179–94.doi:10.1016/j.jclinepi.2010.04.011
      OpenUrlCrossRefPubMedWeb of Science
    36. 36.↵
      2. Reeve BB ,
      3. Hays RD ,
      4. Bjorner JB , et al
      . Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care 2007;45(5 Suppl 1):S22–31.doi:10.1097/01.mlr.0000250483.85507.04
      OpenUrlCrossRefPubMedWeb of Science
    37. 37.↵
      2. Hung M ,
      3. Stuart AR ,
      4. Higgins TF , et al
      . Computerized Adaptive Testing Using the PROMIS Physical Function Item Bank Reduces Test Burden With Less Ceiling Effects Compared With the Short Musculoskeletal Function Assessment in Orthopaedic Trauma Patients. J Orthop Trauma 2014;28:439–43.doi:10.1097/BOT.0000000000000059
      OpenUrlCrossRefPubMedWeb of Science
    38. 38.↵
      2. Hung M ,
      3. Clegg DO ,
      4. Greene T , et al
      . Evaluation of the PROMIS physical function item bank in orthopaedic patients. J Orthop Res 2011;29:947–53.doi:10.1002/jor.21308
      OpenUrlCrossRefPubMed
    39. 39.↵
      2. Hung M ,
      3. Franklin JD ,
      4. Hon SD , et al
      . Time for a paradigm shift with computerized adaptive testing of general physical function outcomes measurements. Foot Ankle Int 2014;35:1–7.doi:10.1177/1071100713507905
      OpenUrlCrossRefPubMed
    40. 40.↵
      2. Hung M ,
      3. Baumhauer JF ,
      4. Latt LD , et al
      . Validation of PROMIS ® Physical Function computerized adaptive tests for orthopaedic foot and ankle outcome research. Clin Orthop Relat Res 2013;471:3466–74.doi:10.1007/s11999-013-3097-1
      OpenUrlCrossRefPubMed
    41. 41.↵
      2. Baumhauer JF ,
      3. Bozic KJ
      . Value-based Healthcare: Patient-reported Outcomes in Clinical Decision Making. Clin Orthop Relat Res 2016;474:1375–8.doi:10.1007/s11999-016-4813-4
      OpenUrl
    42. 42.↵
      2. Brodke DS ,
      3. Goz V ,
      4. Voss MW , et al
      . PROMIS PF CAT Outperforms the ODI and SF-36 Physical Function Domain in Spine Patients. Spine 2017;42:921–9.doi:10.1097/BRS.0000000000001965
      OpenUrl
    43. 43.↵
      2. Ho B ,
      3. Houck JR ,
      4. Flemister AS , et al
      . Preoperative PROMIS Scores Predict Postoperative Success in Foot and Ankle Patients. Foot Ankle Int 2016;37:911–8.doi:10.1177/1071100716665113
      OpenUrlCrossRefPubMed
    44. 44.↵
      2. Hunt KJ ,
      3. Alexander I ,
      4. Baumhauer J , et al
      . The Orthopaedic Foot and Ankle Outcomes Research (OFAR) network: feasibility of a multicenter network for patient outcomes assessment in foot and ankle. Foot Ankle Int 2014;35:847–54.doi:10.1177/1071100714544157
      OpenUrlCrossRefPubMed
    45. 45.↵
      2. Hung M ,
      3. Baumhauer JF ,
      4. Brodsky JW , et al
      . Psychometric Comparison of the PROMIS Physical Function CAT With the FAAM and FFI for Measuring Patient-Reported Outcomes. Foot Ankle Int 2014;35:592–9.doi:10.1177/1071100714528492
      OpenUrlCrossRefPubMed

    Footnotes

    • Contributors AJ: conception and design of the study/survey, data collection and interpretation, manuscript drafting and revision, approval of final manuscript. CK and VK: data analysis and interpretation, revision and approval of final manuscript. AB: data interpretation, manuscript drafting and revision, approval of final manuscript. GG: data interpretation, manuscript revision and final approval.

    • Funding This work was financially supported by the AO Foundation via AOTrauma and AO Clinical Investigation and Documentation (AOCID).

    • Competing interests We received financial support from AOCID and AOTrauma for the submitted work; AJ, CK, AB and VK are employees of AOCID; no other relationships or activities that could appear to have influenced the submitted work.

    • Patient consent Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement If additional data are of interest for the reader, please contact the corresponding author.