Article Text
PDF
Abstract
Objectives Evaluating an existing suite of health system performance (HSP) indicators for continued reporting using a systematic criteria-based assessment and national consensus conference.
Design Modified Delphi approach with technical and leadership groups, an online survey of stakeholders and convening a national consensus conference.
Setting A national health information steward, the Canadian Institute for Health Information (CIHI).
Participants A total of 73 participants, comprised 61 conference attendants/stakeholders from across Canada and 12 national health information steward staff.
Primary and secondary outcome measures Indicator dispositions of retention, additional stakeholder consultation, further redevelopment or retirement.
Results 4 dimensions (usability, importance, scientific soundness and feasibility) typically used to select measures for reporting were expanded to 18 criteria grouped under the 4 dimensions through a process of research and testing. Definitions for each criterion were developed and piloted. Once the definitions were established, 56 of CIHI's publicly reported HSP indicators were evaluated against the criteria using modified Delphi approaches. Of the 56 HSP indicators evaluated, 9 measures were ratified for retirement, 7 were identified for additional consultation and 3 for further research and development. A pre-Consensus Conference survey soliciting feedback from stakeholders on indicator recommendations received 48 responses (response rate of 79%).
Conclusions A systematic evaluation of HSP indicators informed the development of objective recommendations for continued reporting. The evaluation was a fruitful exercise to identify technical considerations for calculating indicators, furthering our understanding of how measures are used by stakeholders, as well as harmonising actions that could be taken to ensure relevancy, reduce indicator chaos and build consensus with stakeholders.
- Consensus methods
- Health system performance
- Health indicators
- Evaluation
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Footnotes
Contributors OF participated in the design of the evaluation, performed data analysis, facilitated internal Delphi sessions and drafted the manuscript. KL participated in the design of the evaluation, facilitated internal Delphi sessions, organised the national consensus conference and drafted the manuscript. YG participated in the design of the evaluation, facilitated internal Delphi sessions and drafted the manuscript.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Ethics approval Ethics approval was not required as the study did not utilise any sensitive data.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.