Article Text
PDF
Abstract
Introduction Australia’s ageing population is driving an increased demand for residential aged care services, yet concerns about the quality and safety of such care remain. The recent Royal Commission into Aged Care Quality and Safety identified various limitations relating to leadership within these services. While some competency frameworks exist globally, there is a need for sector-specific leadership competencies in the Australian residential aged care setting to promote and protect quality of care.
Methods and analysis This study uses the Delphi technique to establish the content validity of a national leadership framework (RCSM-QF) for promoting and protecting the quality of residential aged care in Australia. Participants will be identifiable experts through current employment within, policy development for or research with the aged care sector. The survey will ask participants to rate the relevance, importance and clarity of RCSM-QF items and their corresponding descriptions and seek suggestions for revisions or additional items. Content validity will be assessed using the Content Validity Index, with items meeting specific criteria retained, revised, or removed.
Ethics and Dissemination Ethics approval has been sought via the James Cook University Human Research Ethics Committee (HREC) to ensure the well-being and convenience of participants while mitigating potential recruitment challenges. Data will be prepared for submission to an appropriate peer-reviewed journal and presentation at relevant academic conferences.
- Aged
- HEALTH SERVICES ADMINISTRATION & MANAGEMENT
- Quality in health care
- Health Services for the Aged
Data availability statement
Previous datasets used and analysed to form the study protocol are available from the corresponding author upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
Purposive sampling will be used as a targeted recruitment method for interviewing participants from peak bodies, primary health networks and researchers, allowing for diverse expert perspectives within the Australian aged care sector.
The Delphi method is a popular tool for framework validation in research, allowing for structured input from Australian aged care experts to refine and validate the preliminary framework.
The Delphi process is based on subjective opinions rather than empirical evidence, potentially affecting the validity and reliability of the framework validation process.
To address limitations in the Delphi process, future phases of the research may require qualitative methods to allow for real-time feedback and in-depth insights from a wider range of industry experts.
The anonymous nature of the Delphi process limits in-depth discussions, debates, and the exchange of ideas, potentially restricting the exploration of alternative perspectives regarding the leadership competencies influencing the quality of Australian residential aged care services.
Introduction
Australia’s population is ageing, with the proportion of people aged 65 years or over projected to increase from 16% (2018) to 23% in 2066.1 In line with national ageing trends, the demand for Australian residential aged care services is also increasing,2 and there have been ongoing concerns about the quality and safety of that care.3 Indeed, the recent Royal Commission into Aged Care Quality and Safety described a ‘cruel and harmful’ national aged care system comprising services that were ‘neglectful’ and ‘woefully inadequate’.4 The leadership of these services, including ‘ground-level’ residential aged care senior management teams, was described as ‘lacking’, and leadership competencies for promoting quality of care were found to be ‘poorly defined’.4 The lack of any sector-specific professional development or leadership framework to guide the acquisition of these required skills within the Australian residential aged-care setting is a concern.5
Existing studies have generated some knowledge regarding leadership requirements, and some competency frameworks exist for aged care services globally, although evidence gaps remain. Seminal work in Australian aged care leadership was conducted by Jeon et al.6 in validating a clinical leadership framework, the Aged care Clinical Leadership Qualities Framework, for middle (mainly clinical) managers in both community-based and residential aged care services.6 in 2014, Aged & Community Services Australia (ACSA) developed the Australian Aged Care Leadership Capability Framework. While this framework reflected an important step forward, its inclusion of different leadership levels (eg, frontline, middle manager and senior managers) and multiple service types (residential, acute and community-based) meant it was necessarily general in nature, with limited specificity concerning the multifaceted and increasingly demanding nature of residential aged care facilities.7 Furthermore, the ACSA framework describes leadership capabilities (statements of behaviours, skills and knowledge that affect an outcome) but not competencies (a measure or index of how well a person performs that capability) and does not explicitly link these to promoting quality care.
With an absence of competency-based frameworks specific to the Australian residential aged care setting, there is a clear need to describe and model the competencies required by leadership teams to provide effective leadership within this increasingly complex environment. A recent programme of work identified the knowledge, skills and abilities senior managers need to promote and protect quality residential aged care.8 Competencies were to form a preliminary leadership competency framework, The Residential Aged Care Senior Manager Quality Framework (RCSM-QF).8 The RCSM-QF comprises two key elements: personal qualities and leadership skills. Leadership skills are broken down into five domains, including (1) culture and environment, (2) stakeholder relations, (3) clinical and aged care expertise, (4) asset management and (5) disaster and change management (figure 1.) This original and empirically grounded competency framework synthesised Australian senior managers’ skills and personal qualities to promote and protect the quality of care in the residential aged care setting. Its formation drew on the experiences and strategic insights of senior managers themselves and Australian industry experts.8 These empirically derived leadership competencies were compared with those extracted from pre-existing senior management-relevant leadership frameworks, including the HLA Competency Directory,9 IPEC Core Competencies10 and Master Health Service Management Competency Framework11 to form a novel (though untested) competency framework.
The preliminary Residential Aged Care Senior Manager Quality Framework (RCSM-QF). Source: Authors own figure.
Although the formation of the preliminary RCSM-QF is a step in addressing the evidence gap relating to sector-specific leadership competencies and professional development requirements for senior managers to promote quality of care, it has not been applied or tested in the Australian residential aged care setting nor the competencies empirically validated. Therefore, the accuracy and usefulness of the RCSM-QF for describing and helping assess the leadership competencies required by senior managers across Australia are not yet confirmed.
This study aims to establish the content validity of the preliminary RCSM-QF within the Australian residential aged care context using a modified Delphi process. Once this validity is established, this programme of work could provide a practical tool to form a professional development infrastructure for current and aspiring Australian residential aged care senior managers who continue to operate within this increasingly complex environment.
Methods and analysis
Aim
To establish the content validity of a national leadership framework to promote and protect the quality of residential aged care in Australia.
Objectives
Rate the relevance, importance and clarity of RCSM-QF items (competencies OR personal qualities) and their descriptions using a 4-point Content Validity Index (CVI) scale.
Suggest RCSM-QF item and description scale revisions.
Suggest RCSM-QF domain name and domain definition revisions.
Suggest additional items (competencies or personal qualities) for the RCSM-QF.
Study design
Content validity refers to the extent to which a measurement tool, such as a test or assessment, accurately represents the specific content it is intended to measure.12 Content validity is an important aspect of validating a leadership competency.13 In the context of the current research, the method will assess whether the RCSM-QF accurately and comprehensively represents the key competencies required for effective leadership within the Australian residential aged care setting. We will evaluate content validity using the Delphi survey technique and a CVI. The Delphi technique is a widely used method for achieving consensus.14 It uses a series of questionnaires to collect information from participants in several iterations, or ‘rounds’. The starting point is an open questionnaire or a prederived list of questions.15 Following each round, each participant receives an individualised report of their responses to the group response.15 In subsequent rounds, participants can reassess their responses in light of this information.14 The process allows for a controlled debate and for consensus to build without necessitating group interaction, an advantage in the context of geographically dispersed and time-constrained experts. It also limits the time and resources required to plan and facilitate group interactions and the bias from dominant individuals within this consensus-building phase.14
Study setting
The current study will be completed with representatives who contribute to or advise regarding the delivery of aged care services in Australia. Examples of different ‘levels’ of aged care include: (1) entry-level community-based care at home; (2) higher levels of care at home (Home Care Packages Program), and, when living at home is not an option; (3) residential aged care.16 This study focuses specifically on the role of senior managers in providing quality care in the Australian residential aged care setting. Residential aged care provides healthcare services and accommodation for older people who are unable to continue living independently in their own homes.17
In Australia, residential aged care providers can span a range of different sectors, including religious, charitable, community, for-profit and government organisations.17 Typical services may include accommodation, personal care assistance, clinical care and a range of social care activities, including recreational activities and emotional support. Approximately 250 000 older Australians received permanent residential aged care at some time during the financial year 2021/2022.1
Participant recruitment
To be eligible for participation, panel participants will need to be self-identified or other-identified aged care experts through current employment within the aged care sector and have high-level knowledge and experience in aged care. Expertise may include clinical practice, management, service delivery, policy, research and education or combinations of the above. From previous work, this study will target five major groupings of expert representatives, including peak advocacy bodies, primary health network representatives, members of state and federal government, aged care researchers and residential aged care executives and governing board members spanning multiple organisation types (non-for-profit, for-profit, non-governmental organisation and government-operated).8 Purposive sampling will be used to ensure that rural, remote and metropolitan settings across Australia are represented on the panel. Purposive sampling will also allow the identification and selection of information-rich participants from the expert groupings with knowledge and experience working within the Australian aged care sector.18 Participant selection will be purposive, and aged care industry experts will be recognised as possessing specific knowledge of the health service needs of older persons in Australia and capable of reflecting critically on the link between senior manager leadership skills and quality residential aged care.
A list of eligible participants will be generated using a combination of investigators’ aged care industry experience and a comprehensive desk search. Participants will be emailed an invitation for involvement in round 1 and followed up by phone if a response has not been received in 2 weeks. Participants will provide electronic consent before commencing the questionnaire.
Data collection
Round 1: in reviewing, modifying and validating the RCSM-QF, two rounds of iterative consultation will be undertaken with the Delphi panel via email. In round 1, experts will be sent email invitations to participate. On clicking the survey link, participants will be redirected to an online platform where they will be asked to confirm their consent to participate and will rate each item and its response scale based on clarity, relevance and importance using a 4-point CVI scale where 1=not clear/relevant/important, 2=somewhat clear/relevant/important, 3=quite clear/relevant/important and 4=highly clear/relevant/important.19 Through open dialogue boxes, experts will also provide suggestions for item wording, domain name and domain definition revisions and propose additional items for any missing experiential aspects of care. Demographic questions will include gender, year of birth, highest educational qualification, place of work and current professional role. Experts will be given a 2-week window to complete the round 1 survey, after which the survey will be closed, and the results will be exported into Microsoft Excel. Reminder emails will be dispatched to participants on days 5 and 12 of the round 1 questionnaire if they have not taken part.
Round 2: the second round will commence 1 week after the conclusion of round 1. Experts will receive a second survey invitation via email, asking them to rate the revised items in terms of clarity, relevance and importance using the 4-point CVI scale and propose item revisions. A 2-week timeframe will be allotted for experts to complete the round 2 survey, following which the survey will be closed, and the results will be exported into Microsoft Excel. Reminder emails will be sent to participants on days 5 and 12 of the round 2 survey if they have not participated.
Data analysis
Content validity
The current study will incorporate the CVI as a verified approach for evaluating content validity.20 The CVI index comprises two computed components: the item-CVI (I-CVI) and the scale-level-CVI (S-CVI).20 To compute the I-CVI, the number of Delphi panel experts assigning a ‘very relevant’ rating to each item is divided by the total number of experts, resulting in values ranging from 0 to 1. An I-CVI surpassing 0.79 deems the item relevant, while values falling between 0.70 and 0.79 indicate the need for item revisions; I-CVI values below 0.70 warrant item elimination.20 Likewise, the S-CVI is determined based on the count of items within a tool that attains a ‘very relevant’ rating.20 To measure the S-CVI, the Universal Agreement (UA) among experts (S-CVI/UA) will be used. The S-CVI/UA involves summing all items with an I-CVI of 1 and dividing by the total number of items, with the ab S-CVI/UA of ≥0.8 denoting excellent content validity.21
Round 1: the demographic and Delphi survey data will be analysed descriptively using Microsoft Excel. Expert responses to the item-level CVI (I-CVI) scales will be binary coded as 0 for ‘not or somewhat relevant/important/clear’ and 1 for ‘quite or highly relevant/important/clear.’ An I-CVI score will then be computed for each item, representing the proportion of experts scoring 1 out of the total number of experts in the round 1 sample.19 Items meeting a score of ≥0.80 for each of relevance, importance and clarity (without revision suggestions) will be retained for the final version of the RCSM—QF. Items achieving scores of ≥0.80 for each of relevance, importance and clarity (with revision suggestions), or ≥0.80 for relevance and importance but <0.80 for clarity, will undergo revision by the research team based on expert feedback and will subsequently be included in the round 2 survey. Items obtaining scores of <0.80 for each of relevance, importance and clarity will be removed from the RCSM-QF. The research team will also consider suggestions provided by experts concerning modifications to domain names, domain definitions and missing items.
Round 2: the round 2 questionnaire results analysis will adhere to the same methodology as in round 1. The research team will thoroughly review additional suggestions for item revisions before implementing further RCSM-QF modifications. A scale-level CVI (S-CVI) score will also be calculated by averaging the I-CVI scores for all items included in the final RCSM-QF.20 Delphi studies are typically carried out in two to three rounds with a deliberately selected panel of experts.14 In the current study, a third round of consultation will be administered if few items achieve scores of ≥0.80 for relevance, importance and clarity. In this third round, data collection and analysis processes will replicate those of the previous rounds.
Ethics and dissemination
The James Cook University Human Research Ethics Committee (HREC) approved this study’s ethics on 14 November 2023. The current project has been deemed ‘negligible risk’ by the HREC, as there is no foreseeable risk of harm or discomfort to the participants other than the inconvenience of completing the questionnaire. A potential risk of this study is difficulties in recruiting the required numbers for this research, which might pose a risk to study completion. If this is the case, alternative recruitment methods will be considered, including broadening networks to include other professional roles and organisations operating within the Australian aged care sector.
Data will be prepared for submission to an appropriate peer-reviewed journal and presentation at relevant academic conferences, including the Australian Association of Gerontology Conference (2024). In addition to the International Journal of Healthcare Management, where a large body of literature regarding aged care management and quality of care is published, several additional avenues have been identified to add variety to the audience accessing the project’s findings. Given that this research focuses on senior managers in the aged care setting, targeted journals include the Australasian Journal on Ageing, Journal of Ageing and Health and BMC Health Services Research or BMJ Open.
Patient and public involvement
This research project was designed without direct patient or public involvement in several key aspects, including determining research priorities, defining research questions, selecting outcome measures and contributing to study design. It is recognised that including patient and public perspectives can significantly enhance the relevance, quality and applicability of research outcomes, and their absence in this study might have implications for the comprehensiveness and relevance of our findings.
Discussion
Senior managers are central in promoting and protecting quality of care in clinically and administratively complex residential aged care service settings.8 Yet globally and in Australia, there remain significant gaps in knowledge regarding the specific competencies and skills required of this leadership cohort.8 This study aims to establish the validity of a novel leadership competency framework, which could provide a practical tool for national regulatory and professional bodies by defining and describing the specific skills, behaviours, knowledge and experience needed by aspiring and current senior managers. It may also inform the development of quality indicators to inform competency-based performance evaluations of senior managers within their current roles.
Evidence-based leadership competency frameworks provide a standardised and consistent approach to leadership development across multiple health settings.22 Once validated for acceptability and applicability, the RCSM-QF competencies may assist residential aged care organisations in establishing a consistent promotion criterion that incorporates demonstrated excellence by senior managers who consistently lead high-quality healthcare operations within their respective organisations. In doing so, it could provide a valuable tool for self-reflection to identify knowledge and skill gaps and guide future training and other career progression opportunities.
The RCSM-QF also offers a set of skills and personal qualities that could inform the development of future courses or qualifications to develop the competencies required by aspiring managers to promote quality of care within their respective organisations and across the broader aged care sector. This focus on quality and continuous improvement may drive organisational excellence for the quality of care, enhance resident health outcomes and foster a culture of accountability and innovation within Australian residential aged care organisations.23 This work thus not only addresses key gaps in the literature and evidence base regarding senior management competencies but represents an essential and timely first step in responding to Royal Commission recommendations to strengthen leadership in the sector.
As with a majority of studies, the design of the current study is subjected to limitations. First, purposive sampling was used to recruit interview participants from three categories of experts (peak bodies, primary health networks and researchers); however, it is possible that not all participants will participate due to scheduling or other issues. It is expected, therefore, that the final sample of experts may introduce an element of bias and not always represent the diverse range of perspectives across multiple professional roles within the Australian aged care sector. For example, suppose a majority of study participants were provider advocates whose primary focus is to support the viability and sustainability of aged care service providers. In that case, experts are potentially less likely to consider the resident experience and personalised healthcare needs. Conversely, consumer advocates play an important role in advocating for the older person and speaking on behalf of that individuals in a way that best represents their interests. With an intense focus on the individualised healthcare needs of older Australians, consumer advocates may have less understanding of the structural elements that adversely influence RAC quality and the leadership competencies required.
While the modified Delphi process is a popular design used for research involving framework validation, some potential limitations within the proposed research are evident.24 First, the Delphi process is largely based on expert opinions rather than empirical evidence, and while these opinions can be valuable, they may not always align with objective facts or data.25 This can limit the validity and reliability of the framework validation process, particularly if more empirical evidence is needed to support the experts’ judgements. A further potential limitation of the study design is that experts provide their input individually and anonymously.26 The Delphi process, therefore, lacks direct interaction among experts, which can restrict the opportunity for in-depth discussions, debates and exchanging ideas.26 Consequently, the method may not capture the full complexity of the research problem or allow for exploring alternative perspectives.25 Depending on the findings from this study, future phases of this programme of work may involve qualitative methods to address this limitation. These sessions would involve Australian aged care industry experts with varying opinions and perceptions to allow real-time feedback, in-depth insights and rich qualitative data regarding the applicability of the RCSM-QF implementation within the Australian residential aged care setting.27
Data availability statement
Previous datasets used and analysed to form the study protocol are available from the corresponding author upon reasonable request.
Ethics statements
Patient consent for publication
Footnotes
X @globalstopp
Contributors ND conducted the primary manuscript draft. ND, SMT and OA completed subsequent manuscript revisions. All authors read and approved the final manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer-reviewed.