Article Text
PDF
Abstract
Background In Brazil, as in most countries nowadays, there is a pursuit for healthcare quality improvement and sustainability in public and private systems. Healthcare professionals’ perceptions, knowledge and attitudes determine evidence-based practice (EBP), which remain uncertain among Brazilian practitioners. A standardised national instrument whose wide use will identify gaps and flaws in establishing an EBP could contribute to an effective resources allocation from health professionals willing to use an EBP.
Objectives To present a study protocol on the development and validation of an instrument to measure Brazilian healthcare professionals’ behaviour, skills, self-efficacy, knowledge and attitudes towards EBP.
Methods This is a validation study with Brazilian healthcare professionals to develop a valid and reliable questionnaire, including selection of domains and formulation of questions. Construct and content validity will be assess by a panel of experts, with data collection and analysis following a Delphi-like methodology. Further, a pilot survey will be accomplished with a representative sample of different healthcare professionals from all main Brazilian regions. An exploratory factor analysis and a confirmatory factor analysis will be conducted afterwards. The ratio of χ2 and df (χ2/df), comparative fit index, goodness of fit index and root mean square error of approximation will be used for assessing the model fit. In addition, the reliability of the instrument will be estimated by test–retest reproducibility and Cronbach’s alpha coefficient (α).
Ethics and dissemination This study has received ethical approval from the Pharmaceutical Sciences Faculty of the São Paulo State University (1.425.808). The use among a wide national sample is expected to promote an extensive view of evidence-based decision-making, identifying the knowledge gaps in this area. Study findings will be circulated to healthcare professionals and scientists in the field through the publication in peer-reviewed journals and conference presentations.
- evidence based practice
- pharmacists
- nurses
- dentists
- physicians
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Understanding how evidence-based practice (EBP) is perceived and implemented across healthcare professionals and practice settings can identify EBP gaps and weaknesses, helping to predict educational needs and to optimise the investments on required resources.
This is the first study in Brazil to attempt to develop a validated instrument able to assess healthcare professionals’ attitudes, behaviours, self-efficacy, knowledge and skills in EBP.
This study is intended to overcome limitations described in other tools to evaluate EBP barriers, knowledge and skills.
The cross-sectional design of this study may lead us to observe an association, but not its direction over time.
Introduction
The evidence-based practice (EBP) is an essential component of good quality, patient-centred healthcare, characterised by the use of the best current scientific evidence in decision-making in patient care.1 The evidence should be provided by a systematic investigation, considering the patient’s peculiarities and expectations, aiming at safe and scientifically based care with the best possible outcome.2 3
The term was first used in medicine, namely evidence-based medicine (EBM) at McMaster University in the 1980s,4 and subsequently it was used in other health disciplines, such as in EBP.
In general, EBP principles are well accepted, although not necessarily incorporated as an integral part in clinical routine by physicians, nurses, pharmacists, dentists and healthcare management teams.5–9 Systematic reviews10–12 describe a gap between scientific knowledge generated through research and its use in professional practice. They also identify many barriers related to patient, professional, organisational and system gaps, which hinder the implementation of health EBP.
Five major themes emerge on barriers and facilitators for doctors’ use of EBP: individual mindset, professional group norms, EBP competencies, balance between confidence and critical reflection and managerial collaboration.13 In addition, EBM has not resolved the problems it set out to address (especially evidence biases and the hidden hand of vested interests), which have become subtler and harder to detect.14 15
Understanding how EBP is perceived and implemented across healthcare professionals can identify educational needs and outcomes, and predict where new research evidence is more necessary to be implemented. In addition, one can identify gaps and weaknesses with real investment needs.16
Several tools are proposed to measure behaviour and professional attitudes as for EBP practices.17–24 A systematic review25 has identified 104 instruments on EBP. Nevertheless, few authors provide detail on how the questionnaires were developed and validated. Others studies25 26 show that the scales developed to evaluate EPB practices either do not include all need domains (perceptions, attitudes, knowledge, skills, self-confidence, behaviour) or have not been validated.
It is important that new questionnaires are based on ‘implementation science’. Implementation science is the field of study of methods to promote adoption and integration of EBPs and policies into routine care. To complement the conceptual frameworks guide implementation and the selection of implementation strategies, ease identifying determinants of implementation and enrich research largely.27
This study aims to propose a study protocol addressing the development, validation and administration of an instrument for determining healthcare professionals’ behaviour, skills, self-efficacy, knowledge and attitude related to EBP in Brazil.
Methods
Study design
This is a protocol of a cross-sectional study aiming to elaborate and validate a questionnaire to measure Brazilian healthcare professionals’ perceptions, knowledge and attitude related to EBP.
This study protocol has been approved and will be carried out from April 2018 to August 2018.
We will conduct a literature review to obtain sources of possible methods already used to address the study aim and then we will start the elaboration of the questionnaire. The elaboration and validation of the questionnaire will be based on validate instruments found in the literature review. To develop a robust questionnaire, psychometric characteristics will be assessed by content validity, construct, criterion and discriminant, reliability and reproducibility.
Development and validation of the questionnaire
Elaboration of instrument questions
We will draw on the conceptual EBP framework (ie, knowledge, behaviour and attitude) proposed by ‘Classification Rubric for Evidence-Based Practice Assessment Tools in Education’ that was elaborated taking into account 104 tools from systematic reviews.26 ,25 We also will use the existing scales in EBP17–23 to garner candidate items and to group them into domains pointed by mentioned systematic review (EBP knowledge, attitude towards EBP, application/use of EBP and practitioners’ EBP behaviour in the clinical setting). At the end we will use Consolidated Framework for Implementation Research (CFIR), CFIR’s theory-based constructs and mechanisms can be used to help explain whether an implementation may or may not succeed. Furthermore, it can be used to identify potential barriers and facilitators if used before or during an implementation, with the EBP.28
If necessary, we will rephrase items in the context of EBP; in order to prevent socially desirable responses, we will phrase them neutrally.
Content validity
Content validity assessment refers to the instrument judgement, if the instrument truly comprehends the different aspects of its object and, at the same time, does not involve elements that could be related to other objects. It is a result of several examiners’ analyses (examiner-judges, panel board members), who analyse the items representation regarding content areas and the relevance of the objectives to be measured. We will use a panel of experts through a consensus technique, according to simplified Delphi’s method.29
The following will be used as inclusion criteria for selecting experts: (1) their curriculum in Lattes Platform (www.cnpq.br/lattes), using the advanced search with the keywords ‘evidence-based health practices’; and (2) professionals with experience in EBP, who will be indicated by the group of panellists that participate as examiner-judges in the content validity stage.
During the Delphi rounds, we will invite panel board members to comment on grammar and phrasing to improve uniform interpretation of items and prevent socially desirable responses, and we will ask them whether they would like to add items or dimensions.
The examiner-judges will evaluate the content and will make suggestions regarding:
Theoretical dimension: the adequacy of each item to the studied theory, that is, the adequacy of selected domains to verifying knowledge of EBP.
Theoretical relevance: the rate of association between the item and the theory.
Clarity: whether the items reported were written in such a way that the concept is comprehensible and whether they express adequately what they are supposed to measure.
Relevance or representativeness: whether items reflect the concepts involved, and whether they are relevant and adequate to reach the goals proposed.
This stage complements and/or helps to decide whether to remove/maintain instrument items. This decision can only be made after confirmatory factor analysis (CFA), that is, either when items with low factor weight are present or when there is difficulty in adjusting the model to the sample (which is verified after assessment of all psychometric properties).
Evaluation of psychometric characteristics
This step is a cross-sectional study, with a non-probabilistic convenience sampling design.
Participants
The estimated minimum sample size was based on the requirement of 10 subjects per model parameter.30
The sample will be composed of physicians, nurses, dentists and pharmacists working in Brazil’s public health sector (Unified HealthCare System—SUS). In 2016, the government database registered 240 750 physicians; 182 861 nurses, 58 421 dentists and 20 593 pharmacists. Thus, we chose to work with a representative sample bigger than that recommended for the statistical analysis.
Considering a 30% response rate, we estimate a sample size of 1270 respondents needed to answer one of our questions (percentage of prior contact, familiarity with EBP), with 5% precision. To obtain this precision, we dichotomised the first item of the survey (being favourable or not to EBP) assuming maximum variability (50% of responses favourable to EBP). A CI of 95% will be applied to the percentage of favourable responses.
Inclusion criteria
Clinical health professionals currently working in the public system.
Exclusion criteria
Professionals on leave from work for limited or unlimited time during the period of application of the questionnaire, or retired professionals.
Survey participants will be gathered from the National Register of Health Establishments database (CNES), which hosts with free access to data all public health institutions of Brazil.
Queries on CNES can be performed at http://cnes.datasus.gov.br/ filtering by geographic location (ie, State and Municipality) and type of establishment. It also provides the name, role, workload and employment contract of each healthcare professional.
The research sample will be selected randomly in a central computer considering some stratifications, for example, type of professional, geography, settings, etc. We will recruit potential participants through email with an invitation letter containing a link to the web survey. Professionals without email address available in CNES will be contacted by phone or fax at their workplace and will be sent a physical survey by postal mail to their work addresses.
Data analysis
Psychometric sensitivity
The summary and shape measures of the questionnaire items distribution will be used to estimate their psychometric sensitivity. Items with a skewness (Sk) greater than 3 and kurtosis (Ku) greater than 7 in absolute values are considered to have psychometric sensitivity issues.31 The diagnosis of multivariate outliers is to be performed by computing the Mahalanobis distance.31
Construct validity
The factorial and discriminant criterion validity will assess the construct validity of the instruments.
Factorial validity
Two separate factor analytic procedures will be conducted. First, the sample will divide by randomly selecting approximately 50% of cases from within each programme and assigning cases to either an exploratory or confirmatory analysis group.
Exploratory factor analyses will be conducted by using principal axis factoring in order to partition systematic and error variance in the solution.31 32 Promax oblique rotation will be used, allowing for factor intercorrelations. To promote simple structure, items are to be retained on a factor if they load at least 0.30 on the primary factor and less than 0.30 on all other factors.30 Item– total correlations and scale reliabilities will also be used to assess scale structure.
Second, a CFA will be conducted on the other half of the sample. CFA allows the determination of the educational aspects that explain the instrument variability. In other words, it aims to show that the instrument is capable of measuring its intended object. The analysis of the psychometric qualities of the scales will be carried out following Maroco’s 16 guidelines. Thus, the analysis of the construct validity will be made by a CFA. The goodness of fit measures used are χ2/df, comparative fit index (CFI), goodness of fit index (GFI) and root mean square error of approximation (RMSEA) indices. We will consider that the model is valid when the factorial validity χ2/df is between 1 and 2, CFI and GFI >0.9 and RMSEA <0.08, according to Maroco’s criteria.33
Discriminant criterion validity
Discriminant criterion validity consists of checking if the instrument shows that two measures that are not supposed to be related are in fact, unrelated. Criterion validity is statistically estimated and the test is considered valid if the ratio between the test score (X) and the criterion variable score (Y) is high.33
At this stage, experts in EBP comprising a new sample of participants will be included and selected according to the same criteria already used before. We will compare the result of the instrument applied to the new expert panel group to that of the health professional’s group and this will be used as external criterion.
The answers of the groups will be compared using Student’s t-test for independent samples. This analysis allows for verifying the discriminatory capacity of the instrument between both groups.34
Reliability
The reliability of an instrument used for data collection is its coherence, determined by the constancy of the results. A reliable (stable) measure is consistent and precise because it provides a constant measurement of the variable.34
To estimate the reliability, both the internal consistency and stability will be evaluated.
We will explore internal consistency, that is, the reliability will be estimated from the internal consistency, by using standardised alpha Cronbach coefficient (α), where Cronbach ɑ of 0.7 to 0.8 is considered satisfactory, 0.8 to 0.9 is good and 0.9 is excellent.35
The stability will be assessed using the test–retest method by verifying the intraclass correlation coefficient. Values above 0.70 will be considered satisfactory, which suggests that the items measure the same way as the constructs, and are therefore appropriate. The stability evaluates the consistency of repeated measurements.33
Two scores from each of the volunteers are necessary to assess the reproducibility of an instrument by the test–retest method. The research instrument will be applied to the same experts, selected for the discriminant validity. They will answer the same instrument at two different moments with a mean difference of 7 days.
Patient and public involvement
Patients and public involvement was not sought at protocol formulation stage, but will be so in the implementation stage and reported acoordingly.
Discussion
Little research has been focused on comparing the use of EBP among different healthcare professions. For EBP to be fully implemented, it is essential to clarify possible differences among professions. No studies were found that perform the cultural adaptation of the instruments that evaluate EBP in Brazil, or that estimate the psychometric qualities of these instruments.
In this sense, it is essential to use reliable and valid instruments or to properly develop them when they are not available. To consider that the characteristics of a given instrument are related to a sample, the assessment of its reliability and validity ought to be performed before presenting the results. Any psychometric study that involves the use of scales needs to undergo such evaluation, as this increases the quality of the data to be collected. However, despite the relevance of these steps, they have not yet been widespread in all fields, especially in EBP.25 26 36
The data collected will provide critical and useful evidence to inform strategies to scale up and qualify EBP in many settings.
Some items of this new EBP instrument will target the identification of facilitators and barriers for the adoption of the new paradigm of evidence-based clinical practice and to improve patient care in relation to professional fulfilment. These data may be useful to inform the design strategies to improve and assess.36
After completing the necessary steps to make the questionnaire available to the Brazilian context, it is suggested that it may be a useful tool for assessing educational strategies and health institutions concerned with the quality of care. In addition, the use of the instrument can be a measure of personal evaluation of practitioners about their practice, awakening critical thinking about the quality of their practice.
The study, conducted with healthcare professionals from different areas of expertise, allows for relevant and comprehensive results. Results will contribute to a wider view of EBP reality in Brazil.
Study limitations
Although the study follows all the recommendations for observational studies considered in the Strengthening the Reporting of Observational Studies in Epidemiology statement, there is the possibility of confounding factors that have not been considered. The cross-sectional design of this study may lead us to observe an association, but not its direction over time.
Ethics and dissemination
The present study has been approved. All study participants will sign an informed consent form agreeing to participate in the study.
We will use a variety of methods to ensure that our work will achieve maximum visibility. The publishing of our study protocol provides an important first step in this direction. In this paper, we have sought to offer a comprehensive overview of the relevant literature, while underlining current research gaps that motivated the design and implementation of the study.
Similarly, given their applicability and implications for the general population, the study results will be disseminated in research meetings and in at least three articles published in scientific journals.
The results of this study will be published in a peer-reviewed journal and brief reports of the findings will be disseminated directly to the class councils, public health managers, via Conass (National Council of Health Secretaries) and in the places where the professionals are to be selected. The results will help decision-makers and managers lower the barriers to the implementation of EBP.
References
Footnotes
Contributors All authors participated in the preparation of the manuscript and agreed to the submitted version of the paper. LCL and ASM had the original idea and they developed the study protocol. ASM, LCL, AC and NMS drafted the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent Obtained.
Ethics approval Faculty of Pharmaceutical Sciences, São Paulo State University “Júlio de Mesquita Filho”.
Provenance and peer review Not commissioned; externally peer reviewed.