Objective Verbal face-to-face feedback on clinical task performance is a fundamental component of health professions education. Experts argue that feedback is critical for performance improvement, but the evidence is limited. The aim of this systematic review was to investigate the effect of face-to-face verbal feedback from a health professional, compared with alternative or no feedback, on the objective workplace task performance of another health professional.
Design Systematic review and meta-analysis.
Methods We searched the full holdings of Ovid MEDLINE, CENTRAL, Embase, CINAHL and PsycINFO up to 1 February 2019 and searched references of included studies. Two authors independently undertook study selection, data extraction and quality appraisal. Studies were included if they were randomised controlled trials investigating the effect of feedback, in which health professionals were randomised to individual verbal face-to-face feedback compared with no feedback or alternative feedback and available as full-text publications in English. The certainty of evidence was assessed using the Grading of Recommendations, Assessment, Development and Evaluations approach. For feedback compared with no feedback, outcome data from included studies were pooled using a random effects model.
Results In total, 26 trials met the inclusion criteria, involving 2307 participants. For the effect of verbal face-to-face feedback on performance compared with no feedback, when studies at high risk of bias were excluded, eight studies involving 392 health professionals were included in a meta-analysis: the standardised mean difference (SMD) was 0.7 (95% CI 0.37 to 1.03; p<0.001) in favour of feedback. The calculated SMD prediction interval was −0.06 to 1.46. For feedback compared with alternative feedback, studies could not be pooled due to substantial design and intervention heterogeneity. All included studies were summarised, and key factors likely to influence performance were identified including components within feedback interventions, instruction and practice opportunities.
Conclusions Verbal face-to-face feedback in the health professions may result in a moderate to large improvement in workplace task performance, compared with no feedback. However, the quality of evidence was low, primarily due to risk of bias and publication bias. Further research is needed. In particular, we found a lack of high-quality trials that clearly reported key components likely to influence performance.
Trial registration number CRD42017081796.
- medical education & training
- health professions education
- formative feedback
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
This systematic review is the first to investigate the impact of face-to-face verbal feedback from a health professional, compared with alternative or no feedback, on the objective workplace task performance of another health professional.
The meta-analysis of verbal feedback compared with no feedback is the first to provide an estimate of the likely benefit of verbal feedback on performance of a workplace task in the health professions.
The quality of evidence was low, primarily due to risk of bias in study design or conduct and publication bias.
Health professions education is embedded in clinical practice for both students and qualified staff as they continue learning and training.1 Face-to-face verbal feedback focused on the performance of a clinical task involving an educator (senior clinician or peer) and a learner (any clinician) plays a crucial role in workplace learning, particularly within competency-based education and programmatic assessment models.2–5
Multiple reviews on feedback in health professional education have been published and include recommendations for effective practice.6–9 Feedback can occur in various forms, including verbal, written or automated (eg, from a simulator or within an online learning module). The unique potential benefits of face-to-face verbal feedback are the opportunities for: (1) real-time interaction, to which the learner and educator bring their different perspectives, priorities and ideas to coconstruct insights and strategies for improvement and (2) interpersonal connection, through which an educator can foster a learner’s feelings of support, self-efficacy and motivation to improve, which are important catalysts in the learning process.8 10–13
There is widespread acceptance that feedback has an important role in maximising learning and achievement.6 14–16 Ende17 said, ‘Without feedback, mistakes go uncorrected, good performance is not reinforced, and clinical competence is achieved empirically or not at all’. However, there is little evidence to support this view that feedback enhances health professionals’ performance. Indeed, a recent scoping review on feedback identified the need for systematic reviews to support evidence-based recommendations.7
The current strongest evidence relates to two systematic reviews that investigated the impact of audit and feedback. In 2006, Veloski et al18 published a BEME systematic review in which almost 75% of included studies reported that audit and feedback could improve an individual physician’s clinical performance, particularly when sustained and from an authoritative source. Feedback was defined as ‘summary information on clinical performance over a defined time period’. They included any empirical study (not just randomised controlled trials) and all types of physicians (most were primary care physicians). The majority of outcomes were clinical processes (such as test ordering), and the most common data sources were medical records and billing records (none involved direct observation of performance).
In 2012, Ivers et al19 updated a Cochrane review and meta-analysis that reported an increase in compliance with desired practice following audit and feedback, compared with usual care. The review included various health professionals (predominantly doctors), the unit of allocation for interventions ranged from individuals to health services and the performance outcomes reported were clinical practice processes, such as the number or quality of prescriptions or tests. The authors argued that although the median risk difference (RD) in favour of feedback was small at 4.3% (IQR 0.5%–16%), the 3rd quartile at 16% indicated that audit and feedback interventions could be much more effective. Using multivariable meta-regression, they identified that the effectiveness of audit and feedback increased when the source was a senior colleague or supervisor (RD 11%), the format involved both written and verbal components (RD 8%), the frequency was at least monthly (RD 7%), the aim was to reduce specific behaviour (RD 6%) and it included both explicit measurable targets and a specific action plan involving advice on how to improve, compared with performance information alone (RD 5%). In addition, two other factors were associated with a higher likelihood of improvement: a lower baseline performance and the type of behaviour being targeted, for example, prescribing (possibly perceived as ‘important’ and ‘straightforward’) had better outcomes than improving diabetes management (more ‘complex’) or test ordering (possibly perceived as ‘less important’).
We found no systematic review that investigated the impact of verbal face-to-face feedback on a health professional’s performance, the typical scenario in clinical practice.
Our research question was therefore:
‘What is known about the effect of face-to-face verbal feedback from a health professional, compared with alternative or no feedback, on the objective performance of an observable workplace task by another health professional?’
The primary aim of the review addressed this question. Secondary aims were to summarise interventions and outcomes reported in included studies.
This review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses’ (PRISMA) statement.20 The protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO).
Eligibility criteria for considering studies for this review
We included randomised controlled trials in which individual health professionals were randomised to feedback, compared with no feedback or alternative feedback. Reports had to be available as English full-text publications.
We included studies in which participants were health professional students or graduates from the disciplines of medicine, dentistry, nursing and midwifery, allied health, psychology, pharmacy, medical radiation practice, optometry, osteopathy or chiropracty.
All studies had to include at least one intervention involving verbal face-to-face feedback generated by a health professional, based on the observable performance of a workplace task performed by another health professional. A broad definition of feedback was permitted with a minimum requirement that it included information regarding learner performance. Studies were excluded if feedback was predetermined or provided only by a simulated patient or machine. Audit and feedback studies, where feedback was based on aggregated quality performance indicators (such as numbers of tests ordered or degree of compliance with quality practice standards) were excluded, as this was deemed to be distinctly different from a workplace task, such as suturing, that could be observed, objectively assessed and targeted for improvement with feedback. Two comparisons were evaluated: (1) verbal face-to-face feedback compared with no feedback and (2) verbal face-to-face feedback compared with alternative feedback.
Performance following feedback interventions had to be objectively assessed. To isolate the effects of feedback, other conditions had to be comparable for both groups. Studies were excluded if the report did not include point estimates of effects and measures of variability (or data from which these could be derived), unless these data could be obtained from the author.
Search methods for identification of studies
We developed the search strategy in collaboration with a senior medical librarian using MEDLINE subject headings. Key words were used, including synonyms, truncation, wildcard and proximity operators related to ‘feedback’ AND ‘health professional’ AND ‘performance’ AND ‘randomised controlled trial’ (see online supplementary appendix 1 for the full search strategy for Ovid MEDLINE). We translated this search strategy for other databases. The full holdings of Ovid MEDLINE (1946 to present with daily update), CENTRAL, Embase (1946 to present with daily update), CINAHL plus (1937 to present) and PsycINFO (1806 to present) were searched until 1 February 2019. We also searched the reference lists of systematic reviews and included studies.
Selection of studies
One review author (CEJ or MPW) screened titles to exclude clearly irrelevant reports. Two authors (CEJ and MPW) independently screened remaining abstracts to identify potential eligible studies, then independently assessed the full text. Decisions were compared using Covidence (online software designed by the Cochrane Collaboration to improve review efficiency via www.covidence.org), and disagreements were resolved through discussion, including a third review author (JLK).
One review author (CEJ) used a prepiloted standardised form to extract data from included studies, and another author (MPW or JLK) checked the data extracted were accurate. We resolved discrepancies through discussion. The following data were recorded: year of publication; study setting; funding sources; key details regarding participants, workplace task, feedback intervention and outcome measures; and information related to the risk of bias assessment. If data were missing, we contacted authors to request the information.
Assessment of risk of bias in included studies
The risk of bias was independently assessed by two authors (CEJ and JLK) for the selected performance outcome for individual studies, using Cochrane’s ‘risk of bias’ tool (Chapter 8, Cochrane Handbook for Systematic Reviews of Interventions).21 In particular, we used the following decision rules in assessing the risk of bias for specific individual domains. For ‘participant and research team blinding’: a participant receiving feedback or an educator giving feedback was deemed not to be blinded, even if they were deliberately not informed about the intervention or any differences between interventions. Nevertheless a ‘low risk’ rating was given if the outcome was not likely to be influenced by this lack of blinding, for example, if there were no changes to protocol or adherence that arose as a consequence of participant knowledge of group allocation.22 For ‘incomplete outcome data’: to be rated as ‘low risk’, studies were required to include outcome data on at least 85% of the participants enrolled in each group (as per PEDRO guidelines),23 and to provide participant numbers at the start and the number that dropped out during the study, from which group and the reasons.
The risk of bias was then summarised within each study across domains for the performance outcome, in accordance with the Cochrane ‘risk of bias’ assessment tool.
Measures of treatment effect
Outcomes from included studies were expressed using point estimates and measures of variability (eg, means (SD) or median (IQR). The effect was quantified using the standardised mean difference (SMD) to combine studies measuring the same outcome (task performance) using different measurement scales. When not reported, we estimated required data using available data or contacted study authors. If multiple outcomes were reported, we preferentially used the outcome that summarised multiple relevant task components, thereby providing a global, task-specific evaluation. If more than one reported outcome met this principle, we combined outcomes to provide a single metric using weighted averages of standardised scores.
We created and visually examined a funnel plot to explore reporting bias (Chapter 10, Cochrane Handbook).24 25
Data synthesis and assessment of heterogeneity
We pooled data from comparable studies for the comparison of feedback to no feedback on any measure of task performance and conducted analysis using random effects modelling in RevMan software (Review Manager Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014). The result of the random effects meta-analysis was presented as the SMD of the treatment effect with 95% CI, as the average effect across multiple studies and its error estimates.
As a sensitivity analysis, we conducted a meta-analysis excluding studies with a high risk of bias. Using this pooled data, we calculated a prediction interval, which describes the range of likely results for new individual studies.26
We rated the overall certainty of evidence for the outcome using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach (Chapter 12, Cochrane Handbook27 28 and GRADE guidelines),29 which considers within-study risk of bias, directness of evidence, heterogeneity, precision of effect estimates and risk of publication bias. Two authors independently rated the certainty of the evidence and resolved disagreements by discussion. We presented a summary of the evidence in a ‘Summary of Findings’ table.
Patient and public involvement
There was no involvement of patients or the public in any part of this research.
The search yielded 1238 articles after 409 duplicates were removed. Based on title or abstract, we excluded 1110 articles. We assessed the remaining 128 full-text articles for eligibility and found 26 randomised controlled trials that met all inclusion criteria. See figure 1 for PRISMA study flow diagram.
Comparison 1: the effect of verbal face-to-face feedback, compared with no feedback, on performance
Table 1 presents the characteristics of included studies in this comparison. Eleven randomised controlled trials investigated the effect of verbal face-to-face feedback compared with no feedback on the objective evaluation of a workplace task. Seven (64%) reports were published in the last 5 years since 2014. The studies were conducted in Europe (4),30–33 Canada (4),34–37 the USA (2)38 39 and Asia (1).40
There were 488 participants, including 196/366 (53.6%) men from seven studies that reported gender data.30 32–35 38 40 Participants included 290 (60%) medical students in four studies,30 33 37 38 60 (12%) dental students in one study32 and 138 (28%) doctors in six studies.31 34–36 39 40 The workplace tasks involved a discrete task such as surgical procedures, cardiopulmonary resuscitation (CPR) or teaching skills, which occurred in clinical practice in four studies34 35 39 40 and a simulation environment in seven studies (7/11, 64%).30–33 36–38 Differences in feedback interventions between included studies involved feedback source (expert or peer), timing (during task performance, directly afterwards or delayed), content (evaluative information only or additional corrective advice, performance video, simulator information or written report) and number of feedback episodes. In addition, there was variation between studies in provision of instruction and expert demonstration of the task, opportunities for practice and duration of feedback intervention (see ‘Included studies’ section in the supplementary material for more details, online supplementary appendix 2).
Risk of bias
The risk of bias graph is presented in figure 2, and the risk of bias summary is presented in figure 3. In summarising the risk of bias across domains within each study, two studies were rated ‘low risk’,34 36 six studies were rated ‘unclear’30 31 33 35 37 39 and three studies were ‘high risk’32 38 40 (see ‘Risk of bias’ section in the supplementary material for more details, online supplementary appendix 2).
The funnel plots are presented in figure 4: for all included studies (figure 4A) and after excluding studies at high risk of bias (figure 4B). Both funnel plots are asymmetrical, with a paucity of small studies with negative effect sizes that are less likely to be published, indicating some potential for publication bias.
A meta-analysis of the impact of verbal face-to-face feedback compared with no feedback on performance included 13 comparisons from the 11 studies, involving 488 participants. Two studies reported data that each enabled two comparisons: in one study, feedback provided during practice in one group and directly after practice in another were compared with no feedback37; in another study, feedback provided by an expert in one group and by a peer in another group31 were compared with no feedback. In the meta-analysis, numbers for the control group for these studies were halved to retain sample independence.27
The meta-analysis of the effect of verbal face-to-face feedback compared with no feedback on workplace task performance found an SMD of 1.09 (95% CI 0.59 to 1.59; p<0. 001) using a random effects model. The forest plot is presented in figure 5A.
As a sensitivity analysis, we repeated the random effects meta-analysis after excluding studies with a high risk of bias. Eight studies (8/11, 73%) were included that involved 392 health professional learners across 10 comparisons.30 31 33–37 39 The SMD was 0.7 (95% CI 0.37 to 1.03; p<0.001). The forest plot is presented in figure 5B. The prediction interval was −0.06 to 1.46.
We judged that the certainty of the evidence was low, using the GRADE approach. We downgraded the overall rating from high to low, in view of a serious risk of bias (in particular, due to a lack of concealment and potential for selective reporting of outcomes) and publication bias41 (see ‘Certainty of evidence’ section in the supplementary material for more details, online supplementary appendix 2). Figure 6 displays the summary of findings table.
Comparison 2: the effect of verbal face-to-face feedback, compared with alternative feedback, on performance
Table 2 presents the characteristics of included studies in the comparison of verbal face-to-face feedback compared with alternative feedback. Twenty studies (22 comparisons) were included in this analysis and involved verbal, face-to-face feedback compared with alternative feedback. Nine studies (9/20, 45%) were published in the last 5 years since 2014. The studies were conducted in Europe (8/20, 40%), USA (7/20, 35%), Canada (4/20, 20%) and Asia (1/20, 5%).
There were 1974 participants, including 660/1463 (45%) men from 13 studies that reported gender data.33 38 42–52 Included studies involved students (medical, mixed health professions and pharmacy) (1869, 95%) in 16 studies,33 37 38 42–45 47–55 and doctors (105, 5%) in four studies.31 39 46 56 All studies included assessment of a discrete task except two studies that involved longitudinal evaluations.39 46 Three studies evaluated performance in a clinical practice setting (involving teaching skills,39 professional and communication skills46 and oral case presentations),55 and the remaining 17 assessed performance in a simulated environment (surgical procedures, nasogastric tube insertion, intubation, hearing test, pharmacy consultation or CPR)31 33 37 38 42–45 47–54 56 (see ‘Included studies’ section in the supplementary material for more details, online supplementary appendix 3).
Risk of bias
In summarising the risk of bias across domains within each study, two studies were rated as low risk,43 50 seven studies were rated as ‘high risk’38 47 51 52 54 56 and the remaining studies were rated as ‘unclear’ (see the risk of bias summary in figure 3) (see ‘Risk of bias’ section in the supplementary material for more details, online supplementary appendix 3).
Effect of interventions
Figure 7 presents the forest plot and SMD. One additional study43 that reported categorical data is not included in the forest plot. It compared a learning conversation (315 participants, pass rate 80.9%) to a feedback sandwich (325 participants, pass rate 77.2%) resulting in an OR of 1.25 (95% CI 0.85 to 1.84) that favoured the learning conversation. The feedback comparisons were markedly diverse, so we did not pool outcomes in meta-analysis.
Comparison 1: the effect of verbal face-to-face feedback, compared with no feedback, on performance
Our meta-analysis found that verbal face-to-face feedback may result in a moderate to large improvement in health professionals’ performance compared with no feedback, with SMD 0.7 (95% CI 0.37 to 1.03; p<0.001) from eight studies involving 392 health professionals, after excluding studies at high risk of bias. However, the quality of evidence was low, primarily due to risk of bias and publication bias. To our knowledge, this is the first report to provide some substantiation for the widely held view that feedback enhances health professionals’ performance and to estimate the benefit (see ‘Discussion’ section in the supplementary material for more details, online supplementary appendix 4).
The consistent positive effects across all included studies, with substantially overlapping confidence intervals, supports the likelihood that verbal face-to-face feedback enhances performance in the health professions. Our pooled effect size was moderate to large at 0.7.57 The calculated prediction interval for the comparison of verbal face-to-face feedback to no feedback (excluding studies with a high risk of bias) was −0.06 to 1.45. This indicates a wide range in the likely feedback effect for any individual study, from a very small detrimental effect to a very large beneficial effect on performance. These results align with previous meta-analyses within health and other professions that have reported beneficial but variable effect sizes with different feedback interventions.14 18 19 For example, within the health professions, Ivers et al19 reported that 0.5% to 16% more participants followed desired practice when involved in an audit and feedback intervention. In comparison, a meta-analysis by Kluger and DeNisi,58 which analysed any type of feedback across any discipline, compared with no feedback, reported a pooled SMD of 0.4; notably one-third of included studies reported a detrimental impact.
One possible explanation for this variability is that some constituents within a feedback intervention are more effective than others. When specific feedback elements were isolated, the largest beneficial effects of feedback reported in Kluger and DeNisi’s meta-analysis were: (1) effect size 0.55 when feedback included information on any changes since the previous attempt, (3) effect size 0.51 when a specific and challenging goal was set, (3) effect size 0.47 when feedback posed little threat to self-esteem and (4) effect size 0.43 when feedback included information on the correct outcome.58
Comparison 2: the effect of verbal face-to-face feedback, compared with alternative feedback, on performance
For the second comparison of the effect of verbal face-to-face feedback compared with alternative feedback on performance, there was a diverse range in the alternative feedback interventions, which precluded meta-analysis. Where individual studies tested the relative impact of different feedback interventions, there was greater performance improvement seen with the following strategies: additional expert coaching sessions compared with routine monthly written feedback from supervisors46; expert feedback early in a practice period compared with later48; additional episodes of feedback from experts45 50; additional episodes of feedback involving expert video analysis56; and expert feedback compared with compliments.44 53
Influences on performance due to variations in the constituents of feedback interventions
The studies assembled in this review illustrate the wide variety of possible constituents within feedback interventions and the potential influence on performance. Within verbal face-to-face feedback interventions, there were important differences between included studies in feedback content, source and timing. Previous studies have noted potential beneficial effects attributable to feedback that contains information to clarify the goal10 19 31 58 is delivered by educators with perceived credibility18 19 59–63 and strategic use of both early and delayed feedback48 59 (see ‘Discussion’ section in the supplementary material for more details, online supplementary appendix 4).
Influences on performance due to factors beyond feedback
Performance improvement is not solely related to feedback. In our review, other important factors influencing performance, such as instruction and practice opportunities, also varied between studies. These included teaching and expert demonstration,37 48 59 64–69 learners’ background, task complexity and practice opportunities10 15 70 71 (see ‘Discussion’ section in the supplementary material for more details, online supplementary appendix 4).
The review has a number of limitations. Despite our attempts to be thorough, we may have missed studies that should have been included. As a number of studies did not report data that would allow easy pooling of data, we either calculated an estimate from available data (including reading off graphs) or excluded the study. Most included studies were conducted in a simulated environment, at Kirkpatrick evaluation level two (change in skills), with only a few situated in authentic clinical practice at Kirkpatrick level three (change in skills applied at work), which may limit application to routine clinical practice.72
Implications for future research and clinical practice
Our review supports the likely beneficial impact of verbal face-to-face feedback on health professionals’ task performance, compared with no feedback. By analysing included studies based on factors known to influence performance, our review assists future researchers by clarifying key parameters that need to be considered. Many of the included studies were ‘one-off’, involved small numbers of participants and included sources of bias. This indicates the need for studies that involve more participants and are methodologically better designed and executed. In addition, to address publication bias, larger published studies or identification of unpublished studies are needed. To advance this field of knowledge, research programmes designed to systematically investigate the constituents required for effective feedback are needed. This is likely to involve a series of studies designed to isolate one factor at a time, with all other key influences on performance standardised, in order to identify and replicate the conditions that are most effective in helping learners to improve, across different contexts. As key elements in effective feedback are established, implementing this knowledge across health professions education will be important to optimise both clinical practice and patient outcomes.
We systematically collated the available evidence regarding the impact of verbal face-to-face feedback on health professionals’ workplace task performance, compared with no or alternative feedback. In a meta-analysis, we found that verbal face-to-face feedback may result in a moderate to large improvement in workplace task performance, compared with no feedback SMD 0.7 (95% CI 0.37 to 1.03; p<0.001), after excluding studies at high risk of bias. We extracted and reported data on factors known to influence performance development, which included both components within feedback interventions and additional factors, such as providing teaching or practice opportunities. The diversity in feedback interventions identified in this review (even within ‘face-to-face feedback’) highlights the need to view feedback as a complex intervention.
The authors wish to thank Anne Young, Subject Librarian, Hargrave-Andrew Library, Monash University for her assistance with the electronic databases searches.
Contributors CEJ and JLK designed the review; CEJ created the search strategy and undertook the literature searches; CEJ and MPW undertook study screening, data extraction and quality assessment with oversight from JLK; CEJ and JLK undertook data analysis and interpreted the findings; CEJ prepared the manuscript; and all authors contributed to the final version.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request. Data access can be requested by contacting the corresponding author.