You are here

  • Home
  • Archive
  • Volume 10, Issue 3
  • Effect of face-to-face verbal feedback compared with no or alternative feedback on the objective workplace task performance of health professionals: a systematic review and meta-analysis

Article Text

Article menu

Download PDFPDF

XML
Medical education and training
Original research
Effect of face-to-face verbal feedback compared with no or alternative feedback on the objective workplace task performance of health professionals: a systematic review and meta-analysis
  1. Christina Elizabeth Johnson1,2,
  2. Mihiri P Weerasuria3,
  3. Jennifer L Keating4
  1. 1Monash Doctors Education, Monash Health; Faculty of Medicine, Nursing and Health Sciences, Monash University; Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Victoria, Australia
  2. 2Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Victoria, Australia
  3. 3Monash Health, Melbourne, Victoria, Australia
  4. 4Department of Physiotherapy, Monash University, Clayton, Victoria, Australia
  1. Correspondence to Dr Christina Elizabeth Johnson; christina.johnson{at}monashhealth.org

Abstract

Objective Verbal face-to-face feedback on clinical task performance is a fundamental component of health professions education. Experts argue that feedback is critical for performance improvement, but the evidence is limited. The aim of this systematic review was to investigate the effect of face-to-face verbal feedback from a health professional, compared with alternative or no feedback, on the objective workplace task performance of another health professional.

Design Systematic review and meta-analysis.

Methods We searched the full holdings of Ovid MEDLINE, CENTRAL, Embase, CINAHL and PsycINFO up to 1 February 2019 and searched references of included studies. Two authors independently undertook study selection, data extraction and quality appraisal. Studies were included if they were randomised controlled trials investigating the effect of feedback, in which health professionals were randomised to individual verbal face-to-face feedback compared with no feedback or alternative feedback and available as full-text publications in English. The certainty of evidence was assessed using the Grading of Recommendations, Assessment, Development and Evaluations approach. For feedback compared with no feedback, outcome data from included studies were pooled using a random effects model.

Results In total, 26 trials met the inclusion criteria, involving 2307 participants. For the effect of verbal face-to-face feedback on performance compared with no feedback, when studies at high risk of bias were excluded, eight studies involving 392 health professionals were included in a meta-analysis: the standardised mean difference (SMD) was 0.7 (95% CI 0.37 to 1.03; p<0.001) in favour of feedback. The calculated SMD prediction interval was −0.06 to 1.46. For feedback compared with alternative feedback, studies could not be pooled due to substantial design and intervention heterogeneity. All included studies were summarised, and key factors likely to influence performance were identified including components within feedback interventions, instruction and practice opportunities.

Conclusions Verbal face-to-face feedback in the health professions may result in a moderate to large improvement in workplace task performance, compared with no feedback. However, the quality of evidence was low, primarily due to risk of bias and publication bias. Further research is needed. In particular, we found a lack of high-quality trials that clearly reported key components likely to influence performance.

Trial registration number CRD42017081796.

  • medical education & training
  • feedback
  • health professions education
  • formative feedback
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

http://dx.doi.org/10.1136/bmjopen-2019-030672

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This systematic review is the first to investigate the impact of face-to-face verbal feedback from a health professional, compared with alternative or no feedback, on the objective workplace task performance of another health professional.

    • The meta-analysis of verbal feedback compared with no feedback is the first to provide an estimate of the likely benefit of verbal feedback on performance of a workplace task in the health professions.

    • The quality of evidence was low, primarily due to risk of bias in study design or conduct and publication bias.

    Introduction

    Health professions education is embedded in clinical practice for both students and qualified staff as they continue learning and training.1 Face-to-face verbal feedback focused on the performance of a clinical task involving an educator (senior clinician or peer) and a learner (any clinician) plays a crucial role in workplace learning, particularly within competency-based education and programmatic assessment models.2–5

    Multiple reviews on feedback in health professional education have been published and include recommendations for effective practice.6–9 Feedback can occur in various forms, including verbal, written or automated (eg, from a simulator or within an online learning module). The unique potential benefits of face-to-face verbal feedback are the opportunities for: (1) real-time interaction, to which the learner and educator bring their different perspectives, priorities and ideas to coconstruct insights and strategies for improvement and (2) interpersonal connection, through which an educator can foster a learner’s feelings of support, self-efficacy and motivation to improve, which are important catalysts in the learning process.8 10–13

    There is widespread acceptance that feedback has an important role in maximising learning and achievement.6 14–16 Ende17 said, ‘Without feedback, mistakes go uncorrected, good performance is not reinforced, and clinical competence is achieved empirically or not at all’. However, there is little evidence to support this view that feedback enhances health professionals’ performance. Indeed, a recent scoping review on feedback identified the need for systematic reviews to support evidence-based recommendations.7

    The current strongest evidence relates to two systematic reviews that investigated the impact of audit and feedback. In 2006, Veloski et al18 published a BEME systematic review in which almost 75% of included studies reported that audit and feedback could improve an individual physician’s clinical performance, particularly when sustained and from an authoritative source. Feedback was defined as ‘summary information on clinical performance over a defined time period’. They included any empirical study (not just randomised controlled trials) and all types of physicians (most were primary care physicians). The majority of outcomes were clinical processes (such as test ordering), and the most common data sources were medical records and billing records (none involved direct observation of performance).

    In 2012, Ivers et al19 updated a Cochrane review and meta-analysis that reported an increase in compliance with desired practice following audit and feedback, compared with usual care. The review included various health professionals (predominantly doctors), the unit of allocation for interventions ranged from individuals to health services and the performance outcomes reported were clinical practice processes, such as the number or quality of prescriptions or tests. The authors argued that although the median risk difference (RD) in favour of feedback was small at 4.3% (IQR 0.5%–16%), the 3rd quartile at 16% indicated that audit and feedback interventions could be much more effective. Using multivariable meta-regression, they identified that the effectiveness of audit and feedback increased when the source was a senior colleague or supervisor (RD 11%), the format involved both written and verbal components (RD 8%), the frequency was at least monthly (RD 7%), the aim was to reduce specific behaviour (RD 6%) and it included both explicit measurable targets and a specific action plan involving advice on how to improve, compared with performance information alone (RD 5%). In addition, two other factors were associated with a higher likelihood of improvement: a lower baseline performance and the type of behaviour being targeted, for example, prescribing (possibly perceived as ‘important’ and ‘straightforward’) had better outcomes than improving diabetes management (more ‘complex’) or test ordering (possibly perceived as ‘less important’).

    We found no systematic review that investigated the impact of verbal face-to-face feedback on a health professional’s performance, the typical scenario in clinical practice.

    Our research question was therefore:

    ‘What is known about the effect of face-to-face verbal feedback from a health professional, compared with alternative or no feedback, on the objective performance of an observable workplace task by another health professional?’

    The primary aim of the review addressed this question. Secondary aims were to summarise interventions and outcomes reported in included studies.

    Methods

    This review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses’ (PRISMA) statement.20 The protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO).

    Eligibility criteria for considering studies for this review

    We included randomised controlled trials in which individual health professionals were randomised to feedback, compared with no feedback or alternative feedback. Reports had to be available as English full-text publications.

    We included studies in which participants were health professional students or graduates from the disciplines of medicine, dentistry, nursing and midwifery, allied health, psychology, pharmacy, medical radiation practice, optometry, osteopathy or chiropracty.

    All studies had to include at least one intervention involving verbal face-to-face feedback generated by a health professional, based on the observable performance of a workplace task performed by another health professional. A broad definition of feedback was permitted with a minimum requirement that it included information regarding learner performance. Studies were excluded if feedback was predetermined or provided only by a simulated patient or machine. Audit and feedback studies, where feedback was based on aggregated quality performance indicators (such as numbers of tests ordered or degree of compliance with quality practice standards) were excluded, as this was deemed to be distinctly different from a workplace task, such as suturing, that could be observed, objectively assessed and targeted for improvement with feedback. Two comparisons were evaluated: (1) verbal face-to-face feedback compared with no feedback and (2) verbal face-to-face feedback compared with alternative feedback.

    Performance following feedback interventions had to be objectively assessed. To isolate the effects of feedback, other conditions had to be comparable for both groups. Studies were excluded if the report did not include point estimates of effects and measures of variability (or data from which these could be derived), unless these data could be obtained from the author.

    Search methods for identification of studies

    We developed the search strategy in collaboration with a senior medical librarian using MEDLINE subject headings. Key words were used, including synonyms, truncation, wildcard and proximity operators related to ‘feedback’ AND ‘health professional’ AND ‘performance’ AND ‘randomised controlled trial’ (see online supplementary appendix 1 for the full search strategy for Ovid MEDLINE). We translated this search strategy for other databases. The full holdings of Ovid MEDLINE (1946 to present with daily update), CENTRAL, Embase (1946 to present with daily update), CINAHL plus (1937 to present) and PsycINFO (1806 to present) were searched until 1 February 2019. We also searched the reference lists of systematic reviews and included studies.

    Selection of studies

    One review author (CEJ or MPW) screened titles to exclude clearly irrelevant reports. Two authors (CEJ and MPW) independently screened remaining abstracts to identify potential eligible studies, then independently assessed the full text. Decisions were compared using Covidence (online software designed by the Cochrane Collaboration to improve review efficiency via www.covidence.org), and disagreements were resolved through discussion, including a third review author (JLK).

    Data extraction

    One review author (CEJ) used a prepiloted standardised form to extract data from included studies, and another author (MPW or JLK) checked the data extracted were accurate. We resolved discrepancies through discussion. The following data were recorded: year of publication; study setting; funding sources; key details regarding participants, workplace task, feedback intervention and outcome measures; and information related to the risk of bias assessment. If data were missing, we contacted authors to request the information.

    Assessment of risk of bias in included studies

    The risk of bias was independently assessed by two authors (CEJ and JLK) for the selected performance outcome for individual studies, using Cochrane’s ‘risk of bias’ tool (Chapter 8, Cochrane Handbook for Systematic Reviews of Interventions).21 In particular, we used the following decision rules in assessing the risk of bias for specific individual domains. For ‘participant and research team blinding’: a participant receiving feedback or an educator giving feedback was deemed not to be blinded, even if they were deliberately not informed about the intervention or any differences between interventions. Nevertheless a ‘low risk’ rating was given if the outcome was not likely to be influenced by this lack of blinding, for example, if there were no changes to protocol or adherence that arose as a consequence of participant knowledge of group allocation.22 For ‘incomplete outcome data’: to be rated as ‘low risk’, studies were required to include outcome data on at least 85% of the participants enrolled in each group (as per PEDRO guidelines),23 and to provide participant numbers at the start and the number that dropped out during the study, from which group and the reasons.

    The risk of bias was then summarised within each study across domains for the performance outcome, in accordance with the Cochrane ‘risk of bias’ assessment tool.

    Measures of treatment effect

    Outcomes from included studies were expressed using point estimates and measures of variability (eg, means (SD) or median (IQR). The effect was quantified using the standardised mean difference (SMD) to combine studies measuring the same outcome (task performance) using different measurement scales. When not reported, we estimated required data using available data or contacted study authors. If multiple outcomes were reported, we preferentially used the outcome that summarised multiple relevant task components, thereby providing a global, task-specific evaluation. If more than one reported outcome met this principle, we combined outcomes to provide a single metric using weighted averages of standardised scores.

    We created and visually examined a funnel plot to explore reporting bias (Chapter 10, Cochrane Handbook).24 25

    Data synthesis and assessment of heterogeneity

    We pooled data from comparable studies for the comparison of feedback to no feedback on any measure of task performance and conducted analysis using random effects modelling in RevMan software (Review Manager Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014). The result of the random effects meta-analysis was presented as the SMD of the treatment effect with 95% CI, as the average effect across multiple studies and its error estimates.

    As a sensitivity analysis, we conducted a meta-analysis excluding studies with a high risk of bias. Using this pooled data, we calculated a prediction interval, which describes the range of likely results for new individual studies.26

    We rated the overall certainty of evidence for the outcome using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach (Chapter 12, Cochrane Handbook27 28 and GRADE guidelines),29 which considers within-study risk of bias, directness of evidence, heterogeneity, precision of effect estimates and risk of publication bias. Two authors independently rated the certainty of the evidence and resolved disagreements by discussion. We presented a summary of the evidence in a ‘Summary of Findings’ table.

    Patient and public involvement

    There was no involvement of patients or the public in any part of this research.

    Results

    Search results

    The search yielded 1238 articles after 409 duplicates were removed. Based on title or abstract, we excluded 1110 articles. We assessed the remaining 128 full-text articles for eligibility and found 26 randomised controlled trials that met all inclusion criteria. See figure 1 for PRISMA study flow diagram.

    Figure 1
    Figure 1

    PRISMA flow diagram for systematic review of verbal face-to-face feedback compared with no or alternative feedback. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCT, randomised controlled trial.

    Comparison 1: the effect of verbal face-to-face feedback, compared with no feedback, on performance

    Included studies

    Table 1 presents the characteristics of included studies in this comparison. Eleven randomised controlled trials investigated the effect of verbal face-to-face feedback compared with no feedback on the objective evaluation of a workplace task. Seven (64%) reports were published in the last 5 years since 2014. The studies were conducted in Europe (4),30–33 Canada (4),34–37 the USA (2)38 39 and Asia (1).40

    View this table:
    Table 1

    Summary of available data on characteristics of trials included in the comparison of verbal face-to-face feedback (intervention) compared with no feedback (control: no feedback from any external source) on performance

    There were 488 participants, including 196/366 (53.6%) men from seven studies that reported gender data.30 32–35 38 40 Participants included 290 (60%) medical students in four studies,30 33 37 38 60 (12%) dental students in one study32 and 138 (28%) doctors in six studies.31 34–36 39 40 The workplace tasks involved a discrete task such as surgical procedures, cardiopulmonary resuscitation (CPR) or teaching skills, which occurred in clinical practice in four studies34 35 39 40 and a simulation environment in seven studies (7/11, 64%).30–33 36–38 Differences in feedback interventions between included studies involved feedback source (expert or peer), timing (during task performance, directly afterwards or delayed), content (evaluative information only or additional corrective advice, performance video, simulator information or written report) and number of feedback episodes. In addition, there was variation between studies in provision of instruction and expert demonstration of the task, opportunities for practice and duration of feedback intervention (see ‘Included studies’ section in the supplementary material for more details, online supplementary appendix 2).

    Risk of bias

    The risk of bias graph is presented in figure 2, and the risk of bias summary is presented in figure 3. In summarising the risk of bias across domains within each study, two studies were rated ‘low risk’,34 36 six studies were rated ‘unclear’30 31 33 35 37 39 and three studies were ‘high risk’32 38 40 (see ‘Risk of bias’ section in the supplementary material for more details, online supplementary appendix 2).

    Figure 2
    Figure 2

    Risk of bias graph: review authors' judgements about each risk of bias itempresented as percentages across all included studies.

    Figure 3
    Figure 3

    Risk of bias summary: review authors’ judgements about each risk of bias item foreach included study.

    Reporting bias

    The funnel plots are presented in figure 4: for all included studies (figure 4A) and after excluding studies at high risk of bias (figure 4B). Both funnel plots are asymmetrical, with a paucity of small studies with negative effect sizes that are less likely to be published, indicating some potential for publication bias.

    Figure 4
    Figure 4

    Funnel plot of the comparison of the effect of verbal face-to-face feedback, compared with no feedback, on performance. Meta-analysis calcuated using a fixed effects model. The dotted vertical line represents the overall effect estimate and the dotted slanted lines represent the 95% CI. SMD, standardised mean difference.

    Meta-analysis

    A meta-analysis of the impact of verbal face-to-face feedback compared with no feedback on performance included 13 comparisons from the 11 studies, involving 488 participants. Two studies reported data that each enabled two comparisons: in one study, feedback provided during practice in one group and directly after practice in another were compared with no feedback37; in another study, feedback provided by an expert in one group and by a peer in another group31 were compared with no feedback. In the meta-analysis, numbers for the control group for these studies were halved to retain sample independence.27

    The meta-analysis of the effect of verbal face-to-face feedback compared with no feedback on workplace task performance found an SMD of 1.09 (95% CI 0.59 to 1.59; p<0. 001) using a random effects model. The forest plot is presented in figure 5A.

    Figure 5
    Figure 5

    Forest plot for the meta-analysis of the effect of verbal face-to-face feedback, compared with no feedback, on performance. Ahlborg 2015: mean and SD read from graph. Boyle 2011: mean and SD read from graph. Bonrath 2015: combined outcome calculated. Pavo 2016: median taken as best estimate of mean and calculated SD from IQR. Xeroulis 2007: SD estimated from 95% CI. SMD, standardised mean difference.

    Sensitivity analysis

    As a sensitivity analysis, we repeated the random effects meta-analysis after excluding studies with a high risk of bias. Eight studies (8/11, 73%) were included that involved 392 health professional learners across 10 comparisons.30 31 33–37 39 The SMD was 0.7 (95% CI 0.37 to 1.03; p<0.001). The forest plot is presented in figure 5B. The prediction interval was −0.06 to 1.46.

    We judged that the certainty of the evidence was low, using the GRADE approach. We downgraded the overall rating from high to low, in view of a serious risk of bias (in particular, due to a lack of concealment and potential for selective reporting of outcomes) and publication bias41 (see ‘Certainty of evidence’ section in the supplementary material for more details, online supplementary appendix 2). Figure 6 displays the summary of findings table.

    Figure 6
    Figure 6

    Summary of findings table for the effect of verbal face-to-face feedback, compared with no feedback, on performance,excluding studies with a high risk of bias.

    Comparison 2: the effect of verbal face-to-face feedback, compared with alternative feedback, on performance

    Included studies

    Table 2 presents the characteristics of included studies in the comparison of verbal face-to-face feedback compared with alternative feedback. Twenty studies (22 comparisons) were included in this analysis and involved verbal, face-to-face feedback compared with alternative feedback. Nine studies (9/20, 45%) were published in the last 5 years since 2014. The studies were conducted in Europe (8/20, 40%), USA (7/20, 35%), Canada (4/20, 20%) and Asia (1/20, 5%).

    View this table:
    Table 2

    Summary of available data on characteristics of trials comparing the effect of verbal face-to-face fb (intervention A), to alternative fb (intervention B), on performance

    There were 1974 participants, including 660/1463 (45%) men from 13 studies that reported gender data.33 38 42–52 Included studies involved students (medical, mixed health professions and pharmacy) (1869, 95%) in 16 studies,33 37 38 42–45 47–55 and doctors (105, 5%) in four studies.31 39 46 56 All studies included assessment of a discrete task except two studies that involved longitudinal evaluations.39 46 Three studies evaluated performance in a clinical practice setting (involving teaching skills,39 professional and communication skills46 and oral case presentations),55 and the remaining 17 assessed performance in a simulated environment (surgical procedures, nasogastric tube insertion, intubation, hearing test, pharmacy consultation or CPR)31 33 37 38 42–45 47–54 56 (see ‘Included studies’ section in the supplementary material for more details, online supplementary appendix 3).

    Risk of bias

    In summarising the risk of bias across domains within each study, two studies were rated as low risk,43 50 seven studies were rated as ‘high risk’38 47 51 52 54 56 and the remaining studies were rated as ‘unclear’ (see the risk of bias summary in figure 3) (see ‘Risk of bias’ section in the supplementary material for more details, online supplementary appendix 3).

    Effect of interventions

    Figure 7 presents the forest plot and SMD. One additional study43 that reported categorical data is not included in the forest plot. It compared a learning conversation (315 participants, pass rate 80.9%) to a feedback sandwich (325 participants, pass rate 77.2%) resulting in an OR of 1.25 (95% CI 0.85 to 1.84) that favoured the learning conversation. The feedback comparisons were markedly diverse, so we did not pool outcomes in meta-analysis.

    Figure 7
    Figure 7

    Forest plot for the effect of verbal face-to-face feedback (feedback A), compared with alternative feedback (feedback B), on performance. Baldwin 2015: categorical data not included in this figure; see text in Results. Al-Jundi 2017: additional information (data to calculate mean and SD for each group) from author. Boehler 2006: additional information (number of participants in each group and SD) from author. Lee 2016: calculated SD from SE. Manzone 2014: calculated standardised score to combine outcome of supine and normal positions. Pavo 2016: median taken as best estimate of mean. Rogers 2012: additional information (standard deviation) from author. Sox 2014: SD derived from reported t, p and mean values. Assumption that SDs were equivalent for intervention and controls. Strandbygaard 2013: SE derived from 95% CI. fb, feedback; GRS, global rating scale; info, information; PRN, ‘as required’.

    Discussion

    Comparison 1: the effect of verbal face-to-face feedback, compared with no feedback, on performance

    Our meta-analysis found that verbal face-to-face feedback may result in a moderate to large improvement in health professionals’ performance compared with no feedback, with SMD 0.7 (95% CI 0.37 to 1.03; p<0.001) from eight studies involving 392 health professionals, after excluding studies at high risk of bias. However, the quality of evidence was low, primarily due to risk of bias and publication bias. To our knowledge, this is the first report to provide some substantiation for the widely held view that feedback enhances health professionals’ performance and to estimate the benefit (see ‘Discussion’ section in the supplementary material for more details, online supplementary appendix 4).

    The consistent positive effects across all included studies, with substantially overlapping confidence intervals, supports the likelihood that verbal face-to-face feedback enhances performance in the health professions. Our pooled effect size was moderate to large at 0.7.57 The calculated prediction interval for the comparison of verbal face-to-face feedback to no feedback (excluding studies with a high risk of bias) was −0.06 to 1.45. This indicates a wide range in the likely feedback effect for any individual study, from a very small detrimental effect to a very large beneficial effect on performance. These results align with previous meta-analyses within health and other professions that have reported beneficial but variable effect sizes with different feedback interventions.14 18 19 For example, within the health professions, Ivers et al19 reported that 0.5% to 16% more participants followed desired practice when involved in an audit and feedback intervention. In comparison, a meta-analysis by Kluger and DeNisi,58 which analysed any type of feedback across any discipline, compared with no feedback, reported a pooled SMD of 0.4; notably one-third of included studies reported a detrimental impact.

    One possible explanation for this variability is that some constituents within a feedback intervention are more effective than others. When specific feedback elements were isolated, the largest beneficial effects of feedback reported in Kluger and DeNisi’s meta-analysis were: (1) effect size 0.55 when feedback included information on any changes since the previous attempt, (3) effect size 0.51 when a specific and challenging goal was set, (3) effect size 0.47 when feedback posed little threat to self-esteem and (4) effect size 0.43 when feedback included information on the correct outcome.58

    Comparison 2: the effect of verbal face-to-face feedback, compared with alternative feedback, on performance

    For the second comparison of the effect of verbal face-to-face feedback compared with alternative feedback on performance, there was a diverse range in the alternative feedback interventions, which precluded meta-analysis. Where individual studies tested the relative impact of different feedback interventions, there was greater performance improvement seen with the following strategies: additional expert coaching sessions compared with routine monthly written feedback from supervisors46; expert feedback early in a practice period compared with later48; additional episodes of feedback from experts45 50; additional episodes of feedback involving expert video analysis56; and expert feedback compared with compliments.44 53

    Influences on performance due to variations in the constituents of feedback interventions

    The studies assembled in this review illustrate the wide variety of possible constituents within feedback interventions and the potential influence on performance. Within verbal face-to-face feedback interventions, there were important differences between included studies in feedback content, source and timing. Previous studies have noted potential beneficial effects attributable to feedback that contains information to clarify the goal10 19 31 58 is delivered by educators with perceived credibility18 19 59–63 and strategic use of both early and delayed feedback48 59 (see ‘Discussion’ section in the supplementary material for more details, online supplementary appendix 4).

    Influences on performance due to factors beyond feedback

    Performance improvement is not solely related to feedback. In our review, other important factors influencing performance, such as instruction and practice opportunities, also varied between studies. These included teaching and expert demonstration,37 48 59 64–69 learners’ background, task complexity and practice opportunities10 15 70 71 (see ‘Discussion’ section in the supplementary material for more details, online supplementary appendix 4).

    Review limitations

    The review has a number of limitations. Despite our attempts to be thorough, we may have missed studies that should have been included. As a number of studies did not report data that would allow easy pooling of data, we either calculated an estimate from available data (including reading off graphs) or excluded the study. Most included studies were conducted in a simulated environment, at Kirkpatrick evaluation level two (change in skills), with only a few situated in authentic clinical practice at Kirkpatrick level three (change in skills applied at work), which may limit application to routine clinical practice.72

    Implications for future research and clinical practice

    Our review supports the likely beneficial impact of verbal face-to-face feedback on health professionals’ task performance, compared with no feedback. By analysing included studies based on factors known to influence performance, our review assists future researchers by clarifying key parameters that need to be considered. Many of the included studies were ‘one-off’, involved small numbers of participants and included sources of bias. This indicates the need for studies that involve more participants and are methodologically better designed and executed. In addition, to address publication bias, larger published studies or identification of unpublished studies are needed. To advance this field of knowledge, research programmes designed to systematically investigate the constituents required for effective feedback are needed. This is likely to involve a series of studies designed to isolate one factor at a time, with all other key influences on performance standardised, in order to identify and replicate the conditions that are most effective in helping learners to improve, across different contexts. As key elements in effective feedback are established, implementing this knowledge across health professions education will be important to optimise both clinical practice and patient outcomes.

    Summary

    We systematically collated the available evidence regarding the impact of verbal face-to-face feedback on health professionals’ workplace task performance, compared with no or alternative feedback. In a meta-analysis, we found that verbal face-to-face feedback may result in a moderate to large improvement in workplace task performance, compared with no feedback SMD 0.7 (95% CI 0.37 to 1.03; p<0.001), after excluding studies at high risk of bias. We extracted and reported data on factors known to influence performance development, which included both components within feedback interventions and additional factors, such as providing teaching or practice opportunities. The diversity in feedback interventions identified in this review (even within ‘face-to-face feedback’) highlights the need to view feedback as a complex intervention.

    Acknowledgments

    The authors wish to thank Anne Young, Subject Librarian, Hargrave-Andrew Library, Monash University for her assistance with the electronic databases searches.

    References

    1. 1.↵
      1. Morris C,
      2. Blaney D
      . Work-based learning. In: Swanwick T, ed. Understanding medical education: evidence, theory and practice. 2nd ed.. Oxford: The Association for the study of Medical Education, 2014: 97–109.
    2. 2.↵
      1. Carraccio C,
      2. Englander R,
      3. Van Melle E, et al
      . Advancing competency-based medical education: a charter for clinician-educators. Acad Med 2016;91:645–9.doi:10.1097/ACM.0000000000001048
      OpenUrl
    3. 3.↵
      1. van der Vleuten CPM,
      2. Schuwirth LWT,
      3. Driessen EW, et al
      . A model for programmatic assessment fit for purpose. Med Teach 2012;34:205–14.doi:10.3109/0142159X.2012.652239
      OpenUrlCrossRefPubMed
    4. 4.↵
      1. Holmboe ES
      . Realizing the promise of competency-based medical education. Acad Med 2015;90:411–3.doi:10.1097/ACM.0000000000000515
      OpenUrlPubMed
    5. 5.↵
      1. Kogan JR,
      2. Holmboe ES,
      3. Hauer KE
      . Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. JAMA 2009;302:1316–26.doi:10.1001/jama.2009.1365
      OpenUrlCrossRefPubMedWeb of Science
    6. 6.↵
      1. Archer JC
      . State of the science in health professional education: effective feedback. Med Educ 2010;44:101–8.doi:10.1111/j.1365-2923.2009.03546.x
      OpenUrlCrossRefPubMedWeb of Science
    7. 7.↵
      1. Bing-You R,
      2. Hayes V,
      3. Varaklis K, et al
      . Feedback for learners in medical education: what is known? A scoping review. Acad Med 2017;92:1346–54.doi:10.1097/ACM.0000000000001578
      OpenUrlCrossRefPubMed
    8. 8.↵
      1. Johnson CE,
      2. Keating JL,
      3. Boud DJ, et al
      . Identifying educator behaviours for high quality verbal feedback in health professions education: literature review and expert refinement. BMC Med Educ 2016;16:96.doi:10.1186/s12909-016-0613-5
    9. 9.↵
      1. Watling CJ,
      2. Ginsburg S,
      3. Assessment GS
      . Assessment, feedback and the alchemy of learning. Med Educ 2019;53:76–85.doi:10.1111/medu.13645
      OpenUrl
    10. 10.↵
      1. Locke EA,
      2. Latham GP
      . Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. Am Psychol 2002;57:705–17.doi:10.1037/0003-066X.57.9.705
      OpenUrlCrossRefPubMedWeb of Science
    11. 11.↵
      1. Telio S,
      2. Ajjawi R,
      3. Regehr G
      . The "educational alliance" as a framework for reconceptualizing feedback in medical education. Acad Med 2015;90:609–14.doi:10.1097/ACM.0000000000000560
      OpenUrlCrossRef
    12. 12.↵
      1. Ten Cate TJ,
      2. Kusurkar RA,
      3. Williams GC
      . How self-determination theory can assist our understanding of the teaching and learning processes in medical education. AMEE guide No. 59. Med Teach 2011;33:961–73.doi:10.3109/0142159X.2011.595435
      OpenUrlCrossRefPubMed
    13. 13.↵
      1. Johnson CE,
      2. Keating JL,
      3. Farlie MK, et al
      . Educators' behaviours during feedback in authentic clinical practice settings: an observational study and systematic analysis. BMC Med Educ 2019;19:129.doi:10.1186/s12909-019-1524-z
    14. 14.↵
      1. Hattie J,
      2. Timperley H
      . The power of feedback. Rev Educ Res 2007;77:81–112.doi:10.3102/003465430298487
      OpenUrlCrossRef
    15. 15.↵
      1. Ericsson KA
      . Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med 2008;15:988-94.doi:10.1111/j.1553-2712.2008.00227.x
      OpenUrl
    16. 16.↵
      1. Molloy E,
      2. Boud D
      . Changing conceptions of feedback. In: Molloy E DB, ed. Feedback in higher and professional education. London: Routledge, 2013: 11–33.
    17. 17.↵
      1. Ende J
      . Feedback in clinical medical education. JAMA 1983;250:777–81.doi:10.1001/jama.1983.03340060055026
      OpenUrlCrossRefPubMedWeb of Science
    18. 18.↵
      1. Veloski J,
      2. Boex JR,
      3. Grasberger MJ, et al
      . Systematic review of the literature on assessment, feedback and physicians' clinical performance: BEME guide No. 7. Med Teach 2006;28:117–28.doi:10.1080/01421590600622665
      OpenUrlCrossRefPubMedWeb of Science
    19. 19.↵
      1. Ivers N,
      2. Jamtvedt G,
      3. Flottorp S, et al
      . Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012:CD000259.doi:10.1002/14651858.CD000259.pub3
    20. 20.↵
      1. Moher D,
      2. Liberati A,
      3. Tetzlaff J, et al
      . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009;151:264–-9.doi:10.7326/0003-4819-151-4-200908180-00135
      OpenUrlCrossRefPubMedWeb of Science
    21. 21.↵
      1. Higgins JPT,
      2. Churchill R,
      3. Chandler J, et al
      . Cochrane Handbook for systematic reviews of interventions version 5.2.0 (updated June 2017), 2017. Available: www.training.cochrane.org/handbook
    22. 22.↵
      1. Sterne JAC,
      2. Savović J,
      3. Page MJ, et al
      . Rob 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898.doi:10.1136/bmj.l4898
      OpenUrlFREE Full Text
    23. 23.↵
      1. Verhagen AP,
      2. de Vet HCW,
      3. de Bie RA, et al
      . The Delphi list. J Clin Epidemiol 1998;51:1235–41.doi:10.1016/S0895-4356(98)00131-0
      OpenUrlCrossRefPubMedWeb of Science
    24. 24.↵
      1. Sterne JAC,
      2. Sutton AJ,
      3. Ioannidis JPA, et al
      . Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 2011;343:d4002.doi:10.1136/bmj.d4002
      OpenUrlFREE Full Text
    25. 25.↵
      1. Sterne JAC,
      2. Egger M,
      3. Moher D, et al
      . Chapter 10: Addressing reporting biases. In: Higgins JPT, Churchill R, Chandler J, et al., eds. Cochrane Handbook for systematic reviews of interventions version 520 (updated June 2017, 2017. www.training.cochrane.org/handbook
    26. 26.↵
      1. Riley RD,
      2. Higgins JPT,
      3. Deeks JJ
      . Interpretation of random effects meta-analyses. BMJ 2011;342:d549.doi:10.1136/bmj.d549
    27. 27.↵
      1. Higgins JPT,
      2. Green S
      . Cochrane Handbook for systematic reviews of interventions version 5.1.0, 2011. Available: www.handbook.cochrane.org [Accessed Mar 2011].
    28. 28.↵
      1. Schünemann HJ,
      2. Oxman AD,
      3. Vist GE, et al
      . Chapter 12: Interpreting results and drawing conclusions. In: Higgins JPT, Churchill R, Chandler J, et al., eds. Cochrane Handbook for systematic reviews of interventions version 520 (updated June 2017, 2017. www.training.cochrane.org/handbook
    29. 29.↵
      Grade guidelines Journal of clinical epidemiology series. Available: https://www.gradeworkinggroup.org/2020
    30. 30.↵
      1. Ahlborg L,
      2. Weurlander M,
      3. Hedman L, et al
      . Individualized feedback during simulated laparoscopic training:a mixed methods study. Int J Med Educ 2015;6:93–100.doi:10.5116/ijme.55a2.218b
      OpenUrl
    31. 31.↵
      1. Boyle E,
      2. O'Keeffe DA,
      3. Naughton PA, et al
      . The importance of expert feedback during endovascular simulator training. J Vasc Surg 2011;54:240–8.doi:10.1016/j.jvs.2011.01.058
      OpenUrlPubMed
    32. 32.↵
      1. Olms C,
      2. Jakstat HA,
      3. Haak R
      . The implementation of Elaborative feedback for qualitative improvement of shade Matching-A randomized study. J Esthet Restor Dent 2016;28:277–86.doi:10.1111/jerd.12231
      OpenUrl
    33. 33.↵
      1. Pavo N,
      2. Goliasch G,
      3. Nierscher FJ, et al
      . Short structured feedback training is equivalent to a mechanical feedback device in two-rescuer BLS: a randomised simulation study. Scand J Trauma Resusc Emerg Med 2016;24:70.doi:10.1186/s13049-016-0265-9
    34. 34.↵
      1. Bonrath EM,
      2. Dedy NJ,
      3. Gordon LE, et al
      . Comprehensive surgical coaching enhances surgical skill in the operating room: a randomized controlled trial. Ann Surg 2015;262:205–12.doi:10.1097/SLA.0000000000001214
      OpenUrl
    35. 35.↵
      1. Kroft J,
      2. Ordon M,
      3. Po L, et al
      . Preoperative practice paired with Instructor feedback may not improve obstetrics-gynecology residents' operative performance. J Grad Med Educ 2017;9:190–4.doi:10.4300/JGME-D-16-00238.1
      OpenUrl
    36. 36.↵
      1. Soucisse ML,
      2. Boulva K,
      3. Sideris L, et al
      . Video coaching as an efficient teaching method for surgical Residents-A randomized controlled trial. J Surg Educ 2017;74:365–71.doi:10.1016/j.jsurg.2016.09.002
      OpenUrl
    37. 37.↵
      1. Xeroulis GJ,
      2. Park J,
      3. Moulton C-A, et al
      . Teaching suturing and knot-tying skills to medical students: a randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery 2007;141:442–9.doi:10.1016/j.surg.2006.09.012
      OpenUrlCrossRefPubMedWeb of Science
    38. 38.↵
      1. O'Connor A,
      2. Schwaitzberg SD,
      3. Cao CGL
      . How much feedback is necessary for learning to suture? Surg Endosc 2008;22:1614–9.doi:10.1007/s00464-007-9645-6
      OpenUrlCrossRefPubMedWeb of Science
    39. 39.↵
      1. Skeff KM
      . Evaluation of a method for improving the teaching performance of attending physicians. Am J Med 1983;75:465–70.doi:10.1016/0002-9343(83)90351-0
      OpenUrlCrossRefPubMedWeb of Science
    40. 40.↵
      1. Vafaei A,
      2. Heidari K,
      3. Hosseini M-A, et al
      . Role of feedback during evaluation in improving emergency medicine residents' skills; an experimental study. Emerg 2017;5:e28.
      OpenUrl
    41. 41.↵
      1. Guyatt G,
      2. Oxman AD,
      3. Sultan S, et al
      . Grade guidelines: 11. making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol 2013;66:151–7.doi:10.1016/j.jclinepi.2012.01.006
      OpenUrlCrossRefPubMed
    42. 41.↵
      1. Al-Jundi W,
      2. Elsharif M,
      3. Anderson M, et al
      . A Randomized Controlled Trial to Compare e-Feedback Versus "Standard" Face-to-Face Verbal Feedback to Improve the Acquisition of Procedural Skill. J Surg Educ 2017;74:390–7.doi:10.1016/j.jsurg.2016.11.011
      OpenUrl
    43. 41.↵
      1. Baldwin LJL,
      2. Jones CM,
      3. Hulme J, et al
      . Use of the learning conversation improves instructor confidence in life support training: an open randomised controlled cross-over trial comparing teaching feedback mechanisms. Resuscitation 2015;96:199–207.doi:10.1016/j.resuscitation.2015.08.010
      OpenUrl
    44. 41.↵
      1. Boehler ML,
      2. Rogers DA,
      3. Schwind CJ, et al
      . An investigation of medical student reactions to feedback: a randomised controlled trial. Med Educ 2006;40:746–9.doi:10.1111/j.1365-2929.2006.02503.x
      OpenUrlCrossRefPubMed
    45. 41.↵
      1. Bosse HM,
      2. Mohr J,
      3. Buss B, et al
      . The benefit of repetitive skills training and frequency of expert feedback in the early acquisition of procedural skills. BMC Med Educ 2015;15:22.doi:10.1186/s12909-015-0286-5
    46. 41.↵
      1. Brinkman WB,
      2. Geraghty SR,
      3. Lanphear BP, et al
      . Effect of multisource feedback on resident communication skills and professionalism: a randomized controlled trial. Arch Pediatr Adolesc Med 2007;161:44–9.doi:10.1001/archpedi.161.1.44
      OpenUrlCrossRefPubMed
    47. 47.↵
      1. DeLucenay AJ,
      2. Conn KM,
      3. Corigliano A
      . An evaluation of the impact of immediate compared to delayed feedback on the development of counselling skills in pharmacy students. Pharmacy Education 2017;17:322–8.
      OpenUrl
    48. 47.↵
      1. Lee JY,
      2. McDougall EM,
      3. Lineberry M, et al
      . Optimizing the timing of expert feedback during simulation-based spaced practice of endourologic skills. Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare 2016;11:257–63.doi:10.1097/SIH.0000000000000165
      OpenUrl
    49. 47.↵
      1. Ozcakar N,
      2. Mevsim V,
      3. Guldal D, et al
      . Is the use of videotape recording superior to verbal feedback alone in the teaching of clinical skills? BMC Public Health 2009;9:474.doi:10.1186/1471-2458-9-474
    50. 47.↵
      1. Strandbygaard J,
      2. Bjerrum F,
      3. Maagaard M, et al
      . Instructor feedback versus no instructor feedback on performance in a laparoscopic virtual reality simulator: a randomized trial. Ann Surg 2013;257:839–44.doi:10.1097/SLA.0b013e31827eee6e
      OpenUrlCrossRefPubMed
    51. 47.↵
      1. van de Ridder JMM,
      2. Peters CMM,
      3. Stokking KM, et al
      . Framing of feedback impacts student's satisfaction, self-efficacy and performance. Adv Health Sci Educ Theory Pract 2015;20:803–16.doi:10.1007/s10459-014-9567-8
      OpenUrl
    52. 52.↵
      1. van de Ridder JMM,
      2. Berk FCJ,
      3. Stokking KM, et al
      . Feedback providers' credibility impacts students' satisfaction with feedback and delayed performance. Med Teach 2015;37:767–74.doi:10.3109/0142159X.2014.970617
      OpenUrl
    53. 52.↵
      1. Rogers DA,
      2. Boehler ML,
      3. Schwind CJ, et al
      . Engaging medical students in the feedback process. Am J Surg 2012;203:21–5.doi:10.1016/j.amjsurg.2011.07.009
      OpenUrlPubMed
    54. 52.↵
      1. Manzone J,
      2. Tremblay L,
      3. You-Ten KE, et al
      . Task- versus ego-oriented feedback delivered as numbers or comments during intubation training. Med Educ 2014;48:430–40.doi:10.1111/medu.12397
      OpenUrlCrossRefPubMed
    55. 52.↵
      1. Sox CM,
      2. Dell M,
      3. Phillipi CA, et al
      . Feedback on oral presentations during pediatric clerkships: a randomized controlled trial. Pediatrics 2014;134:965–71.doi:10.1542/peds.2014-1209
      OpenUrlAbstract/FREE Full Text
    56. 56.↵
      1. Backstein D,
      2. Agnidis Z,
      3. Sadhu R, et al
      . Effectiveness of repeated video feedback in the acquisition of a surgical technical skill. Can J Surg 2005;48:195–200.
      OpenUrlPubMed
    57. 56.↵
      1. Cohen J
      . Statistical power analysis for the behavioral sciences. Hillsdale, NJ: L. Erlbaum Associates, 1988.
    58. 58.↵
      1. Kluger AN,
      2. DeNisi A
      . The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 1996;119:254–84.doi:10.1037/0033-2909.119.2.254
      OpenUrlCrossRefWeb of Science
    59. 59.↵
      1. Shute VJ
      . Focus on formative feedback. Rev Educ Res 2008;78:153–89.doi:10.3102/0034654307313795
      OpenUrlCrossRef
    60. 58.↵
      1. Telio S,
      2. Regehr G,
      3. Ajjawi R
      . Feedback and the educational alliance: examining credibility judgements and their consequences. Med Educ 2016;50:933–42.doi:10.1111/medu.13063
      OpenUrl
    61. 61.↵
      1. Bing-You RG,
      2. Paterson J,
      3. Levine MA
      . Feedback falling on deaf ears: residents' receptivity to feedback tempered by sender credibility. Med Teach 1997;19:40–4.doi:10.3109/01421599709019346
      OpenUrlCrossRef
    62. 62.↵
      1. Sargeant J,
      2. Mann K,
      3. Sinclair D, et al
      . Challenges in multisource feedback: intended and unintended outcomes. Med Educ 2007;41:583–91.doi:10.1111/j.1365-2923.2007.02769.x
      OpenUrlCrossRefPubMedWeb of Science
    63. 61.↵
      1. Eva KW,
      2. Armson H,
      3. Holmboe E, et al
      . Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and Reasoning processes. Adv Health Sci Educ Theory Pract 2012;17:15–26.doi:10.1007/s10459-011-9290-7
      OpenUrlCrossRefPubMed
    64. 64.↵
      1. Lake FR,
      2. Hamdorf JM
      . Teaching on the run tips 5: teaching a skill. Med J Aust 2004;181:327–8.doi:10.5694/j.1326-5377.2004.tb06301.x
      OpenUrlPubMed
    65. 65.↵
      1. Fitts PM,
      2. Posner MI,
      3. Performance H
      . Belmont Ca. Brooks/Cole, 1967.
    66. 66.↵
      1. Li Q,
      2. Ma E-L,
      3. Liu J, et al
      . Pre-training evaluation and feedback improve medical students' skills in basic life support. Med Teach 2011;33:e549–55.doi:10.3109/0142159X.2011.600360
      OpenUrlCrossRefPubMed
    67. 66.↵
      1. Elliott SN
      . Educational psychology: effective teaching, effective learning. 3rd edn. Boston: McGraw-Hill, 2000.
    68. 66.↵
      1. Wadsworth BJ
      . Piaget’s theory of cognitive and affective development: Foundations of constructivism. 5th edn. White Plains: Longman Publishing, 1996.
    69. 69.↵
      1. Sweller J,
      2. van Merrienboer JJG,
      3. Paas FGWC
      . Cognitive architecture and instructional design. Educ Psychol Rev 1998;10:251–96.doi:10.1023/A:1022193728205
      OpenUrlCrossRefWeb of Science
    70. 69.↵
      1. Kaufman DM,
      2. Mann KV
      . Teaching and learning in medical education: How theory can inform practice. In: Swanwick T, ed. Understanding medical education evidence, theory and practice. 2nd edn. Oxford: Wiley Blackwell, 2014: 7–29.
    71. 71.↵
      1. Deci EL,
      2. Ryan RM
      . The "What" and "Why" of Goal Pursuits: Human Needs and the Self-Determination of Behavior. Psychol Inq 2000;11:227–68.doi:10.1207/S15327965PLI1104_01
      OpenUrlCrossRefWeb of Science
    72. 72.↵
      1. Belfield C,
      2. Thomas H,
      3. Bullock A, et al
      . Measuring effectiveness for best evidence medical education: a discussion. Med Teach 2001;23:164–70.doi:10.1080/0142150020031084
      OpenUrlCrossRefPubMedWeb of Science
    73. 72.
      1. Rudolph JW,
      2. Simon R,
      3. Dufresne RL, et al
      . Thereʼs No Such Thing as “Nonjudgmental” Debriefing: A Theory and Method for Debriefing with Good Judgment. Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare 2006;1:49–55.doi:10.1097/01266021-200600110-00006
      OpenUrl

    Footnotes

    • Twitter @chrisj_au

    • Contributors CEJ and JLK designed the review; CEJ created the search strategy and undertook the literature searches; CEJ and MPW undertook study screening, data extraction and quality assessment with oversight from JLK; CEJ and JLK undertook data analysis and interpreted the findings; CEJ prepared the manuscript; and all authors contributed to the final version.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement Data are available on reasonable request. Data access can be requested by contacting the corresponding author.