The pros and cons of funnel plots as an aid to risk communication and patient decision making

Funnel plots, which simultaneously display a sample statistic and the corresponding sample size for multiple cases, have a range of applications. In medicine, they are used to display treatment outcome rates and caseload volume by institution, which can inform strategic decisions about health care delivery. We investigated lay people's understanding of such plots and explored their suitability as an aid to individual treatment decisions. In two studies, 172 participants answered objective questions about funnel plots representing the surgical outcomes (survival or mortality rates) of institutions varying in caseload, and indicated their preferred institutions. Accuracy for extracting objective information was high, unless question phrasing was inconsistent with the plot's survival/mortality framing, or participants had low numeracy levels. Participants integrated caseload-volume and outcome-rate data when forming preferences, but were influenced by reference lines on the plot to make inappropriate discriminations between institutions with similar outcome rates. With careful choice of accompanying language, funnel plots can be readily understood and are therefore a useful tool for communicating risk. However, they are less effective as a decision aid for individual patient's treatment decisions, and we recommend refinements to the standard presentation of the plots if they are to be used for that purpose.