NHS ‘Learning from Deaths’ reports: a qualitative and quantitative document analysis of the first year of a countrywide patient safety programme

Objectives To review how National Health Service (NHS) Secondary Care Trusts (NSCTs) are using the Learning from Deaths (LfDs) programme to learn from and prevent, potentially preventable deaths. Introduction Potentially preventable deaths occur worldwide within healthcare organisations. In England, inconsistencies in how NSCTs reviewed, investigated and shared LfDs, resulted in the introduction of national guidance on ‘LfDs’ in 2017. This guidance provides a ‘framework for identifying, reporting, investigating and LfDs’. Amendments to NHS Quality Account regulations, legally require NSCTs in England to report quantitative and qualitative information relating to patient deaths annually. The programme intended NSCTs would share this learning and take measurable action to prevent future deaths. Method We undertook qualitative and quantitative secondary data, document analysis of all NSCTs LfDs reports within their 2017/2018 Quality Accounts (n=222). Results All statutory elements of LfDs reporting were reported by 98 out of 222 (44%) NSCTs. The percentage of deaths judged more likely than not due to problems in healthcare was between 0% and 13%. The majority of NSCTs (89%) reported lessons learnt; the most common learning theme was poor communication. 106 out of 222 NSCTs (48%) have shared or plan to share the learning within their own organisation. The majority of NSCTs (86%) reported actions taken and 47% discussed or had a plan for assessment of impact. 37 out of 222 NSCTs (17%) mentioned involvement of bereaved families. Conclusions The wide variation in reporting demonstrates that some NSCTs have engaged fully with LfDs, while other NSCTs appear to have disengaged with the programme. This may reveal a disparity in organisational learning and patient safety culture which could result in inequity for bereaved families. Many themes identified from the LfDs reports have previously been identified by national and international reports and inquiries.

That said, I am sure a family member of a patient that has died in England between 2017 and 2018 would struggle to read about the sobering issues highlighted by the authors. If we flip the percentages presented in the abstract alone, 11% did not report lessons learnt, 52% not sharing plans and learning within their organisations, 83% do not involve bereaved families. From their evaluation, organisations can to do better and it will be important to carry out a follow-up study to see if improvements have been made by each organisation (and collectively) over time. In some ways, the observations described here, represent some form of indicator about patient safety and just culture / openness in organisations.
The methods section will benefit from a clearer position about the study design more detail in parts. Their PPI involvement, benefitting from the support available to a larger programme of work, is very good.
The detailed description of the results is a credit to all the hard work arising from forensically examining these Quality Account reports. I think they have done a really nice job at synthesising key points, and notably specifics about the variations in how different trusts demonstrated attainment of the regulation. A consistent reporting on Introduction 1. In the opening sentence, please amend 'medical treatment' to the term 'healthcare'. Patient safety incidents arise in complex systems for a myriad of reasons and it is not always due to 'treatment' nor the involvement of 'medics' exclusively. 2. Start a new paragraph from "Healthcare organisations are made up of..." 3. Line 38-39: syntax issueconsider amending from "why deaths contributed to by problems in care happen" to "why deaths arising from problems in care occur" 4. Para 2 is helpfully very detailed but to aid readability it could be divided up from where you say "Guidance was not given on..."consider starting a new paragraph here but do revise the opening frame slightly so it is clear to the reader you are now going to describe the specifics of what support / guidance the trusts were given (or not). 5. Line 56-58, revise to "Given the lack of consistency that led to establishing the..." 6. Where you say, "and to understand if organisational learning in its truest sense is occurring" is unclear / non-specificcould this be revised to something like "to determine whether there is evidence of organisational learning apparent explicitly stated in NHS England Quality Accounts" 7. Overall, a clearer aims / objectives statement is needed. "sets out to analyse LfDs reporting" would be an appropriate way of pitching this to an audience in a colloquial way but falls short of the standards needed for reporting in an academic journal. Methods 8. Where you say "This is a qualitative and quantitative study...", I was wondering where in the manuscript you were using numerical data to evaluate this programme. This is a secondary analysis of NHS Quality Accounts produced by organisations to examine how they have adopted the principles and mandatory requirements expected following the implementation of the LfD programme. You are using a qualitative study design [content analysis] to analyse Quality Accounts which produced mainly quantitative data and in doing so you have used descriptive statistics to summarise what you observed. It is unclear how you then identified themeswas this from a more in-depth interpretive analysis of a smaller sub-sample, for example (for qualitative detail) therefore generating breath / depth of insight? 9. Amend "We excluded ambulance trusts (they are not required..." to "We excluded Quality Accounts from ambulance trusts because at the time of our analysis they were not required to report." 10. Whilst it is positive the authors have included reference to O'Brien et al. Standards for Reporting Qualitative Research and included a visual of the standards as a supplementary file. For transparency purposes, it would be helpful if they could explicitly highlight against each standard where and how in the manuscript they have addressed this or not. 11. Line 17-18, be consistent and capitalise the Q and A in Quality Account. This is a recurring problem throughout the manuscript which should be addressed because it is the name of the document central to your work. 12. It is unclear what you mean by "Data not found from the trust 2017/2018 Quality Account was not included in the analysis"do you actually mean, "We noted apparent omissions for each of the Regulations for each secondary care trust's Quality Account report.". 13. Where you say "In addition to statutorily required reporting..." make it clear you looked for that evidence because it was included in the guidance sent to trusts [mentioned in the introduction]. 14. More description is needed about the content analysis. Given the authors used an initial classification to inform a review of LfDs learning and action themes for reportingwas this more a Framework Analysis. At times, there is a subtle difference between these with the former being more inductive in nature. I am not persuaded either way due to lack of detail. Further, the statement "Quantitative analysis was undertaken and reported using descriptive statistics." is made prior to any explanation of how the data were analysedthis is really putting the cart before the horse. Please expand and provide more detail in your description of what you did and be clear on which qualitative study design and method(s) were used. That said, the detail about the initial review, followed by the remaining reports and double-coding is helpful content. 15. State the version of Microsoft Excel used. 16. Please include a statement about research ethics.

Results
17. Line 22-24: State the median and then the range in parentheses where you say "There was variation between 0 and 13% in the number of deaths..."; do this consistently, where relevant, elsewhere in results. 18. You helpfully signpost the relevant published Quality Accounts so it is clear to the reader which organisations the finding applies. You could delineate between this list of Quality Accounts (as you might, for example, when you are describing studies included in a systematic review) and tabulate these separately in order to not clutter your main reference list for the paper. You would need to adopt a roman numeral (or similar) labelling system so this was clear though. 19. In the line which starts "Trusts not using SJRs...", can the reader confidently assume trusts cited in references 34 and 35 both used CESDI and RCA and PRISM methodologies? Or did 34 do say CEDSI and RCA, whilst 35 did RCA and PRISM? 20. Check journal style, but generally write numbers <10 in full, e.g. " Table 2. The five most" change to "Table 2. The five most..." (the obvious exception is the numbering of tables and figures) 21. The themes described in the Tables 2 and 3 are very clear and read as standalone statements that do not required explanation or an operational definition. 22. Give numbers and percentages in Table 2 and 3. Discussion 23. Syntax issue in the sentence "The most common learning theme was..." 24. Consider starting "A penalty arises..." with "However, penalties can arise during CQC inspections when an assessment of the implementation of LfD is carried out." 25. Consider reviewing, and citing, Dr Maria Panagioti's systematic review from BMJ in 2019 (https://www.bmj.com/content/366/bmj.l4185.abstract) where you say "The variation in deaths judged more likely than not..." 26. A more holistic reflection of the strengths and limitations of the study is needed. You only just scratch the surface of this when you acknowledge the reasons for the underrepresentation of family involvement and caution the interpretation of that result. 27. Caution where you say "...prevent errors from occurring...."we can minimise risk by optimising human factors and the related structures / processes in systems. Suggest you amend to "minimise the risk of fatal patient safety incidents occurring". 28. "from the patients who survive against the odds" suggests they have been subject to some form of patient safety incident and the learning arising from their care experience will be of value. What I think / hope you are actually saying is more focus on patient safety incidents more broadly, and not just the deaths, is needed, as well as 'safety-II'. Given WHO released Guidelines for Incident Reporting and Learning Systems in September 2020, this would seem like a logical conclusion. We know from non-healthcare High Reliability Organisations that vast learning can be gained from understanding near-misses / no harm incidents too. 29. Please provide existing evidence of work that has tackled / is tackling 'improving communication', 'patient and family involvement' and 'dissemination of learning at a local and national level'citation of the latest evidence on interventions / solutions for these issues could be helpful for the reader.
Conclusion 30. When you say "This research shows that the LfDs programme has improved the way that NHS trusts identify, report, investigate and learn from deaths in care."it implies you had a baseline in which to make this judgement against. Does the research really show that "Organisations are variably reporting against...in terms of X, Y and Z in their Quality Accounts". Be careful not to overreach.

REVIEWER
Nanako Tamiya University of Tsukuba, Health Services Research REVIEW RETURNED 03-Jan-2021

GENERAL COMMENTS
The authors conducted a quantitative and qualitative analysis to assess how organizations are using the Learning from Deaths (LfDs) program. This is really an important attempt, but I have some concerns which should be addressed. Especially I would like to mention that mixed-method and Qualitative method is not my area and I was not sure this style of writing is acceptable in this journal or not. Some of the points I mentioned are due to the structure of this paper from my point of view as a reviewer with the Quantitative method. I would like to ask editors to judge it.
As it is written "In some qualitative traditions, the results and discussion may not have distinct boundaries" in Standards for Reporting Qualitative Research: A Synthesis of Recommendations ( https://journals.lww.com/academicmedicine/Fulltext/2014/09000/Stan dards_for_Reporting_Qualitative_Research__A.21.aspx) These styles may be acceptable but it is true that is was so difficult for me to understand the construct of this paper.

Major Comments 1. Introduction and all
For the international readers, ore explanation about "trust" which may be equivalent to "NHS secondary care trust" will be needed to understand well because this is a unit of analysis and important. 2. Methods, p6 l.24 More explanation of Regulation number may be needed. Is it written in some guidelines and how they were defined? It may be written in ref 17 but more explanation is helpful to understand these important categories.
3. Results, p8 l. 25-3 Are they free comments and each of them appeared only in one trust report which was cited? There may be some similar reasons mentioned by different trust. Which one was the most frequent comment? More information may be interesting and helpful. 4. Discussion, p.11, l.29-32. The authors discussed the possibility that the trusts with experiences of mortality review process were able to implement LfDs process more easily. Is it possible to check this hypothesis by comparing the percentage of deaths being reviewed between trusts with different experiences of mortality review process? 5. Discussion. The authors should describe the limitation of the study in the discussion section. 6. Recommendations, p.13, l.17. The authors mentioned, "Annual collection and collation of all trust LfDs reporting for wider sharing", but the term "wider sharing" seems vague. Please clarify to what extent the authors recommend the reporting should be shared. Minor Comments 7. Strengths and limitations of this study, p.4 Mentioning what "LfD" and "PPI" stand for will help readers to get the gist of the study before reading the main body of text. 8. Methods, Patient and public involvement, p.7, l.22-24 The authors used the abbreviation "PPI" in l.22, before they described "PPI" means "patient and public involvement" in l.24. 9. Results, Plans for assessment of impact, p.8, l.54-55. The authors stated "Several trusts appear to have misunderstood…", but the interpretation of the results should be done in the discussion part, not in the result part. 10. Results, Lessons learnt, p.10, l.16. The authors interpreted the result by stating "This could reduce the transferability…", but the interpretation of the results should be included in the discussion part, not in the result part. Have made change suggested 3. Line 38-39: syntax issueconsider amending from "why deaths contributed to by problems in care happen" to "why deaths arising from problems in care occur"

VERSION 1 -AUTHOR RESPONSE
Have made change suggested 4. Para 2 is helpfully very detailed but to aid readability it could be divided up from where you say "Guidance was not given on..."consider starting a new paragraph here but do revise the opening frame slightly so it is clear to the reader you are now going to describe the specifics of what support / guidance the trusts were given (or not).
Thankyou, have made change suggested.
5. Line 56-58, revise to "Given the lack of consistency that led to establishing the..." Have made change suggested 6. Where you say, "and to understand if organisational learning in its truest sense is occurring" is unclear / nonspecificcould this be revised to something like "to determine whether there is evidence of organisational learning apparent explicitly stated in NHS England Quality Accounts" Have changed to "and to determine whether there is evidence of effective organisational learning".
7. Overall, a clearer aims / objectives statement is needed. "sets out to analyse LfDs reporting" would be an appropriate way of pitching this to an audience in a colloquial way but falls short of the standards needed for reporting in an academic journal.
Have rewritten the study aim at the end of the introduction section to ensure it is more formal and clearer. 8. Where you say "This is a qualitative and quantitative study...", I was wondering where in the manuscript you were using numerical data to evaluate this programme. This is a secondary analysis of NHS Quality Accounts produced by organisations to examine how they have adopted the principles and mandatory requirements expected following the implementation of the LfD programme. You are using a qualitative study design [content analysis] to analyse Quality Accounts which produced mainly quantitative data and in 11. Line 17-18, be consistent and capitalise the Q and A in Quality Account. This is a recurring problem throughout the manuscript which should be addressed because it is the name of the document central to your work.
Thankyou, have addressed this 12. It is unclear what you mean by "Data not found from the trust 2017/2018 Quality Account was not included in the analysis"do you actually mean, "We noted apparent omissions for each of the Regulations for each secondary care trust's Quality Account report.".
Some trusts have additionally produced LfDs reports elsewhere for example as part of board papers. These were not a requirement of the LfDs programme, or a statutory requirement, but could be evaluated further (will include this as a limitation of the study). Have deleted this statement as agree it could be confusing.
13. Where you say "In addition to statutorily required reporting..." make it clear you looked for that evidence because it was included in the guidance sent to trusts [mentioned in the introduction].
Have added an additional sentence at the end of this paragraph to make this clear.
14. More description is needed about the content analysis. Given the authors used an initial classification to inform a review of LfDs learning and action themes for reportingwas this more a Framework Analysis. At times, there is a subtle difference between these with the former being more inductive in nature. I am not persuaded either way due to lack of detail. Further, the statement "Quantitative analysis was undertaken and reported using descriptive statistics." is made prior to any explanation of how the data were analysedthis is really putting the cart before the horse. Please expand and provide more detail in your description of what you did and be clear on which qualitative study design and method(s) were used. That said, the detail about the initial review, followed by the remaining reports and double-coding is helpful content.
Have given further explicit information that the document analysis (through content and thematic analysis) as described by Bowen 2009, and exploratory data analysis (as described by Stebbins 2008) were used. And that both deductive and inductive approaches were used. Agree there is some cross over with framework analysis (although this was not the type of analysis used). Have further described exactly where each type of analysis was used ( Thanks, have made change suggested 18. You helpfully signpost the relevant published Quality Accounts so it is clear to the reader which organisations the finding applies. You could delineate between this list of Quality Accounts (as you might, for example, when you are describing studies included in a systematic review) and tabulate these separately in order to not clutter your main reference list for the paper. You would need to adopt a roman numeral (or similar) labelling system so this was clear though.
I think it would be too confusing to have two referencing systems in one paper, but understand that the number of references is large and may seem cluttered because of this We have added to the strengths and limitations section and in addition reflected further on the limitations of the studying in accurately representing family engagement in the LfDs process.
27. Caution where you say "...prevent errors from occurring...."we can minimise risk by optimising human factors and the related structures / processes in systems. Suggest you amend to "minimise the risk of fatal patient safety incidents occurring".
Have changed, thankyou 28. "from the patients who survive against the odds" suggests they have been subject to some form of patient safety incident and the learning arising from their care experience will be of value. What I think / hope you are actually saying is more focus on patient safety incidents more broadly, and not just the deaths, is needed, as well as 'safety-II'. Given WHO released Guidelines for Incident Reporting and Learning Systems in September 2020, this would seem like a logical conclusion. We know from nonhealthcare High Reliability Organisations that vast learning can be gained from understanding near-misses / no harm incidents too.
Thankyou, have changed this to make it clearer 29. Please provide existing evidence of work that has tackled / is tackling 'improving communication', 'patient and family involvement' and 'dissemination of learning at a local and national level'citation of the latest evidence on interventions / solutions for these issues could be helpful for the reader.
Thankyou, we have provided references for these.
30. When you say "This research shows that the LfDs programme has improved the way that NHS trusts identify, report, investigate and learn from deaths in care."it implies you had a baseline in which to make this judgement against. Does the research really show that "Organisations are variably reporting against...in terms of X, Y and Z in their Quality Accounts". Be careful not to overreach.
Thankyou, have changed the start of the conclusion to address these concerns.

Reviewer 2 comments
Comment Action taken

Introduction and all
For the international readers, ore explanation about "trust" which may be equivalent to "NHS secondary care trust" will be needed to understand well because this is a unit of analysis and important.
To ensure consistency and ease of understanding for international readers we have replaced Hospital/Organisation/provider/trust with NHS Secondary Care trust; abbreviated to NSCT except where alternative terms are used as a direct quote 2. Methods, p6 l.24 More explanation of Regulation number may be needed. Is it written in some guidelines and how they were defined? It may be written in ref 17 but more explanation is helpful to understand these important categories.
As described in the introduction: "The reporting mechanism was built into the NHS "Quality Accounts" systemwhere NSCTs are legally required to produce a publicly available annual report about the quality of their services". We have included in parentheses "therefore forming United Kingdom government legislation" to ensure it is completely clear.
3. Results, p8 l. 25-3 Are they free comments and each of them appeared only in one trust report which was cited?
There may be some similar reasons mentioned by different trust. Which one was the most frequent comment? More information may be interesting and helpful.
The comments have been taken from the Quality Accounts of many different NSCT LfDs reports (please see references). Ideally we would include further comments, but are unfortunately limited by the word count.
The authors discussed the possibility that the trusts with experiences of mortality review process were able to implement LfDs process more easily. Is it possible to check this hypothesis by comparing the percentage of deaths being reviewed between trusts with different experiences of mortality review process?
This suggestion has arisen from the LfD reports themselves, several trusts discuss that they have been undertaking mortality reviews for many years. However the reporting on experiences of mortality review process is not consistent and it would therefore not be possible to check this hypothesis.
The authors should describe the limitation of the study in the discussion section.
Thankyou, we have done this.
The authors mentioned, "Annual collection and collation of all trust LfDs reporting for wider sharing", but the term "wider sharing" seems vague. Please clarify to what extent the authors recommend the reporting should be shared.
Thankyou, I have changed this to "that is made publicly available".