Discussion

Across the 16 studies judged to be eligible, the majority of data was obtained from (30%, 4934/15817) and most requests (44%, 757/1732) were made to Primary Care Trusts (PCT). A total of 15 817 pieces of data were retrieved by the combined studies and they covered areas as diverse as litigation, surgical provision and surveillance, costs and funding of rarer cancer, laboratory provision, pharmacological safety and midwifery discipline. The median number of pieces of data received per study was 387, a relatively large dataset, especially given the ease of access to it. The response rate was 86% (1480/1732) for all included studies. This is comparatively favourable than other methods of data collection.23 ,24

Three studies reported the reasons why they felt data were not obtained, but none provided actual reasons.9 ,17 ,18 One study reported the cost and commercial interests as being barriers to full information disclosure.18 One other study felt there were difficulties in interpreting the request by the bodies.9 Finally, two studies felt that trusts were unable to compile the data due to unavailability of information.17 ,18 The average response time of 29 days is longer than the 20 working days outlined in the Act but only three articles reported response time and did not clarify if they were working days.1

The positive correlation between the number of data obtained and the number of requests made is to be expected. Obviously, as the number of sources from which data are requested increases, the number of data obtained from such requests should increase. The improvement in response rate as more requests are made indicates that the large majority of providers respond well and the effect of those that do not is reduced. As such, more data may be obtained by maximising the number of requests made, as may the response rate.

No other feature was significantly associated with obtaining more information, but the lack of a significantly negative correlation between the number of questions asked and the response rate indicates that there is no optimal number of questions. This was examined to see if higher numbers of questions led to a reduced response rate. It is possible that by asking too many questions, the cost limit of £600 outlined in the Act might be reached and therefore data not obtained. We cannot comment on the optimal number of questions or of requests. To encourage data collection, it seems sensible that questions should be carefully considered, stated clearly and simply. Before a request is made, a thorough search of the existing national databases and the public bodies own website should be made to determine if such information is already in the public domain.

Questions were being asked across a cross-section of NHS, indicating that these studies are investigating the wider scope of NHS. Previously, obtaining such information apart from official reports and documents was challenging. One of the included articles attempted to obtain data by a simple letter, and achieved a response rate of 11%; this improved to 83% after making the FOI Act request.19 A similar act passed in the USA in 1966 led to a vast public health and medical research output.25 The FOI Act in the UK now enables researchers to access a similarly large amounts of data pertaining to the entirety of the health service at minimal expense—both financially and with respect to time for researchers.

While the use of FOI may be financially beneficial for those performing research, questions have been raised about the collective cost of the FOI Act.26 ,27 Cost is one of the exemptions made in the Act, where a request must cost under £600 to respond to.1 If requests are made to multiple trusts then the cost of the research to the NHS would be larger, rather than relying on the funding received by researchers.14 A report compiled investigating the financial impact of the FOI Act estimated that over £24 million was spent by the central government on FOI requests in 2006.3 Requests going to those outside of central government (including Social Health Authorities) were estimated to cost £8 million. Such estimations are limited due to the variable local costs in response to requests.3 If every request included in this analysis cost half the recommended amount of £600, then the total cost to the NHS of such research would be over £450 000.

There have been many criticisms of the FOI Act. These include the ease of ‘fact fishing’ where huge amounts of data can be obtained and trawled for any discrepancy.28 As with any retrospective analysis, proper methodology with a hypothesis formed before obtaining data should help allay such concerns for medical literature. Indeed, such data mining may lead to serendipitous results. There have been many prominent critics, including many members of the government that introduced the initial white paper.29 Former Prime Minister Tony Blair himself described it as a ‘dangerous act’, and while he felt it would empower people by shining ‘light’ on the government and making them more accountable, it has had some unintended consequences, such as the extensive use by the media to investigate all aspects of the government.30

A report by a former senior civil servant into the FOI Act found that it had improved transparency and accountability, but had reduced trust and participation in government.31 The importance of government being able to debate in confidence, to allow civil servants to give frank analysis and the negative reporting employed by the mass media were other key issues that the former Prime Minister raises in his memoirs.17

In the era of open access research and preregistration of trial information and protocols on publically accessible databases, having raw data available openly is a key aspect of academic freedom.32 In an increasingly connected world, data being easily available allows timely dissemination of important advances and findings. Having data available for scrutiny is a key part of the scientific method, just as having a protocol and ethics are. Data should be made available for independent assessment and analysis; the BMJ has made this a requirement for clinical trials of drugs and devices as of 2013.33

With a universal ease of access to data comes the issues surrounding research ethics and access to data obtained by research. The most prominent example of this was a request made by a major tobacco conglomerate in 2011. They requested data from a group at the University of Stirling who were investigating the opinion of teenagers about plain packing.34 ,35 Academics argued that releasing such information could jeopardise future work if young people were thinking that their opinions were being used to form the tobacco industry's marketing campaigns. If forced to disclose, as they eventually were, they argued it would have vast implications on academic freedom.22

One prominent criticism of the FOI Act is the provision for ministerial veto.36 This came to light with regard to a request for access to the NHS ‘risk register’. A document was prepared outlining the potential damage caused by NHS restructuring under the white paper of 2010.37 This led to the questions surrounding what data and predictions were contained within the document and worries related to the NHS changes.38 This is perhaps the most worrying aspect of the FOI Act for healthcare researchers. That is the highly politicised nature of the NHS and the potential political problems of such a document's release. Data that might be useful to researchers could be held back to prevent such embarrasment.9 ,17

The FOI Act gives access to a plethora of data and the potential of using such data is only as limited as the questions posed by researchers. In this review, the FOI Act was primarily used to investigate the metrics of performance across large portions of the NHS but should not be limited to this. Research utilising the FOI will become especially important for monitoring the impact of changes to the structure and practice of the whole NHS on outcomes following from the reforms of the NHS Bill 2012.24 The possibility of utilising the FOI act itself to ask about how many FOI requests had been made for the purposes of research was considered. However, the condition that there is no requirement to state the purposes for requesting information and the broad, heterogeneous bodies from which this information were requested means this would likely have been futile.

Greater transparency itself can help improve outcomes. For instance, Bridgewater et al39 demonstrated that publication of surgeon-specific mortality data following the Bristol inquiry led to a reduced mortality in cardiac surgery. A greater transparency in a wide range of medical specialities may well have a similar impact and FOI could be an important tool facilitating such transparency as part of a wider information revolution.26 ,40

This analysis is limited by the fact that the actual number of studies utilising the UK incarnation of the FOI Act for healthcare research are relatively few and there was inclusion of only English language papers. The lack of studies located limit the analysis of the FOI Act's value as a research tool. The impact of language discrimination is likely to be minimal, given the research pertained to the UK FOI Act. Inclusion of articles utilising the Scottish Freedom of Information Act—as opposed to the UK FOI Act—may have yielded further examples of FOI use.41 It is also possible that research that had no findings or found it very difficult to obtain information would not be submitted or accepted for publication and as such there may be a certain level of publication bias. Authors may also not describe their work as utilising the FOI Act for data collection, therefore meaning that research using the Act may not be characterised as doing so.28

Whether or not a paper was true research, rather than just making large scale FOI requests and then scouring the data for inconsistencies was hard to distinguish, and was based on whether or not the article outlined it's research methodology. This may have excluded some valid studies, although given their lack of described methodology; such papers would likely have been of poor quality. Another difficulty was in extraction of data; specifically discerning the number of data obtained. For example, one article reported over 800 000 data relating to Clostridium difficile infections. However, this was the answer to a question, rather than obtaining 800 000 individual data points, so was not included in the number of data obtained.5 ,9 Finally, we had no way of assessing the quality of data collected. While there is a legal requirement to disclose accurate information, the methodology of the initial collection of the data may not be uniform and therefore raises unknown biases within the data set.

To allow greater assessment on how the FOI Act has been used, the patterns of data distribution and to standardise the data collection, a formatted checklist would be beneficial. This would hopefully result in future studies of this kind having equivalent methodology to allow better comparison; and therefore more significant conclusions, to be drawn from the data. This would ideally include:

1. Which body/bodies data were requested from?

2. How many and what questions were asked?

3. The rate and timeframe of responses

4. Reasons given when data were not released

5. Amount of data received