What does oseltamivir do, and how will we know?

Jonathan Nguyen-Van-Tam, a virologist and researcher from the University of Nottingham, told a group of triallists and virologists last month, “We must remember why we’re here: because of the controversies. The clinical world doesn’t believe that Tamiflu [oseltamivir] works. We should assess whether the regulatory approval/product insert for Tamiflu is valid.”

That group, the Multiparty Group for Advice on Science (MUGAS), was at a workshop in Brussels on 18 June organised by the European Scientific Working Group on Influenza and supported by an unrestricted grant from Roche, manufacturer of oseltamivir. Led by several of the original oseltamivir regulatory triallists, the workshop heard plenty of evidence to challenge current claims about the drug’s effects. MUGAS decided to plan and conduct individual participant data (IPD) meta-analyses of the randomised trial data—and observational data. That’s quite a remarkable turnaround, given the strength of claims made by some of the same people over the past decade.

BMJ readers will already be familiar with growing concerns about oseltamivir’s effectiveness. Earlier this year Harlan Krumholz and coauthors concluded in an editorial in the BMJ that, “despite government claims, we should acknowledge the uncertainty surrounding oseltamivir’s effectiveness and the gaps in publicly available evidence.”1 They continued, “On the basis of the available data, at best the drug shortens symptoms by about a day when used within the first two days of symptoms, but it has no effect on hospital admissions. In addition, trial data from which to draw conclusions about complications and transmission of flu are lacking.”

The World Health Organization made particularly firm claims about oseltamivir in August 2009 during the swine flu (H1N1) pandemic. WHO then stated, “The guidelines represent the consensus reached by an international panel of experts who reviewed all available studies on the safety and effectiveness of these drugs . . . Evidence reviewed by the panel indicates that oseltamivir, when properly prescribed, can significantly reduce the risk of pneumonia (a leading cause of death for both pandemic and seasonal influenza) and the need for hospitalization.”

That expert panel advising WHO was anonymous: all its members had signed a confidentiality agreement. But at least one member was identified later as Arnold Monto of the University of Michigan, who coauthored several trials of neuraminidase inhibitors back in the 1990s. Monto is one of the four convenors of MUGAS, along with Ab Osterhaus (of Erasmus MC University in Rotterdam, a scientific adviser to Roche and another oseltamivir triallist), Menno de Jong (of Academic Medical Center Amsterdam, a virologist and researcher), and Rich Whitley (of University of Alabama at Birmingham, professor of paediatric infectious diseases and an adviser on flu to the Obama administration).

Barry Clinch, principal clinical scientist at F Hoffmann-La Roche, presented at the MUGAS workshop an overview of all the studies in the company’s oseltamivir research programme. His slides showed that, in all, 18 928 patients had taken part in trials, observational studies of treatment, and studies of prophylaxis; around 11 500 of them were treated with oral oseltamivir. Randomised controlled trials had included 4799 adults and 1368 children; along with some open label studies, a total of 8078 patients had taken part. All but one of the 12 randomised controlled trials took place in the late 1990s. Roche’s lab researchers had also done experimental flu studies and clinical pharmacology studies.

Clinch confirmed that the investigators were told to always report admissions to hospital during the Roche trials but warned that “caution should be exercised in interpreting results on hospitalisation.” This is because there were such low event rates: for example, in trial WV15671 just one of 201 participants was hospitalised. The main intention to treat analyses in the 12 Roche randomised controlled trials (for all patients enrolled with flu-like illness and who had at least one dose of the trial drug) failed to find statistically significant evidence that hospitalisation rates were any lower in the oseltamivir group than in the placebo group. For the ITTI population (“intention to treat infected”—the subset with flu confirmed by culture or a greater than fourfold rise in antibody titre) the overall P value was 0.06, but this was grossly underpowered.

Presenting the key oseltamivir trials that yielded data on complications of flu, Clinch explained that a standard case report form was used to collect data on “secondary illness.” “We didn’t ask physicians to actively look for complications,” he said. “They simply reported them if they thought patients had, for example, sinusitis, otitis media, bronchitis, pneumonia, or other chest infections.” Clinicians could also say whether patients needed antibiotics or radiography—thereby implying there might be some secondary illness—but there was no requirement to confirm diagnoses without anything more than a clinical diagnosis. “To be honest, we weren’t that stringent at the time,” acknowledged Clinch. (And, of course, at that time Roche was testing oseltamivir primarily as another antiviral for seasonal flu, not as a lifesaver in a pandemic.) Only two trial protocols required reporting of clinically diagnosed complications (coded as bronchitis, pneumonia, lower respiratory tract complication, or antibiotic prescription occurring at least 48 hours after start of treatment with oseltamivir) as a formal secondary endpoint: those among older and at-risk participants. Statistical analyses of these outcomes were exploratory, said Clinch.

The ensuing questions and discussion among the MUGAS review board members (some of whom coauthored some of the trials in question) confirmed that “secondary illness” was indeed only a clinical diagnosis, that in most of the trials its reporting was ad hoc, and that Roche did not know the extent of any missing data. Decisions to give antibiotics were made for unspecified clinical reasons, and, as someone pointed out, antibiotic usage rates will have varied a lot from site to site because of geographical variations in prescribing behaviour.

However, some of the MUGAS virologists said that we know from Roche’s original observational studies and from subsequent case series from the H1N1 pandemic that oseltamivir does reduce the incidence of complications. Do we? Not yet, although Van-Tam’s group has conducted a systematic review and individual patient data meta-analysis of published and unpublished observational study data during the H1N1 pandemic that will be submitted soon for publication. This study was sponsored by Roche, but Van-Tam said that the data agreement gave Roche no access to the data. The authors plan, however, to share the data with other investigators on request.

When asked what proportion of the original Roche oseltamivir research programme had results in the public domain, Clinch said, “About 90% in one form or another.” The virologist Fred Hayden, coauthor of several Roche trials, asked, “The contentious area is the 8-10 unpublished trials done some time ago. Are there any plans to publish those?” Clinch replied, “It’s a good question . . . We’d like MUGAS to discuss that. Roche has no objection to that.” Osterhaus said that MUGAS wanted to analyse the individual patient data from those trials and to publish the resulting papers in peer reviewed journals.

So, of course, does the Cochrane Acute Respiratory Infections Group, which has been struggling for years to get the unpublished trial data from Roche (bmj.com/tamiflu). That data release has now begun, and the Cochrane group is poring over the partly redacted clinical study reports right now.

So, it’s from famine to feast, with two different groups working towards meta-analyses of the unpublished patient-level data on the effects of oseltamivir in flu. Will they come to the same conclusions?