Jump to comment:

• Further clarifications to the comments of the authors
  Guido Bertolini, Elena Garbero, Carlotta Rossi, Stefano Finazzi and Sergio Livigni
  Published on: 20 February 2020
• Response to the comments of the COMPACT-2 researchers
  Antonio Palazón-Bru, Francisco Colomina-Climent and Vicente Francisco Gil-Guillén
  Published on: 22 January 2020
• Clarifications on misreported aspects of the COMPACT-2 trial
  Guido Bertolini, Elena Garbero, Carlotta Rossi, Stefano Finazzi and Sergio Livigni
  Published on: 19 December 2019

• Published on: 20 February 2020

  Further clarifications to the comments of the authors

  • Guido Bertolini, Head, Laboratory of Clinical Epidemiology Istituto di Ricerche Farmacologiche Mario Negri IRCCS - Bergamo, Italy
  • Other Contributors:
    • Elena Garbero, Project manager, Laboratory of Clinical Epidemiology
    • Carlotta Rossi, Statistician, Laboratory of Clinical Epidemiology
    • Stefano Finazzi, Statistician, Laboratory of Clinical Epidemiology
    • Sergio Livigni, Head, Intensive Care Unit

  Dear colleagues,

  We would like to respond to your comments.

  1. We were rather concerned to learn of your confusion regarding our reasons for stopping the COMPACT-2 trial. We will try to better clarify what happened. In our report communicating our intention to interrupt the trial, we openly reported the results of our interim analysis, which were also summarized in our comment to your paper. We reiterate that we observed a close to significant difference in the trial’s primary endpoint (last hospital mortality), suggesting higher mortality in patients randomized to CPFA (59.6% vs. 41.3%; p=0.076). We observed significantly higher 3-day mortality in patients randomized to CPFA (32.8% vs. 12.5%; p=0.02). We observed higher mortality in the CPFA group at the survival analysis performed in the first 90 days after randomization (Log Rank test, p=0.03; Pet test, p=0.01). On the basis of these results, we concluded that: “The interim analysis requested by the EDSMC shows higher mortality for the CPFA group compared to the controls, particularly in the first days of treatment. Although the absolute number of treated patients is relatively small, the results raise concern that the use of CPFA may cause harm or worsen the clinical condition of septic shock patients”. In our understanding, the aim of this conclusion was to report exactly what we observed (significantly higher mortality in the CPFA group, particularly in the first days of treatment), while underlining th...

  Show More

  Dear colleagues,

  We would like to respond to your comments.

  1. We were rather concerned to learn of your confusion regarding our reasons for stopping the COMPACT-2 trial. We will try to better clarify what happened. In our report communicating our intention to interrupt the trial, we openly reported the results of our interim analysis, which were also summarized in our comment to your paper. We reiterate that we observed a close to significant difference in the trial’s primary endpoint (last hospital mortality), suggesting higher mortality in patients randomized to CPFA (59.6% vs. 41.3%; p=0.076). We observed significantly higher 3-day mortality in patients randomized to CPFA (32.8% vs. 12.5%; p=0.02). We observed higher mortality in the CPFA group at the survival analysis performed in the first 90 days after randomization (Log Rank test, p=0.03; Pet test, p=0.01). On the basis of these results, we concluded that: “The interim analysis requested by the EDSMC shows higher mortality for the CPFA group compared to the controls, particularly in the first days of treatment. Although the absolute number of treated patients is relatively small, the results raise concern that the use of CPFA may cause harm or worsen the clinical condition of septic shock patients”. In our understanding, the aim of this conclusion was to report exactly what we observed (significantly higher mortality in the CPFA group, particularly in the first days of treatment), while underlining that that this could not be taken as definitive evidence of the harmful effect of CPFA. Indeed, the second sentence of our conclusion (not reported in your citation) clearly acknowledges that we did not consider these results to be conclusive. In summary, we collected evidence of potential harm associated with the use of CPFA in patients with septic shock. For this reason, we prematurely stopped the trial for futility, acknowledging, by openly presenting the results, the possibility that the treatment might have harmful effects. We did not use the term “futility” in our communication but, when you stop a trial due to suspicion of a harmful effect, futility is out of the question.

  4. We are happy you agree that excluding from the analysis the patients assigned to the experimental arm who died before receiving the treatment, would have introduced an important bias. Indeed, our comment referred to a couple of sentences in your paper, specifically: “Since a significant number of patients died in the first 3 days and were unable to receive the technique (n=3, 15.8%) we decided to perform the analysis by actual intervention received (AIR). Although this analysis was not initially planned (clinicaltrials.gov), the fact that one out of six patients did not receive the intervention necessitated it”; and “if what we are considering is the potential harmfulness of the technique in a non-prespecified subgroup of an underpowered sample size, we cannot ignore the fact that the technique was not applied”. These sentences probably misled us. We firmly support the “intention-to-treat” approach in trial analysis and, along with many statisticians and methodologists, assign the “treatment received” or “protocol adherence” approaches to limited, specific situations, or to the assessment of the effects of protocol violations.

  5. We are pleased to hear that you do not feel our communication forced you to stop your trial. By letting you know about our results and decisions even before any official communication, it was our intention to act in the framework of the data sharing principle.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top
• Published on: 22 January 2020

  Response to the comments of the COMPACT-2 researchers

  • Antonio Palazón-Bru, Professor Miguel Hernández University
  • Other Contributors:
    • Francisco Colomina-Climent, Honorary collaborating professor
    • Vicente Francisco Gil-Guillén, Professor

  Dear Compact-2 researchers,
  Thank you very much for your comments, which we have read with great interest as they certainly contribute to the opening of an interesting debate. Below there are some points which go to clarify some issues highlighted in your letter:

  1 – We are confused by the reasons you decided to stop your trial. In your comment you have indicated that this was due to futility reasons. However, in the conclusion of the report which was uploaded to your web page, we read the following phrase: “The interim analysis requested by the EDSMC shows higher mortality for the CPFA group compared to the controls, particularly in the first days of treatment.” Furthermore, on April 27th, 2018 Medtronic issued an Urgent Field Safety Notice for the intervention product: “In a clinical study, higher early mortality (within 72 hours of randomization) was observed in septic shock patients receiving CPFA Coupled Plasma Filtration Adsorption therapy compared to patients receiving standard care. Septic shock patients often have clinical characteristics (hemodynamic instability, coagulation disorders) that increase the risk of extracorporeal treatment. Based on the preliminary data from this study, CPFA should not be used in patients with septic shock.” Finally, in the Annual Update in Intensive Care and Medicine 2019 (Chapter 29), we can read the following information about the COMPACT-2 trial. “The trial was prematurely terminated because of higher early mortality...

  Show More

  Dear Compact-2 researchers,
  Thank you very much for your comments, which we have read with great interest as they certainly contribute to the opening of an interesting debate. Below there are some points which go to clarify some issues highlighted in your letter:

  1 – We are confused by the reasons you decided to stop your trial. In your comment you have indicated that this was due to futility reasons. However, in the conclusion of the report which was uploaded to your web page, we read the following phrase: “The interim analysis requested by the EDSMC shows higher mortality for the CPFA group compared to the controls, particularly in the first days of treatment.” Furthermore, on April 27th, 2018 Medtronic issued an Urgent Field Safety Notice for the intervention product: “In a clinical study, higher early mortality (within 72 hours of randomization) was observed in septic shock patients receiving CPFA Coupled Plasma Filtration Adsorption therapy compared to patients receiving standard care. Septic shock patients often have clinical characteristics (hemodynamic instability, coagulation disorders) that increase the risk of extracorporeal treatment. Based on the preliminary data from this study, CPFA should not be used in patients with septic shock.” Finally, in the Annual Update in Intensive Care and Medicine 2019 (Chapter 29), we can read the following information about the COMPACT-2 trial. “The trial was prematurely terminated because of higher early mortality rates (within 72hrs of randomization) in septic shock patients treated with CPFA compared to patients receiving standard therapy. After this ad interim analysis, the company delivered an urgent field safety notice reporting the results.” Consequently, we think that the COMPACT-2 researchers should help to clarify the confusion generated by conflicting reports.

  2 – We recognize that we have described your method of randomization incorrectly.

  3 – It is stated by yourselves that in the ROMPA trial there was a great imbalance in the number of patients between the two branches. We had 19 patients (39%) in the intervention group and 30 (61%) in the control group. In the COMPACT-2 trial, 63 patients were treated in the intervention group (56%) and 50 in the control group (44%). Your figures appear to be close to our figures and as a consequence we do not fully understand the statement regarding great imbalance. Furthermore, we have indicated the base-line characteristics of our patients in our paper, showing that there were no clinical differences between the groups. This information is a key point in order to understand whether the groups were imbalanced or not and to date we have not seen the same key information from the COMPACT-2 trial. Finally, both studies are completely underpowered since they are far from the initial sample size calculated in the protocol.

  4 – In response to your assertion on patients dying before receiving CPFA, we would like to confirm that these patients were included in the control group as they did receive standard care. Both the contrast of hypothesis calculation for intention-to-treat or by received-intervention showed similar results. We agree that had the patients been excluded as suggested, it would have been incorrect but this was not the case.

  5 – Finally, we would like to make it clear that the COMPACT-2 authors have not directly conditioned our decision to close the ROMPA trial. However, your decision to stop your trial indirectly conditioned our work. We had a mandatory ethical duty to inform any future patient what had happened in your trial and that the product to be tested had been issued with a warning by the manufacturer (“CPFA should not be used in patients with septic shock.”) Moreover, our insurance company had to be informed and logically the conditions of the policy would have had to change. Our Ethical Committee agreed with our decision to halt the trial.

  Show Less

  Conflict of Interest:
  The authors are researchers of the ROMPA clinical trial.

  • Back to top
• Published on: 19 December 2019

  Clarifications on misreported aspects of the COMPACT-2 trial

  • Guido Bertolini, Epidemiologist Istituto di Ricerche Farmacologiche Mario Negri IRCCS - Bergamo, Italy
  • Other Contributors:
    • Elena Garbero, Data manager
    • Carlotta Rossi, Statistician
    • Stefano Finazzi, Statistician
    • Sergio Livigni, Intensivist

  We read with interest this paper reporting the results of the ROMPA trial on the efficacy of Coupled Plasma Filtration and Adsorption (CPFA) in reducing mortality in patients with septic shock. The trial was prematurely closed, after we informed the investigators of ROMPA that we had stopped our COMPACT-2 trial, designed on the same topic, for reasons of futility. COMPACT-2 (NCT01639664), whose protocol inspired ROMPA, was prompted by a pre-planned subgroup analysis coming from the overall negative COMPACT trial, suggesting that CPFA might have been effective, had a high volume of plasma been treated.
  The first planned interim analysis of COMPACT-2, aimed at assessing the feasibility of the technique, revealed a number of early deaths during CPFA. This induced the External Data and Safety Monitoring Committee (EDSMC) to request an unplanned interim analysis of safety. Such analysis, performed on the 113 recruited patients, showed significantly higher mortality in the CPFA group compared to controls, both at 3 days from randomization and at the 90-day survival analysis. We have now completed the clinical review of each recruited patient, performed through site visits by a team of independent experts, and plan to submit the manuscript to a scientific journal.
  In the meantime, we would like to clarify a couple of aspects related to our trial that were misreported in the present article and to comment on some sections of the paper. First, COMPACT-2 was not stopped...

  Show More

  We read with interest this paper reporting the results of the ROMPA trial on the efficacy of Coupled Plasma Filtration and Adsorption (CPFA) in reducing mortality in patients with septic shock. The trial was prematurely closed, after we informed the investigators of ROMPA that we had stopped our COMPACT-2 trial, designed on the same topic, for reasons of futility. COMPACT-2 (NCT01639664), whose protocol inspired ROMPA, was prompted by a pre-planned subgroup analysis coming from the overall negative COMPACT trial, suggesting that CPFA might have been effective, had a high volume of plasma been treated.
  The first planned interim analysis of COMPACT-2, aimed at assessing the feasibility of the technique, revealed a number of early deaths during CPFA. This induced the External Data and Safety Monitoring Committee (EDSMC) to request an unplanned interim analysis of safety. Such analysis, performed on the 113 recruited patients, showed significantly higher mortality in the CPFA group compared to controls, both at 3 days from randomization and at the 90-day survival analysis. We have now completed the clinical review of each recruited patient, performed through site visits by a team of independent experts, and plan to submit the manuscript to a scientific journal.
  In the meantime, we would like to clarify a couple of aspects related to our trial that were misreported in the present article and to comment on some sections of the paper. First, COMPACT-2 was not stopped simply because of higher 3-day mortality. Rather, the in-depth reasoning behind our decision took account of the significantly higher mortality at the 90-day survival analysis, the close to significantly higher hospital mortality rate (our primary endpoint), and the low conditional power for treatment benefit at the end the study. In this regard, we do not fully follow the authors’ reasoning that a statistically significant difference could be the result of random error. The statistical test is performed for the very purpose of computing the probability that a difference can be attributed to random error. Also, COMPACT-2 did not use a randomization scheme based on prognostic score. We used a blocked randomization schedule (randomly permuting blocks of four and six), with stratification according to site and the presence of septic shock at admission. It is well known that stratified blocked randomization guarantees strict balance between arms. The authors’ statement that stratified randomization “means that the groups will not be similar until the end of recruitment” contrasts with common methodological understanding. In this respect, the high imbalance between the arms of the ROMPA trial (with the control group having 50% more patients than the CPFA arm) cannot be due to the adoption of stratified randomization, irrespective of the random strata. Moreover, we strongly disagree with excluding from the analysis patients assigned to the experimental arm who died before receiving the CPFA treatment. Also in the control group it is to be expected that a similar number of patients would have died too early to receive the experimental treatment, had they been randomized to the CPFA instead of the control arm. Excluding these patients only from the experimental arm introduces a well-recognized bias, which distorts the results. Finally, we decided to inform the ROMPA researcher of our decision to interrupt COMPACT-2 for futility, given the high similarity between the ROMPA trial design and our own. We did not intend in any way to force the ROMPA researcher to adopt the same decision. We find the statement that they were “obliged to communicate” their partial results a little odd, partly considering that their Data and Safety Monitoring Committee was not independent, being formed by the principal investigator, the senior investigator and the biostatistician of the project.
  Besides these comments, we find the results of the ROMPA trial interesting, despite being based on a very limited sample size, particularly when pooled with those of COMPACT-2. Indeed, they seem to corroborate our findings.

  Show Less

  Conflict of Interest:
  The authors are investigators of the COMPACT and COMPACT-2 trials

  • Back to top