Article Text
PDF
Abstract
Objective To assess the impact of different phase-out measures approved by several European governments.
Design This is a longitudinal observational study.
Settings European countries, from 20 February 2020 to 11 May 2020.
Participants All European countries that implemented at least one phase-out measure dictated by governments, during the follow-up period.
Main outcome New COVID-19 cases, analysed as daily rate by countries.
Methods We compared the observed versus the predicted rates of new confirmed cases, hospital admission, intensive care unit (ICU) admission and deaths by regions in Spain, to assess the accuracy of the proposed generalised estimating equations and hurdle models. Based on these models, we defined and calculated two indices to quantify the impact of the phase-out measures approved in several European countries.
Results After 2-month follow-up, we confirmed the good performance of these models for the prediction of the incidence of new confirmed cases, hospital admission, ICU admission and death in a 7-day window. We found that certain phase-out measures implemented in Italy, Spain and Denmark showed moderate impact in daily new confirmed cases. Due to these different phase-out measures, in Italy, the estimated increment of new confirmed cases per 100 000 inhabitants was 4.61, 95% CI (4.42 to 4.80), in Spain 2.58, 95% CI (2.54 to 2.62) and in Denmark 2.55, 95% CI (2.40 to 2.69). Other significant measures applied in other countries had no impact.
Conclusion The two indices proposed can be used to quantify the impact of the phase-out measures and to help other countries to make the best decision. Monitoring these phase-out measures over time can minimise the negative effects on citizens.
- COVID-19
- EPIDEMIOLOGY
- Health policy
- Public health
Data availability statement
Data are available on reasonable request. Statistical code is provided as an online supplemental material.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
Predictions obtained with this approach show high reliability for a 7-day window.
This study provides two indices which allow to quantify the effects of the phase-out measures dictated by governments.
The process can be easily extended to different countries, other phase-out measures and periods.
Modelling daily cases is challenging because the available information in some countries, is not entirely accurate, particularly after the weekend.
Introduction
The COVID-19 infection has obliged governments to adopt extreme measures to minimise the impact of the pandemic.1 With the aim of dealing with the rapid expansion of the SARS-COV-2 threat, it was necessary to implement certain drastic measures such as: borders closure, not only between countries, but also between regions; movement restrictions, schools, stores and restaurants closed or the confinement of citizens. Many countries were paralysed, with important consequences in both, social and economic levels.
The limited number of beds available in hospitals and, particularly, in intensive care units (ICUs), in relation to the high number of infected people, was one of the main problems. Cities, such as Wuhan, Lombardy, Madrid, Barcelona, London, New York or Florida, among others, had to enable the construction of hospitals to assist the hundreds of patients who needed hospital healthcare.
After several restrictive lockdowns, two facts were taken into consideration for future decisions. First, the improvement of the Health System situation, as tension had decreased in hospitals and ICUs. Second, the pressure of the socioeconomic sectors, since lockdowns ordered by governments had become very unpopular, with devastating effects on workers and employers in all sectors. For these reasons, countries gradually started to relax restrictions.
In this sense, Tedros Adhamon, WHO’s director, stated ‘The risks of returning to lockdown remains very real if countries do not manage the transition extremely carefully and in a phased approach’. Following this recommendation, numerous European governments, among others, proposed different phases to ease the COVID-19 lockdown measures. These phases were progressively implemented, based on several technical reports. Due to this, many researchers worldwide are developing predictive models that can guide nations during this process.
One commonly used model is the susceptible, infected and recovered(SIR) model2 for human-to-human transmission, which describes the flow of individuals through these three mutually exclusive stages of infection. Most of the papers focus on this model and its extensions, analysing data for countries, such as China and Italy. Roda et al3 used a simple SIR model to predict the COVID-19 pandemic in Wuhan after the lockdown and quarantine of the city, between 23 January 2020 and 6 February 2020. In addition, they reported the effectiveness of the strict quarantine undertaken after 7 February 2020.
Time-varying SIR models were used by Huang et al4 under two different assumptions. First, they developed the optimal SIR model assuming that all confirmed cases in a relatively independent area, came from similar routes of transmission and had been promptly quarantined and treated, and that all close contacts of them had also been promptly tested and effectively quarantined. That is to say, the most optimistic estimate. Second, they considered an optional model which was the worst estimate, assuming that everyone was highly exposed to COVID-19. According to these two models, a prediction of the end of the epidemic in China, South Korea, Italy and Iran was given.
Giordano et al5 proposed a new mean-field epidemiological model for the COVID-19 pandemic. It was focused on Italy and it extended the classical SIR model, assuming several lockdown measures. They concluded that restrictive social-distancing measures were needed to be combined with widespread testing and contact tracing in order to end the ongoing COVID-19 pandemic.
Flaxman et al6 focused on estimating the number of infections and the impact of non-pharmaceutical interventions on COVID-19 in 11 European countries. They fitted a Bayesian hierarchical model combined with COVID-19 data, to evaluate if the adopted interventions in all these countries had had the expected effects.
Kharroubi7 fitted Poisson autoregressive models using Bayesian Markov chain Monte Carlo simulation methods to estimate the number of new confirmed cases in Lebanon to determine when containment measures can be applied and/or relaxed. Later, this model was fitted prequarantine and postquarantine by Kharroubi and Saleh8 to assess the country lockdown imposed by the Lebanese government to supress the virus transmission.
Therefore, most of the published papers related to non-pharmaceutical interventions have focused on studying the evolution of the pandemic, identifying risk factors and assessing the effectiveness of lockdown measures. However, in this study, we aim to quantify the impact of the phase-out measures. We assess the phase-out measures adopted for different European countries, through two indices based on generalised estimating equations (GEEs) and hurdle models. To our knowledge, there is no any study which considers these models to predict the number of new confirmed cases.
Methods
Datasets
European data set was downloaded from the website of the Johns Hopkins University CSSE team9 that presents data from January 2020. The Spanish data set was complemented with the information provided by the official website of the Instituto de Salud Carlos III,10 which reported daily cases from February 2020, and included data regarding hospital admissions, ICU admissions and deaths. Government measures adopted from February 2020 were provided by the Assessment Capacities Project (ACAPS).11 Additional region and country-level information was obtained from the Instituto Nacional de Estadística,12 Eurostat13 and Word Bank.14 The last day included in this study was 11 May 2020.
Individual participant data
The main challenge when fitting regression models to aggregated data from public websites, is that daily predictions at region (or country)-level requires repeated observations. Ideally, the use of individual participant data would be the best strategy to obtain valid estimates. Since this solution was not feasible, we proposed an indirect method of extracting these estimates by simulating random samples15 for each day and each location assuming a Binomial distribution of the daily number of events. The region (or country) size and the ratio between daily incidence and its size were the parameters of the distribution. If we simulated, for example, 30 random samples for each day and region (or country) this outcome could be interpreted as the observed number of new cases in 30 areas randomly selected from the same region (or country). With this approach, the reported ‘daily rates of event’ was interpreted as a random variable unlike other methods that considered this information as ‘true’ value.
Explanatory variables and outcome of interest
We considered the number of daily new cases as the primary outcome. We analysed this outcome as a count variable, that is, the rate of events per day and region (or country). To predict daily incidence, the independent variables were ‘census’, ‘region’ or ‘country’ (as appropriate) and the interaction between this last variable and ‘time’ point.
Statistical analyses
Evolution of daily incidence: assessing the accuracy of the regression models
First, we analysed the evolution of daily incidence for each Spanish region fitting a GEE Poisson regression model.16 GEE is an extension of generalised linear models to accommodate correlated data. We considered a GEE-Smooth spline model because it allows to consider more flexible functional dependence of the ‘time’ on the ‘number of events’. The ‘census’ of each region (or country) was included in the model as the offset and a logarithm function was the link function. We considered an interchangeable working matrix. A detailed mathematical description of this model is presented in online supplemental appendix A. Models were fitted using geepack17 and splines packages in R.
Supplemental material
Despite GEE approach can accommodate overdispersion in Poisson model, for some regions and countries, the data sets not only exhibit overdispersion, even an excess number of non-detected cases. The former issue can be addressed by using a negative binomial (NB) regression. The latter issue can be solved fitting a hurdle model18 19 (more details are presented in online supplemental appendix A). Hurdle longitudinal models were fitted using pscl R-package.20 Therefore, the best model between GEE and hurdle was considered in each case.
Table 1 summarises the process for the prediction of daily incidence in a 7-day window for each Spanish region. In this case, four different events were independently analysed: new confirmed cases, hospital admissions, ICU admissions and deaths. Afterwards, prediction versus observed incidence were compared daily through the mean absolute percentage error (MAPE) and the mean absolute error to evaluate the performance of the GEE and hurdle models, for almost 2 months.
Process for predicting daily incidence
Quantifying the impact of the phase-out measures on the incidence of new confirmed cases of COVID-19
Figure 1 describes the whole process. Following the first three steps presented in table 1, we fitted two models for daily confirmed cases in several European countries, M1 and M2. Both models have the same mathematical structure, that is, if GEE (online supplemental appendix A) is the most appropriate model for country A, then M1 and M2 are GEE models for that country, they differ in the follow-up. Model 1 was fitted considering only data collected before the impact of the phase out measure, while model 2 incorporated cases after the impact of the measure. The first model, M1, included observations from the beginning, to 1 week after the implemented date (ID) of the measure, phase 1 in figure 1 (online supplemental appendix A). In that period, 1 week after ID, we assumed that the effect of the measure could not be determined yet. This model M1 was used for obtaining the 7-day projections, phase 2 in figure 1. In the second model, M2, we included observations from the beginning until 2 weeks after ID, phase 3 in figure 1 (online supplemental appendix A). We used M2, for predicting the daily incidence of new cases in the whole period, phase 4 in figure 1. Finally, phase 5 in figure 1, we compared last week predictions obtained in phase 4 with the 7-day projections obtained in phase 2. The number increased (NI) was obtained as the difference between the estimated incidence rates. The NI per population (NIPP) was defined as the NI per 100 000 inhabitants. Comparing the 7-day observed incidence rate in that period, IR(7d), with the NIPP, we determined the percentage increased (PI).
Study workflow to obtain both indices. ID denotes implementation date of the phase-out measurement. Day 1 corresponds to date 22 January 2020.
Therefore, the indices, NI and NIPP, are considered to determine if the rate of virus transmission has changed or not. If these numbers are positive and large, compare to the IR(7d), then it can be concluded that the situation has been aggravated after applying that phase out measure and also how much. If these numbers are close to zero or even negative, there is no reason for concerning. Finally, PI, allows to compare between different periods and countries. An R-based code has been developed to obtain these measures. One example for Italy is presented in online supplemental appendix B.
Patient and public involvement
Patients were not involved in this research
Results
Testing the model for Spain
From 6 April 2020 to mid-May, models were fitted daily for the four outcomes: confirmed cases, hospital admissions, ICU admissions and deaths as it is described in table 1. Figure 2 shows predictions and observed cases for a week beginning 6 April 2020. The rest of events are compared in online supplemental appendix C. In online supplemental appendix D (table D1), we present the average model performance errors from 1 to 7 days window for the whole follow-up period. In particular, for confirmed cases, average MAPE ranges from 0.73% for 1 day to 3.62% for 7 days window, meaning that the average difference between 7 days prediction and the observed incidence was less than 4%.
Observed versus predicted values by Spanish regions and projection day. Observations correspond with dates 6 April 2020 until 12 April 2020. AN= Andalucía, AR=Aragón, AS=Asturias, CB=Cantabria, CE=Ceuta, CL=Castilla y León, CM=Castilla la Mancha, CN=Canarias, CT=Cataluña, EX=Extremadura, GA=Galicia, IB=Islas Baleares, MC=Murcia, MD=Madrid, ML=Melilla, NC=Navarra, PV=País Vasco, RI=La Rioja, VC=Comunidad Valenciana.
Results obtained for Italy, Germany, France and Spain
Once we checked that our models were able to predict new cases with accuracy for a 7-day window, we followed the phases described in figure 1 for assessing the impact of the phase-out measures. We chose those countries, close to Spain, that had implemented at least one phase-out measure dictated by governments, according to the information provided by the ACAPS.11 That is to say: Italy, Germany, France and Spain.
Figure 3 shows results achieved for Italy, Spain and Germany. The greatest impact was observed in Italy. On 10 April 2020, they implemented the measure: ‘companies may restart their business, if they have been communicating with the prefect and a number of security aspects are in place and certain criteria met’. The application of this measure could lead to an increase of NI=2782 new confirmed cases in a week, 95% CI (2667 to 2897). In other words: the change in the curve represents around NIPP=4.61 people more per 100 000 inhabitants, 95% CI (4.41 to 4.80). During that week (18 April 2020–24 April 2020) the observed incidence rate in Italy was 34.1, meaning a raise of PI=15.6%. That is to say, in that 7-day period, the number of new confirmed cases exceeded a 15.6% the expected without the implementation of this measure.
Models M1, M2, projection, NI (95% CI) and NIPP (95% CI) for Italy, Spain and Germany. Left side shows the whole curve from M2. Right side shows the tail of both curves. NI, number increased; NIPP, number increased per population.
Similar results were obtained in Spain, in relation to the measure implemented on 27 April 2020: ‘Children are allowed to leave their homes again under certain restrictions: children under 14 years old can take controlled walks, accompanied by an adult for 1 hour a day, during a wide schedule to avoid crowds (between 09:00 and 11:00 hours), to a distance of one kilometre around your home and without access to outdoor recreation areas for children or sports facilities’. The corresponding indices were NI=1215, 95% CI (1195 to 1234) and NIPP=2.58, 95% CI (2.54 to 2.62). During that week (5 May 2020–11 May 2020) the observed incidence rate was 20.1, which corresponds to an increase of PI=14.8% in the number of new confirmed cases.
Smaller effect was obtained in Germany NI=788, 95% CI (751 to 824), NIPP=0.95, 95% CI (0.91 to 0.99) when on 20 April 2020 they implemented the measure: ‘Stores with less than 800 m2 can reopen; irrespective of size, also car dealers, bike shops and book shops allowed to reopen; all under strict hygiene regulations’, PI=11.9%.
However, the measure analysed in France implemented on 20 April 2020 ‘Visits to elderly homes will be allowed again under certain restrictions’ had no impact. In online supplemental appendix E, we present figures for France and the other measures analysed in Italy and Spain with slight or no impact.
Results obtained for other European countries
Table 2 shows the results obtained for those European countries where significant measures dictated by governments had been applied during the follow-up. Figures are included in online supplemental appendix F.
Summary of the indices obtained for several countries that had implemented significant measures dictated by governments, during the follow-up
As it can be observed, only indices for Denmark were positives. We observed that NIPPs were 2.54, 95% CI (2.40 to 2.69) (PI=15.5%) and 2.14, 95% CI (2.06 to 2.22) (PI=14.7%). They corresponded with the following measures: ‘Slow increase of public transport network, in combination with gradual opening’. ‘“Companies may bring employees back from home office, under certain circumstances and with certain measures’. ‘Day cares and primary schools to reopen’. ‘Gradual opening for a number of providers, including psychologists, physiotherapists, private hospitals and clinics, etc’.
Discussion
This study found that some phase-out measures executed in countries such as Italy, Spain or Denmark might have had moderate impact on daily new confirmed cases. The greatest impact was observed in Italy, followed by Spain. Smaller impact was achieved in Germany and the measure analysed in France had no impact.
We have shown that, either GEE or hurdle models are flexible but robust and no strong assumptions are needed, unlike SIR models which may not be met in real scenarios. In addition, the method proposed in this paper is fast and efficient. Results show that more complex models may not be necessary because 7-day predictions are quite good. We attempted to fit alternative models such as generalised additive models (GAM) or generalised mixed models but they did not converge. Other techniques such as spatial temporal models led us to good predictions for the next 2 or 3 days but failed in 7-day window estimations. In a recent collaborative project developed in Spain, 45 different models have been combined to build a meta-predictor on the spread of the pandemic. The aim was to provide the health authorities with accurate information on the short-term behaviour of the significant variables. We confirmed that our model had good overall performance when predicting daily incidence, in fact it is the best one estimating the event ICU admission and the third one estimating hospital admission. Note that these two outcomes are very important to clinicians, hospital administrators and healthcare policy makers. In previous studies,21 22 we found that GEE models were a good alternative to the standard methodology in longitudinal studies but in completely different scenarios. In addition, the process presented in this paper to assess the impact of the phase-out measures, can be adapted to alternative models if their performance is appropriated. This can be done by following the phases described in figure 1.
Both indices NI and PI could have been calculating by comparing directly 7-day predictions, phase 2 in figure 1, with the observed incidence for that week, however, we consider that fitting the same model twice, with and without that window, homogenises predictions and makes them more comparable.
Another alternative, could be to assess the impact of the phase-out measures modelling the rate of deaths, which is easier. Nevertheless, estimating daily new confirmed cases has the advantage that a shorter period is needed, avoiding delays in the case that an intervention plan should be necessary. An analysis performed by the Institute for Health Metrics and Evaluation23 projecting deaths due to COVID-19 has been quite controversial. They approximate the shape of the epidemic curve from China and Italy and they extrapolate it elsewhere, assuming that the effects of lockdowns are homogeneous everywhere.24 In our study, we fit different models for each country, and we have not included, parameters associated with the COVID-19 transmission, as they are poorly reported.
The literature assessing the effect of relaxing restrictions on new daily cases is limited. Most of the papers focus on predicting the end of the epidemic and/or the assessment of the adopted lockdown measures.3–8 However, to our knowledge, no studies quantifying the impact of the phase out measures have been published. The aim of this paper is to fill this gap. It is very important to have a follow-up of the effects of relaxing some measures in a short-time of period since their implementation in order to be able to take actions in case it was necessary.
It should be pointed out that our methodology is proposed to provide an estimation of the possible effect of relaxing the restrictions. Unfortunately, we cannot determinate whether the differences are due to the adopted measures, or to other factors which cannot be controlled. Furthermore, it may not be fully realistic to assume that the effect of a measure is the same in all countries. Our predictions show high reliability for a 7-day window, although this approach should not be used for a much longer period.
Conclusions
Based on the results obtained in this study, monitoring phase-out measures over time can minimise the negative effects on citizens. The methodology described in this study is a quick way to check their impact, and could guide governments about when and how to adopt the most appropriate decision to establish preventive public health policies.
Data availability statement
Data are available on reasonable request. Statistical code is provided as an online supplemental material.
Ethics statements
Patient consent for publication
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors MCP: conceptualisation, methodology, data curation, software, formal analysis, validation, supervision, writing—original draft, writing—review and editing. TP: conceptualisation, methodology, data curation, software, formal analysis, validation, supervision, writing—original draft, writing—review and editing. Both authors act as guarantor for the overall content.
Funding This work has been partially supported by Ministerio de Ciencia e Innovación of Spain, grant number PID2019-104681RB-I00.
Disclaimer The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.