Effect of lifting COVID-19 restrictions on utilisation of primary care services in Nepal: a difference-in-differences analysis

Introduction An increasing number of studies have reported disruptions in health service utilisation due to the COVID-19 pandemic and its associated restrictions. However, little is known about the effect of lifting COVID-19 restrictions on health service utilisation. The objective of this study was to estimate the effect of lifting COVID-19 restrictions on primary care service utilisation in Nepal. Methods Data on utilisation of 10 primary care services were extracted from the Health Management Information System across all health facilities in Nepal. We used a difference-in-differences design and linear fixed effects regressions to estimate the effect of lifting COVID-19 restrictions. The treatment group included palikas that had lifted restrictions in place from 17 August 2020 to 16 September 2020 (Bhadra 2077) and the control group included palikas that had maintained restrictions during that period. The pre-period included the 4 months of national lockdown from 24 March 2020 to 22 July 2020 (Chaitra 2076 to Ashar 2077). Models included month and palika fixed effects and controlled for COVID-19 incidence. Results We found that lifting COVID-19 restrictions was associated with an average increase per palika of 57.5 contraceptive users (95% CI 14.6 to 100.5), 15.6 antenatal care visits (95% CI 5.3 to 25.9) and 1.6 child pneumonia visits (95% CI 0.2 to 2.9). This corresponded to a 9.4% increase in contraceptive users, 34.2% increase in antenatal care visits and 15.6% increase in child pneumonia visits. Utilisation of most other primary care services also increased after lifting restrictions, but coefficients were not statistically significant. Conclusions Despite the ongoing pandemic, lifting restrictions can lead to an increase in some primary care services. Our results point to a causal link between restrictions and health service utilisation and call for policy makers in low- and middle-income countries to carefully consider the trade-offs of strict lockdowns during future COVID-19 waves or future pandemics.

This study presents a contribution to the literature on the indirect impacts of the COVID-19 pandemic on health services. Several previous studies compare service volumes during periods with different levels of COVID-19 restrictions. The use of the DID approach, while not perfect, provides a more convincing estimation of the effect of restrictions on utilization. The authors are very transparent about the assumptions and the limitations of the approach. The analysis of data prior to the lifting of the restrictions supports the use of the approach. The paper is also very clear and well organized. I do, however, have a few comments that I hope can be helpful.
1. Which types of facilities report to the HMIS? Does it cover private facilities? Is data from hospitals included as well?
2. Why are deliveries not included in the analysis? A study that found that there were fewer deliveries in Nepal during the pandemic is cited. Therefore, it seems important to test whether deliveries increased once restrictions were lifted. Delivery services are a good example of a service that cannot be delayed, unlike other services. For example, women can delay initiation of antenatal care rather than forgo care. Therefore, responsiveness to restrictions and their removal could be different.
3. Can the three types of restrictions be separated? The current definition of restrictions combines stay-at-home orders, business/workplace closures, and limits on public transportation. I assume that workplace closures should have the least impact on service utilization and their inclusion might weaken the estimated impacts. If the data allows to separate the type of restrictions, the analysis could also provide clearer policy implications.
4. Restrictions are defined to be implemented in a municipality if any type of restriction was implemented for at least 10 days? Why was this threshold chosen? This implies that some treated municipalities could have no restrictions for most of the month, thus "diluting" the treatment. Have you tried a higher threshold? Alternatively, can you define the treatment as the number (or share) of days during a month in which restrictions were implemented? 5. Why did some municipalities choose to lift the restrictions while others didn't? Is there any information on that in government documents or in the press? This could be helpful in terms of thinking of time-variant confounders. I also think it would be informative for readers to have a comparison of the two groups of municipalities in terms of their characteristics (population size, urban/rural,…) and number of COVID cases. 6. I do not understand the argument that reproductive, maternal and child health services are more resilient than other types of primary care services. Couldn't the significant relationship between lifting the restriction and the volume of the services suggest that these services are more sensitive to the mobility restrictions? The paragraph in the Discussion section makes the case that these services are resilient because they rebounded. However, the paper doesn't show which services were at all impacted by the pandemic and the initial introduction of the restrictions. 7. In the Conclusions section of the abstract, the authors write that the results call for policy makers to "…prepare health systems for potential rebounds in service utilization as restrictions are lifted." I think it would be good to be clearer about what is meant by "rebound". In what ways should health systems need to prepare? Does it imply that utilization increased above pre-pandemic levels when the restrictions are just removed?

GENERAL COMMENTS
Overall reflection: Although the authors have brought up a sensible context suggestive of common sense in health policy making, the conclusion is yet inconclusive.
There is still scope to improve the quality of English, which is why it is recommended for further editing, preferably by a native speaker.
It makes difficult for the reviewer to make specific comments in the specific section as line numbers are missing. news sources? Could you please be specific on this, which news portal/headlines/COVID update section and cite them with a date or date range? The authenticity of the news! (As per journal guidelines, these must be cited.) Who were the local researchers? Are they acknowledged? Potential conflict of interest!!! #5 We aimed to include 12 primary care services………….
The following model was used and repeated for each of the 10 health service analyzed,……………… Comment: The above two sentences are contradictory.
What was the sampling method used in the study? Please clarify. If sampling was not done this must be highlighted within limitation. Were all the palikas included or there were some which were excluded? There doesn't seem any focus to this. Please clarify this in the methods. What was the reason for selecting 10 services? Were two services excluded? If so, please elaborate on why. Services like female community health volunteers (very active within the community) and mental health were not included. It would have been interesting to see that data as well, as there are multiple reports of mental health issues amid the COVID pandemic and lockdown. If these were not covered, then they must be included in the limitation segment. Results: #6

Reviewers' Comments:
Reviewer #1: The authors estimate the effect of lifting COVID-19 restrictions on health service utilization in Nepal. Using difference-in-difference approach, they compare trends in service volumes in municipalities that lifted restrictions (treatment group) to those that kept enforcing restrictions (control group). The identification of the impacts relies on the assumption that if restrictions would not be lifted in the treatment group, then utilization would follow the same trend as in the control group. The authors find that lifting the restrictions caused an increase in three out of the 10 analyzed services, in a range of 9 to 34 percent.
This study presents a contribution to the literature on the indirect impacts of the COVID-19 pandemic on health services. Several previous studies compare service volumes during periods with different levels of COVID-19 restrictions. The use of the DID approach, while not perfect, provides a more convincing estimation of the effect of restrictions on utilization. The authors are very transparent about the assumptions and the limitations of the approach. The analysis of data prior to the lifting of the restrictions supports the use of the approach. The paper is also very clear and well organized.
I do, however, have a few comments that I hope can be helpful. 2. Why are deliveries not included in the analysis? A study that found that there were fewer deliveries in Nepal during the pandemic is cited. Therefore, it seems important to test whether deliveries increased once restrictions were lifted. Delivery services are a good example of a service that cannot be delayed, unlike other services. For example, women can delay initiation of antenatal care rather than forgo care. Therefore, responsiveness to restrictions and their removal could be different. RESPONSE: Thank you for this question. The focus of the paper was on primary care services, which is why we excluded deliveries (which are generally considered as part of tertiary care). To address the reviewer's question and because we had access to this data, we ran the analysis on the number of deliveries performed. We found no statistically significant effect from lifting restrictions. Looking at the graph below, deliveries appear to be increasing similarly in both groups of palikas (those that lifted restrictions and those that maintained them) in the post period.
3. Can the three types of restrictions be separated? The current definition of restrictions combines stay-at-home orders, business/workplace closures, and limits on public transportation. I assume that workplace closures should have the least impact on service utilization and their inclusion might weaken the estimated impacts. If the data allows to separate the type of restrictions, the analysis could also provide clearer policy implications. RESPONSE: Thank you for this suggestion. Unfortunately, these policies were almost always implemented together which would prevent us from separating them in the analysis. In addition, the sources used to track the implementation and lifting of restrictions by palika were largely qualitative and occasionally lacked this level of precision. We revised the methods section to clarify the process used for tracking restrictions and creating the exposure variable (lines 151 to 161). We also added to the limitations section (lines 355 to 357) to address this point.

4.
Restrictions are defined to be implemented in a municipality if any type of restriction was implemented for at least 10 days? Why was this threshold chosen? This implies that some treated municipalities could have no restrictions for most of the month, thus "diluting" the treatment. Have you tried a higher threshold? Alternatively, can you define the treatment as the number (or share) of days during a month in which restrictions were implemented? RESPONSE: Thank you for these suggestions. Although this might be an interesting sensitivity analysis to add, as mentioned above, our sources for the types of policies in place and the dates of implementation and lifting occasionally lacked this level of precision. We have described the policy tracking process more clearly in the revised manuscript (lines 151 to 161) as well as clarified this in the limitations section (lines 355 to 360).
5. Why did some municipalities choose to lift the restrictions while others didn't? Is there any information on thain government documents or in the press? This could be helpful in terms of thinking of time-variant confounders. I also think it would be informative for readers to have a comparison of the two groups of municipalities in terms of their characteristics (population size, urban/rural,…) and number of COVID cases. RESPONSE: Thank you for this question. According to news sources and local co-authors, districts or palikas that implemented restrictions after the national lockdown was lifted often did so because there were COVID cases found in the palika. This is why we chose to control for the number of new COVID cases. To address the reviewer's comment, we revised table 1 and added the average palika population size and number of COVID cases in the treated vs. control groups (line 221). In addition to the average number of visits per palika and month for each service, these data indicate that palikas in the treated group tend to have smaller populations, less COVID and serve fewer patients. They are therefore more likely to be rural. We commented on this in the results and discussion sections (lines 216 to 218 and 348 to 352).
6. I do not understand the argument that reproductive, maternal and child health services are more resilient than other types of primary care services. Couldn't the significant relationship between lifting the restriction and the volume of the services suggest that these services are more sensitive to the mobility restrictions? The paragraph in the Discussion section makes the case that these services are resilient because they rebounded. However, the paper doesn't show which services were at all impacted by the pandemic and the initial introduction of the restrictions.

RESPONSE:
Thank you for bringing up these points. We agree that this argument might not be appropriate given our study design and have revised this. We instead highlighted the fact that many interventions implemented by the Government of Nepal and partners during COVID have focused on maintaining maternal and child health services specifically (lines 325 to 333). This might explain the increase in utilization of these services after restrictions were lifted, while other services were not directly targeted by these interventions. We also cited a previous study in the Background and discussion sections (lines 321 to 323) that estimated the magnitude of disruptions in several services following the declaration of the pandemic and found that MCH services were less affected than other services like NCDs.
7. In the Conclusions section of the abstract, the authors write that the results call for policy makers to "…prepare health systems for potential rebounds in service utilization as restrictions are lifted." I think it would be good to be clearer about what is meant by "rebound". In what ways should health systems need to prepare? Does it imply that utilization increased above pre-pandemic levels when the restrictions are just removed? RESPONSE: Thank you for highlighting this. We agree that "rebound" might not be the appropriate term and have changed this to "potential increases" (line 27). We do know that these services were disrupted at the start of the pandemic, as mentioned above, but they do not appear to have surpassed pre-pandemic levels after restrictions were lifted.

Reviewer #2:
Overall reflection: Although the authors have brought up a sensible context suggestive of common sense in health policy making, the conclusion is yet inconclusive.

RESPONSE: Thank you for highlighting this important point. We have revised the conclusion section. Please see answers to comments 6 and 7 of reviewer 1.
There is still scope to improve the quality of English, which is why it is recommended for further editing, preferably by a native speaker.

RESPONSE: Thank you. Co-authors have reviewed the text and edited any grammatical errors.
It makes difficult for the reviewer to make specific comments in the specific section as line numbers are missing.

RESPONSE: Thank you. We have now added line numbers in hopes that this makes the review process easier. The line numbers that follow are based on the unmarked manuscript.
The format of the date is not consistent throughout the manuscript. It is recommended to write the English date first followed by the Nepali date within parenthesis.

RESPONSE:
Thank you for pointing this out. We have updated this throughout the manuscript where appropriate.

Specific Comments:
Abstract: Conclusion #1 Despite the ongoing pandemic, lifting restrictions can lead to an increase in reproductive, maternal, and child health service utilization. Our results call for policy makers in low-and middle-income countries to carefully consider the tradeoffs of strict lockdowns during future COVID waves or future pandemics and prepare health systems for potential rebounds in service utilization as restrictions are lifted.
Comment: The service utilisation (average visit numbers) during the COVID lockdown seems higher as compared to post-lockdown. In what way is it said it may lead to an increase? Please justify how it will build trust with stakeholders and policymakers who believe this statement.

RESPONSE:
Thank you for this question. According to table 1, the average number of visits per palika and month is generally lower in the pre-period compared to the post-period, although there is some variability. For example, in the pre-period, palikas in the treatment group conducted an averaged of 88 HIV tests compared to 126 in the post period. Nonetheless, the main findings of an increase in some services after lifting the restrictions are obtained using a difference-in-differences model where we adjust for time-fixed confounders and secular trends. We're confident that these methods are appropriate and the main findings in table 2 allow us to drathese conclusions.

To improve interpretability for stakeholders and policy makers we revised the conclusion section and have also added a "Strengths and Limitations of the study" section at the beginning of the manuscript as requested by the editor.
Background: #2 On March 24, 2020, the Government of Nepal implemented a country-wide lockdown with strict restrictions on movement of people and closure……….
Comment: This paragraph looks weird here.

RESPONSE:
We have rephrased these sentences to make the paragraph clearer (lines 81 to 88).

#3 Palika is referred as municipality government
Comments: Palika is made up of both municipal and rural municipal governments.

Methods:
#4 These data were validated using the Nepal COVID Crisis Management Coordination Center (CCMCC) government websites……………..additional online news sources…… reviewed by local researchers. Comment: 1. news sources? Could you please be specific on this, which news portal/headlines/COVID update section and cite them with a date or date range? The authenticity of the news! (As per journal guidelines, these must be cited.)

RESPONSE: Thank you for noting this. We revised the methods section to clarify the process used to compile the data on restrictions implemented and lifted per palika (lines 151 to 161). We have cited the primary sources in the manuscript and additional news sources used in supplementary material table 1 for improved transparency and reproducibility.
2. Who were the local researchers? Are they acknowledged? Potential conflict of interest!!! RESPONSE: Thank you for noting this. The local researchers are co-authors of this paper. Three of the seven co-authors are Nepali researchers or policy makers. In the revised methods (line 153) we clarified that the local researchers are co-authors. The first author did the first extraction which was revised thoroughly by the three Nepali co-authors.
#5 We aimed to include 12 primary care services…………. The following model was used and repeated for each of the 10 health service analyzed,……………… well, as there are multiple reports of mental health issues amid the COVID pandemic and lockdown. If these were not covered, then they must be included in the limitation segment.

RESPONSE:
Thank you for mentioning these important services. Unfortunately, we were limited by the services available in the Nepal DHIS2. We have now added this as a limitation (lines 366 to 368). Results: #6 Table 1 shows t Comment: 1. It is very unusual to see that the result starts directly with an illustration.

RESPONSE:
We revised this section in the manuscript to present the other text first (line 213).
2. Demographic information about patients who used health care services is lacking.

RESPONSE:
Thank you for noting this. The health service utilization data currently available in the Nepali DHIS2 are only at the palika or health facility level and we do not have any information on the patients using these services. Nonetheless, in the revised table 1, we added information on the demographic characteristics of the palikas, notably the population sizes and number of COVID cases.

Discussion:
#7 Finally, pneumonia is one of the leading causes of death for children……….. Comment: The common cold has a similar presentation to COVID initially. This may have led to postponing the visit during restriction and increasing the severity of the disease.

RESPONSE: Thank you for noting this. We have added a sentence that highlights this point (381-382).
Conclusion: Please rewrite the conclusion, as it may confuse readers from various fields. RESPONSE: Thank you. As described in the first comment, we have revised the conclusion in the abstract and the discussion section. In addition, we believe that the reorganized and revised methods, the added characteristics of palikas in table 1, and the additional information on the number and types of health facilities analyzed, will help improve clarity. 1. It is surprising to see that despite restrictions, patient visits within palika are consistently higher.

RESPONSE:
Thank you for noting this. This is likely because the palikas that maintained the restrictions in place were generally larger palikas. As described in the revised table 1, the control palikas that maintained restrictions had higher average health service visits, larger populations and more COVID cases.
We describe at lines 218-219 that the palika fixed effects in the DID design will account for these differences across the two groups.

GENERAL COMMENTS
Abstract and main Conclusion: The conclusion still says that: "Our results call for policymakers in low-and middle-income countries to carefully consider the tradeoffs of strict lockdowns during future COVID waves or future pandemics and prepare health systems for potential increases in service utilisation as restrictions are lifted." But, given that the results show that service utilisation increased primarily during the COVID restrictions, how can the authors conclude that "health systems should prepare for potential increases in service utilisation as restrictions are lifted?" In common sense, palika health centres are in the community, and if there are restrictions on travel, then people are bound to visit there, but if the restrictions are eased, they go to seek service at higher centres. This is the reason why there was an increase when there was a restriction.
Suggestion: Conclude by strengthening community health systems so that they are able to provide standard care despite hard times like the pandemic.

Introduction:
Why is all this information in line numbers 81-88 in the introduction? description of palika and things that can go into methodology.

Methods:
Line 116: Comment: It is still unclear why this number of services was included. Aim says 12 but the data says 10. Elaborate on the question below as well.
What was the reason for selecting 10 services?
The below quoted comment is not addressed.
Services like female community health volunteers (very active within the community) and mental health were not included. It would have been interesting to see that data as well, as there are multiple reports of mental health issues amid the COVID pandemic and lockdown. If these were not covered, then they must be included in the limitation segment. In addition, our data included all health facilities in Nepal including community health centers as well as higher-level centers. A switch in facility types would therefore not affect our study as the total number of services provided in the country as a whole would remain the same.
We appreciate the reviewer's comment that community health centers should be strengthened to provide essential care during pandemics or other shocks. We have added a sentence on this in the main text conclusion at lines 410-412.

Introduction:
Why is all this information in line numbers 81-88 in the introduction? description of palika and things that can go into methodology.

RESPONSE: Thank you for this comment. Based on the reviewer's suggestion, we have shortened this paragraph in the introduction and moved the information to the methods section.
Methods: Line 116: Comment: It is still unclear why this number of services was included. Aim says 12 but the data says 10. Elaborate on the question below as well.
What was the reason for selecting 10 services?

RESPONSE:
Thank you for this comment. We aimed to include all 12 primary care services. In the results, we explain that/span>two services (visits for child diarrhea and pentavalent vaccinations) had to be excluded due to violation of the parallel trends assumption, which is a requirement for difference-in-differences estimation (lines 243 to 245). If trends in an outcome are not parallel in the pre-intervention period, the control group is not a good counterfactual for the treatment group in the post period. Therefore, these two services cannot be analyzed using a DID design.
In the supplementary materials, we provide a In the methods (lines 118-121), we also added a sentence describing the selection of these 12 services.
The below quoted comment is not addressed.
Services like female community health volunteers (very active within the community) and mental health were not included. It would have been interesting to see that data as well, as there are multiple reports of mental health issues amid the COVID pandemic and lockdown. If these were not covered, then they must be included in the limitation segment.

RESPONSE:
We apologize for missing this comment in the first revision. We agree these services would be very interesting to study, but unfortunately, they are not included in the Nepali DHIS2. In the Discussion section we had previously mentioned the limitations of the Nepali DHIS2 and that it does not include data on several important services including mental health care (lines 369-370).
Based on the reviewer's suggestion, we added a sentence on the fact that home-visits from community volunteers were not included (lines 370-372).
Comment: Demographic information about patients who used health care services is still missing. If this was not collected, why is it not mentioned in the limitation? Epidemiologists would benefit greatly from demographic information on health-care seeking behavior.

RESPONSE:
Thank you for noting this. The Nepali DHIS2 only contains information aggregated at the health facility level (e.g., total volume of services provided any given month). It does not contain any patient-level information.
We have added a sentence on this as a limitation at lines 372-374.
In the first revision, we aimed to partially address this comment by adding the average population size in the treated vs. control palikas as well as the average number of new Covid cases (Table 1). This shows that our treated groups tend to include smaller municipalities and faced fewer COVID cases.