Advertisement

Systematic review of empirical studies comparing the effectiveness of non-pharmaceutical interventions against COVID-19

  • Alba Mendez-Brito
    Affiliations
    Evidence-Based Public Health Unit, Centre for International Health Protection, Robert Koch Institute, Nordufer 20, Berlin 13353, Germany

    Institute of Tropical Medicine and International Health, Charité University Berlin, Campus Virchow-Klinikum, Augustenburger Platz 1, Berlin 13353, Germany
    Search for articles by this author
  • Charbel El Bcheraoui
    Affiliations
    Evidence-Based Public Health Unit, Centre for International Health Protection, Robert Koch Institute, Nordufer 20, Berlin 13353, Germany
    Search for articles by this author
  • Francisco Pozo-Martin
    Correspondence
    Corresponding author.
    Affiliations
    Evidence-Based Public Health Unit, Centre for International Health Protection, Robert Koch Institute, Nordufer 20, Berlin 13353, Germany
    Search for articles by this author
Open AccessPublished:June 20, 2021DOI:https://doi.org/10.1016/j.jinf.2021.06.018

      Highlights

      • Evidence from empirical studies suggests that a range of non-pharmaceutical interventions are effective against COVID-19.
      • School closing was the most effective NPI, followed by workplace closing, business and venue closing and public event bans.
      • Public information campaigns and mask wearing requirements are less disruptive for the population and also proved to be effective.
      • There was no evidence on the effectiveness of public transport closure, testing and contact tracing strategies and quarantining or isolation of individuals.
      • Early implementation was associated with a higher effectiveness.

      Abstract

      Objectives

      To evaluate which non-pharmaceutical interventions (NPIs) have been more and less effective in controlling the COVID-19 pandemic.

      Methods

      We performed a systematic review of published and unpublished empirical studies, either observational or interventional, analysing the comparative effectiveness of NPIs against the COVID-19 pandemic. We searched Embase/Medline and medRxiv to identify the relevant literature.

      Results

      We identified 34 studies. During the first wave of the COVID-19 pandemic, school closing was the most effective NPI, followed by workplace closing, business and venue closing and public event bans. Public information campaigns and mask wearing requirements were also effective in controlling the pandemic while being less disruptive for the population than other NPIs. There was no evidence on the effectiveness of public transport closure, testing and contact tracing strategies and quarantining or isolation of individuals. Early implementation was associated with a higher effectiveness in reducing COVID-19 cases and deaths, while general stringency of the NPIs was not.

      Conclusions

      In this systematic review, we found that school closing, followed by workplace closing, business and venue closing and public event bans were the most effective NPIs in controlling the spread of COVID-19. An early response and a combination of specific social distancing measures are effective at reducing COVID-19 cases and deaths. Continuous monitoring of NPIs effectiveness is needed in order to adapt decision making.

      Keywords

      Introduction

      In December 2019, a pneumonia-like disease caused an outbreak in the city of Wuhan, China. This disease, later named COVID-19, spread globally and was declared a pandemic in March 2020 by the World Health Organisation. By April 2021 it has already affected around 145 million people and resulted in more than three million deaths globally.

      Worldometers. COVID-19 Coronavirus Pandemic [Available from: https://www.worldometers.info/coronavirus/?utm_campaign=homeAdvegas1?]

      Until effective treatments are available and vaccines are extensively accessible and administered, governments rely on non-pharmaceutical interventions (NPIs) to control the epidemic. The positive effects of implementing NPIs in controlling the COVID-19 pandemic have been widely studied both at the national
      • Lai S.
      • Ruktanonchai N.W.
      • Zhou L.
      • Prosper O.
      • Luo W.
      • Floyd J.R.
      • et al.
      Effect of non-pharmaceutical interventions to contain COVID-19 in China.
      • Thu T.P.B.
      • Ngoc P.N.H.
      • Hai N.M.
      • Tuan L.A.
      Effect of the social distancing measures on the spread of COVID-19 in 10 highly infected countries.
      • Sebastiani G.
      • Massa M.
      • Riboli E.
      Covid-19 epidemic in Italy: evolution, projections and impact of government measures.
      and the international level.
      • Askitas N.
      • Tatsiramos K.
      • Verheyden B.
      Estimating worldwide effects of non-pharmaceutical interventions on COVID-19 incidence and population mobility patterns using a multiple-event study.
      However, due to the high social and economic costs of many of the interventions implemented, it is essential to understand their individual effectiveness to optimize implementation and lifting strategies.
      • Nicola M.
      • Alsafi Z.
      • Sohrabi C.
      • Kerwan A.
      • Al-Jabir A.
      • Iosifidis C.
      • et al.
      The socio-economic implications of the coronavirus pandemic (COVID-19): a review.
      ,
      • Trougakos J.P.
      • Chawla N.
      • McCarthy J.M.
      Working in a pandemic: exploring the impact of COVID-19 health anxiety on work, family, and health outcomes.
      A wide range of responses has been implemented worldwide, relying on previous knowledge of NPIs in controlling other epidemics.
      • Peak C.M.
      • Childs L.M.
      • Grad Y.H.
      • Buckee C.O.
      Comparing nonpharmaceutical interventions for containing emerging epidemics.
      Several intervention types have been implemented, including containment measures such as domestic or international travel bans, individual protection measures like mask wearing requirements, social distancing measures such as school closing and gathering bans and health system measures like testing and contact tracing strategies. Evidence on the effectiveness of NPIs is largely based on mathematical modeling, with a limited number of empirical studies, either observational or interventional, exploring this topic. The assessment of empirical studies provides real world effectiveness estimations that do not rely strongly on assumptions as do simulations in modeling. In this review, we summarize the current evidence from empirical studies on the comparative effectiveness of NPIs that have been implemented worldwide to control the current COVID-19 pandemic.

      Methodology

      In this review we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)
      • Moher D.
      • Liberati A.
      • Tetzlaff J.
      • Altman D.G.
      Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
      statement. We searched Embase (including Medline resources) and MedRxiv for published literature and preprints, respectively. We restricted the search to articles available in English from January 1, 2020. The search was conducted on March 4 2021, for Embase and on March 9 for MedRxiv. The search strategies used for both databases are available in Annex 1.
      Studies were included in the review if they:
      • Assessed NPI effectiveness only in the context of COVID-19.
      • Were either observational or interventional (i.e. quasi-experimental or experimental) studies of empirical data.
      • Included at least one of the following NPIs, as described and categorised in the Oxford COVID-19 Government Response Tracker (OxCGRT):
        • Hale T.
        • Angrist N.
        • Goldszmidt R.
        • Kira B.
        • Petherick A.
        • Phillips T.
        • et al.
        A global panel database of pandemic policies (Oxford COVID-19 government response tracker).
        school closing, workplace closing, public event cancelation, social gathering restrictions, public transport closure, stay-at-home requirements, internal movement restrictions, international travel restrictions, public information campaigns, testing policies, contact tracing policies and facial covering policies.
      • Compared the effectiveness of at least two NPIs.
      • Analysed NPI effectiveness in the general population of any geographical area.
      • Measured any health outcome.
      Studies were excluded from the review if they:
      • Were based on forecasts or simulations.
      • Did not assess the direct link between NPIs and the health outcome (for example, if the link was based on mobility).
      • Analysed the impact of adherence or compliance to NPIs.
      • Did not pre-specify the NPIs explored before the analysis (for example, breaking point analysis of epidemic curves were excluded).
      To perform the quality assessment of the studies, we used a risk of bias tool based on a bibliometric review of ecological studies
      • Dufault B.
      • Klar N.
      The quality of modern cross-sectional ecologic studies: a bibliometric review.
      and previously used in two published systematic reviews.
      • Cortes-Ramirez J.
      • Naish S.
      • Sly P.D.
      • Jagals P.
      Mortality and morbidity in populations in the vicinity of coal mining: a systematic review.
      ,
      • Betran A.P.
      • Torloni M.R.
      • Zhang J.
      • Ye J.
      • Mikolajczyk R.
      • Deneux-Tharaux C.
      • et al.
      What is the optimal rate of caesarean section at population level? A systematic review of ecologic studies.
      The tool assesses the study design, statistical methodology and quality of reporting. We added one question to the tool to expand the methodological assessment of the studies included. The risk of bias tool checklist and the final risk of bias rating of the studies are available in Annex 2.
      One reviewer (AMB) screened the records, selected the studies for the review, extracted the data and assessed the risk of bias. A second reviewer (FPM) screened ten percent of the total records, all the records that were selected by abstract, and verified risk of bias judgments.

      Results

      34 studies were included in the review, from which 28 have been published, one of them in a journal without peer review, and six were preprints. The PRISMA diagram flow is presented in Fig. 1. An overview of the characteristics of all studies included in the review is provided in Annex 3.
      Fig 1
      Fig. 1PRISMA flow diagram for the selection of studies.

      Methodological characteristics of the studies

      Table 1 presents a summary of the setting, outcomes and NPIs assessed in the 34 studies.
      Table 1Setting, outcomes assessed, and NPIs included in the studies.
      Authors and outcome(s) assessedSettingNPIs studied
      Banholzer et al. (a)
      • Banholzer N.
      • van Weenen E.
      • Kratzwald B.
      Impact of non-pharmaceutical interventions on documented cases of COVID-19.


      Confirmed cases
      20 countries: 15 EU countries, USA, Canada, Australia, Norway and Switzerland(1) School closing / (2) Border closures / (3) Public event bans / (4) Gathering bans / (5) Venue closing / (6) Lockdowns prohibiting public movements without valid reason / (7) Work bans on non-essential business activities
      Banholzer et al. (b)
      • Banholzer N.
      • van Weenen E.
      • Lison A.
      Estimating the effects of non-pharmaceutical interventions on the number of new infections with COVID-19 during the first epidemic wave.


      Confirmed cases
      20 countries: 15 EU countries, USA, Canada, Australia, Norway and Switzerland(1) School closing / (2) Border closures / (3) Public event bans / (4) Gathering bans / (5) Venue closing / (6) Lockdowns prohibiting public movements without valid reason / (7) Work bans on non-essential business activities
      Bo et al.
      • Bo Y.
      • Guo C.
      • Lin C.
      • Zeng Y.
      • Li H.B.
      • Zhang Y.
      • et al.
      Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020.


      Reproduction number
      Worldwide, 190 countries(1) Mandatory face mask in public / (2) Isolation or quarantine

      (3) Social distancing / (4) Traffic restrictions
      Brauner et al.
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.


      Reproduction number
      Worldwide, 41 countries: 34 European and 7 non-European(1–3) Gatherings limited to 1000/100/10 people or less / (4–5) Some/ All but essential shops closed / (6–7) Schools or universities closed / (8) Stay-at-home orders with exemptions
      Chaudhry et al.
      • Chaudhry R.
      • Dranitsaris G.
      • Mubashir T.
      • Bartoszko J.
      • Riazi S.
      A country level analysis measuring the impact of government actions, country preparedness and socioeconomic factors on COVID-19 mortality and related health outcomes.


      Confirmed cases, recovered cases, critical cases and deaths per million
      Worldwide, 50 countries(1) Travel restrictions: None/ Partial/ Complete border closure / (2) Containment measures: None/ Partial / Complete lockdown / (3) Curfew implemented (stay-at-home orders limited to specific hours)
      Chernozhukov et al.
      • Chernozhukov V.
      • Kasahara H.
      • Schrimpf P.
      Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S.


      Cases and deaths growth rate
      USA, all states(1) Stay at-home / (2) Closed nonessential businesses / (3) Closed K-12 schools / (4) Face mask mandates for employees in public facing businesses
      Courtemanche et al.
      • Courtemanche C.
      • Garuccio J.
      • Le A.
      • Pinkston J.
      • Yelowitz A.
      Strong Social distancing measures in the United States reduced the COVID-19 growth rate.


      Cases growth rate
      USA, 3138 counties(1) Shelter-in-place orders / (2) Public school closing / (3) Bans on large social gatherings / (4) Closures of entertainment-related businesses
      Deb et al.
      • Deb P.
      • Furceri D.
      • Ostry J.D.
      The effect of containment measures on the COVID-19 pandemic.


      Confirmed cases and deaths
      Worldwide, 129 countries(1) School closing / (2) Workplace closing / (3) Public event cancellations / (4) Gathering restrictions / (5) Public transportation closures / (6) Stay-at-home orders / (7) Restrictions on internal movement / (8) International travel bans
      Dreher et al.
      • Dreher N.
      • Spiera Z.
      • McAuley F.M.
      • Kuohn L.
      • Durbin J.R.
      • Marayati N.F.
      • et al.
      Policy interventions, social distancing, and SARS-CoV-2 transmission in the United States: a retrospective state-level analysis.


      Reproduction number and case fatality rate
      USA, all states(1) Stay-at-home orders / (2) Educational facilities closure / (3) Non-essential business closure / (4) Limitations on mass gatherings
      Duhon et al.
      • Duhon J.
      • Bragazzi N.
      • Kong J.D.
      The impact of non-pharmaceutical interventions, demographic, social, and climatic factors on the initial growth rate of COVID-19: a cross-country study.


      Case growth rate
      Worldwide, unclear number of territories(1) School closing / (2) Workplace closing / (3) Cancellation of public events / (4) Restrictions on gatherings / (5) Public transit closures / (6) Stay-at-home requirements / (7) Restrictions on internal movement / (8) International travel controls
      Ebrahim et al.
      • Ebrahim S.
      • Ashworth H.
      • Noah C.
      • Kadambi A.
      • Toumi A.
      • Chhatwal J.
      Reduction of COVID-19 incidence and nonpharmacologic interventions: analysis using a US county-level policy data set.


      Case growth rate and reproduction number
      USA, 1320 counties(1) Closure of nonessential workplaces / (2) Shelter-in-place/stay-at-home orders / (3) Enforcement of shelter-in-place or stay-at-home / (4) Size restrictions on public gatherings / (5) School closing / (6) Public transport closures / (7) Publicly available testing
      Esra et al.
      • Esra R.
      • Jamieson L.
      • Fox M.P.
      • Letswalo D.
      Evaluating the impact of non-pharmaceutical interventions for SARS-CoV-2 on a global scale.


      Reproduction number
      Worldwide, 26 countries and 34 US states(1) Quarantine and isolation policies / (2) Limits on gatherings / (3) School closing (primary, secondary and tertiary educational institutions) / (4) Mask policies / (5) Household confinements (stay-at-home-orders, shelter-in-place orders and lockdowns)
      Flaxman et al.
      • Flaxman S.
      • Mishra S.
      • Gandy A.
      • Unwin H.J.T.
      • Mellan T.A.
      • Coupland H.
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.


      Reproduction number
      Europe, 11 countries(1) Lockdown / (2) Cancel public events / (3) School closing / (4) Self-isolation / (5) Social distancing encouraged
      Fountoulakis  et al.
      • Fountoulakis K.N.
      • Fountoulakis N.K.
      • Koupidis S.A.
      • Prezerakos P.E.
      Factors determining different death rates because of the COVID-19 outbreak among countries.


      Death rate
      Europe, 40 countries(1) School closing / (2) Workplace closing / (3) Public events ban / (4) Gathering ban / (5) Public transport closure / (6) Lockdown implementation / (7) Domestic travel ban / (8) International travel ban
      Haug et al.
      • Haug N.
      • Geyrhofer L.
      • Londei A.
      • Dervic E.
      • Desvars-Larrive A.
      • Loreto V.
      • et al.
      Ranking the effectiveness of worldwide COVID-19 government interventions.


      Reproduction number
      Worldwide, 79 territories, 56 countriesDifferent categories of NPIs in their hierarchical levels (42,151 measures)
      Hsiang et al.
      • Hsiang S.
      • Allen D.
      • Annan-Phan S.
      • Bell K.
      • Bolliger I.
      • Chong T.
      • et al.
      The effect of large-scale anti-contagion policies on the COVID-19 pandemic.


      Case growth rate
      China, South Korea, Italy, Iran, France, US(1) Restricting travel / (2) Social distancing / (3) Quarantine and lockdown / (4) Additional policies
      Hunter et al.
      • Hunter P.
      • Colón-González F.
      • Brainard J.
      Impact of non-pharmaceutical interventions against COVID-19 in Europe: a quasi-experimental study.


      Incident risk ratio of NPIs on the number of cases and deaths
      Europe, 30 countries(1) Mass gathering restrictions / (2) Initial business closure / (3) Educational facilities closed / (4) Non-essential services closed / (5) Stay-at-home order / (6) Travel severely limited - none European country
      Islam et al.
      • Islam N.
      • Sharp S.J.
      • Chowell G.
      • Shabnam S.
      • Kawachi I.
      • Lacey B.
      • et al.
      Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries.


      Incidence rate ratio
      Worldwide, 149 countries(1) Closures of schools / (2) Workplace / (3) Public transport /

      (4) Restrictions on mass gatherings and public events / (5) Restrictions on movement (stay-at-home regulations and restrictions on movements within a country)
      Jalali et al.
      • Jalali A.M.
      • Khoury S.G.
      • See J.
      • Gulsvig A.M.
      Delayed interventions, low compliance, and health disparities amplified the early spread of COVID-19.


      Case rates, mortality rates

      and case-fatality rates
      USA, 30 most populous counties and 10 most populous counties of CA, FL, NY and TX3 broad categories:

      (1) Restrictions on mass gatherings / (2) Stay-at-home orders / (3) Face mask requirements
      Jüni et al.
      • Jüni P.
      • Rothenbühler M.
      • Bobos P.
      • Thorpe K.E.
      • Da Costa B.R.
      • Fisman D.N.
      • et al.
      Impact of climate and public health interventions on the COVID-19 pandemic: a prospective cohort study.


      Epidemic growth
      Worldwide, 144 territories, (not China, South Korea, Iran, Italy)(1) School closing / (2) Restrictions of mass gatherings / (3) Measures of social distancing
      Koh et al.
      • Koh W.C.
      • Naing L.
      • Wong J.
      Estimating the impact of physical distancing measures in containing COVID-19: an empirical analysis.


      Reproduction number
      Worldwide, 142 countries(1) International travel controls (including screening, quarantine and bans on international movement) / (2) Restrictions on mass gatherings (including public event bans and size restrictions on gatherings) / (3) Lockdown-type measures (including workplace closure, internal movement restrictions)
      Leffler et al.
      • Leffler C.T.
      • Ing E.
      • Lykins J.D.
      • Hogan M.C.
      • McKeown C.A.
      • Grzybowski A.
      Association of country-wide coronavirus mortality with demographics, testing, lockdowns, and public wearing of masks.


      Mortality
      Worldwide, 200 countries(1) School closing / (2) Workplace closing / (3) Cancel public events / (4) estrictions on gatherings / (5) Close public transport / (6) Stay-at-home requirements / (7) Internal movement restrictions / (8) International travel restrictions / (9) Income support / (10) Public information campaigns / (11) Testing policy / (12) Contact tracing policy / (13) Public wearing of masks
      Li et al. (a)
      • Li Y.
      • Campbell H.
      • Kulkarni D.
      • Harpur A.
      • Nundy M.
      • Wang X.
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.


      Reproduction number
      Worldwide, 131 countries(1) Closure of schools / (2) Closure of workplaces / (3) Public events bans / (4) Restrictions on the size of gatherings / (5) Closure of public transport / (6) Stay-at-home orders / (7) Restrictions on internal movement / (8) Restrictions on international travel
      Li et al. (b)
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.


      Case growth rate and death case growth rate
      USA, all states(1) School closing / (2) Workplace closures / (3) Public event cancellations / (4) Public information campaigns / (5) Public transport closures / (6) Stay-at-home orders / (7) International/national travel controls
      Liu et al.
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.


      Reproduction number
      Worldwide, 130 countries(1) Internal containment and closure (School and workplace closure, public event cancelation, limits on gathering sizes, public transport closure, stay-at-home requirement, internal movement restriction) / (2) International travel restrictions /

      (3) Economic policies / (4) Health systems policies (Public information campaign, testing policy, contact tracing)
      Olney et al.
      • Olney A.M.
      • Smith J.
      • Sen S.
      • Thomas F.
      • Unwin H.J.T.
      Estimating the effect of social distancing interventions on COVID-19 in the United States.


      Reproduction number
      USA, all states(1) Social distancing encouraged / (2) Schools or universities closing / (3) Public events (ban for more than 100 people)/ (4) Lockdown / (5) Self-isolating ill / (6) Sports (public event ban of more than 1000 people)
      Papadopoulos et al.
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.


      Total cases and total deaths
      Worlwide, 137 countries(1) School closing / (2) Workplace closing / (3) Cancelling of public events / (4) Restriction on gatherings / (5) Closure of public transport / (6) Stay-at-home restrictions / (7) Domestic travel restrictions / (8) International travel restrictions /

      (9) Public information / (10) Testing framework / (11) Contact tracing
      Piovani et al.
      • Piovani D.
      • Christodoulou M.N.
      • Hadjidemetriou A.
      • Pantavou K.
      • Zaza P.
      • Bagos P.G.
      • et al.
      Effect of early application of social distancing interventions on COVID-19 mortality over the first pandemic wave: an analysis of longitudinal data from 37 countries.


      Mortality
      OECD countries(1) Mass gathering ban (more than 1000 people) / (2) School closures
      Pozo-Martin et al.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.


      Case growth rate
      OECD countries(1) School closing requirements / (2) Workplace closing requirements / (3) Public events cancelling requirements / (4) Restrictions on gatherings / (5) Public transport restrictions / (6) Stay-at-home requirements / (7) Restrictions on internal travel / (8) International travel controls / (9) Public health information campaigns / (10) Mask wearing requirements / (11) Testing policy / (12) Contact tracing policy
      Siedner et al.
      • Siedner M.J.
      • Harling G.
      • Reynolds Z.
      • Gilbert R.F.
      • Haneuse S.
      • Venkataramani A.S.
      • et al.
      Social distancing to slow the US COVID-19 epidemic: longitudinal pretest-posttest comparison group study.


      Case growth rate and deaths growth rate
      USA, all states(1) Social distancing measures (closures of schools, closures of workplaces, cancellations of public events, restrictions on internal movement, closures of state borders) / (2) Internal movement restrictions (shelter-in-place orders, lockdowns)
      Stokes et al.
      • Stokes J.
      • Turner A.J.
      • Anselmi L.
      • Morciano M.
      • Hone T.
      The relative effects of non-pharmaceutical interventions on early COVID-19 mortality: natural experiment in 130 countries.


      Mortality rate
      Worldwide, 130 countries(1) School closing / (2) Workplace closing / (3) Public event cancelling / (4) Gathering restrictions / (5) Public transport closure / (6) Stay-at-home requirements / (7) Restrictions on internal movement / (8) International travel controls / (9) Public information campaigns
      Wibbens et al.
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.


      Case growth rate
      40 territories: 17 countries and 23 US states(1) Closing of schools / (2) Closing of workplaces / (3) Public event cancelling / (4) Gathering bans / (5) Public transport closure / (6) Shelter-in-place orders and home confinement / (7) Restrictions on internal movement / (8) Restrictions on international travel / (9) Public information campaigns / (10) Testing access / (11) Contact tracing
      Wong et al.
      • Wong M.C.
      • Huang J.
      • Teoh J.
      • Wong S.H.
      Evaluation on different non-pharmaceutical interventions during COVID-19 pandemic: an analysis of 139 countries.


      Cumulative incidence
      Worldwide, 139 countries(1) School closure / (2) Workplace closure / (3) Public event cancelation / (4) Restrictions on gathering size / (5) Public transport closure / (6) Staying at home requirements / (7) Internal movement restrictions / (8) International travel restrictions / (9) Public information campaigns
      Zhang et al.
      • Zhang X.
      • Warner M.E.
      Covid-19 policy differences across us states: shutdowns, reopening, and mask mandates.


      Case growth rate
      USA, all states(1) Implementation of shutdowns / (2) Mask mandates

      Study type, timeframe and geographical scope

      All studies identified were ecological studies with data aggregated at population level. While most of the studies analysed country level data, some included more granular analysis at regional,
      • Hsiang S.
      • Allen D.
      • Annan-Phan S.
      • Bell K.
      • Bolliger I.
      • Chong T.
      • et al.
      The effect of large-scale anti-contagion policies on the COVID-19 pandemic.
      ,
      • Jüni P.
      • Rothenbühler M.
      • Bobos P.
      • Thorpe K.E.
      • Da Costa B.R.
      • Fisman D.N.
      • et al.
      Impact of climate and public health interventions on the COVID-19 pandemic: a prospective cohort study.
      or city level.
      • Bo Y.
      • Guo C.
      • Lin C.
      • Zeng Y.
      • Li H.B.
      • Zhang Y.
      • et al.
      Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020.
      Most of the studies were based on data from the first wave of the pandemic. Only Wibbens et al
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      assessed the impact of NPIs until November 2020 and Pozo-Martin et al.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      performed a first analysis of the initial phase of the pandemic and a second from October until December 2020. Zhang et al.
      • Zhang X.
      • Warner M.E.
      Covid-19 policy differences across us states: shutdowns, reopening, and mask mandates.
      analysed another relatively long study period until August, and several authors performed an analysis until July 2020.
      • Duhon J.
      • Bragazzi N.
      • Kong J.D.
      The impact of non-pharmaceutical interventions, demographic, social, and climatic factors on the initial growth rate of COVID-19: a cross-country study.
      • Ebrahim S.
      • Ashworth H.
      • Noah C.
      • Kadambi A.
      • Toumi A.
      • Chhatwal J.
      Reduction of COVID-19 incidence and nonpharmacologic interventions: analysis using a US county-level policy data set.
      • Li Y.
      • Campbell H.
      • Kulkarni D.
      • Harpur A.
      • Nundy M.
      • Wang X.
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      Some authors standardised the start and/or end of the study period in order to be able to compare the effectiveness of NPIs across units of analysis at similar stages of the epidemic.
      • Banholzer N.
      • van Weenen E.
      • Kratzwald B.
      Impact of non-pharmaceutical interventions on documented cases of COVID-19.
      • Banholzer N.
      • van Weenen E.
      • Lison A.
      Estimating the effects of non-pharmaceutical interventions on the number of new infections with COVID-19 during the first epidemic wave.
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
      • Esra R.
      • Jamieson L.
      • Fox M.P.
      • Letswalo D.
      Evaluating the impact of non-pharmaceutical interventions for SARS-CoV-2 on a global scale.
      • Fountoulakis K.N.
      • Fountoulakis N.K.
      • Koupidis S.A.
      • Prezerakos P.E.
      Factors determining different death rates because of the COVID-19 outbreak among countries.
      • Flaxman S.
      • Mishra S.
      • Gandy A.
      • Unwin H.J.T.
      • Mellan T.A.
      • Coupland H.
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      • Olney A.M.
      • Smith J.
      • Sen S.
      • Thomas F.
      • Unwin H.J.T.
      Estimating the effect of social distancing interventions on COVID-19 in the United States.
      With respect to the geographical scope, twelve studies analysed more than 65% of the world's territories.
      • Jüni P.
      • Rothenbühler M.
      • Bobos P.
      • Thorpe K.E.
      • Da Costa B.R.
      • Fisman D.N.
      • et al.
      Impact of climate and public health interventions on the COVID-19 pandemic: a prospective cohort study.
      ,
      • Bo Y.
      • Guo C.
      • Lin C.
      • Zeng Y.
      • Li H.B.
      • Zhang Y.
      • et al.
      Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020.
      ,
      • Duhon J.
      • Bragazzi N.
      • Kong J.D.
      The impact of non-pharmaceutical interventions, demographic, social, and climatic factors on the initial growth rate of COVID-19: a cross-country study.
      ,
      • Li Y.
      • Campbell H.
      • Kulkarni D.
      • Harpur A.
      • Nundy M.
      • Wang X.
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      ,
      • Deb P.
      • Furceri D.
      • Ostry J.D.
      The effect of containment measures on the COVID-19 pandemic.
      • Islam N.
      • Sharp S.J.
      • Chowell G.
      • Shabnam S.
      • Kawachi I.
      • Lacey B.
      • et al.
      Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries.
      • Koh W.C.
      • Naing L.
      • Wong J.
      Estimating the impact of physical distancing measures in containing COVID-19: an empirical analysis.
      • Leffler C.T.
      • Ing E.
      • Lykins J.D.
      • Hogan M.C.
      • McKeown C.A.
      • Grzybowski A.
      Association of country-wide coronavirus mortality with demographics, testing, lockdowns, and public wearing of masks.
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      • Wong M.C.
      • Huang J.
      • Teoh J.
      • Wong S.H.
      Evaluation on different non-pharmaceutical interventions during COVID-19 pandemic: an analysis of 139 countries.
      • Stokes J.
      • Turner A.J.
      • Anselmi L.
      • Morciano M.
      • Hone T.
      The relative effects of non-pharmaceutical interventions on early COVID-19 mortality: natural experiment in 130 countries.
      Some focused on specific areas, with nine studies analysing data from the United States, performing mostly state-level,
      • Zhang X.
      • Warner M.E.
      Covid-19 policy differences across us states: shutdowns, reopening, and mask mandates.
      ,
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
      ,
      • Olney A.M.
      • Smith J.
      • Sen S.
      • Thomas F.
      • Unwin H.J.T.
      Estimating the effect of social distancing interventions on COVID-19 in the United States.
      ,
      • Chernozhukov V.
      • Kasahara H.
      • Schrimpf P.
      Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S.
      • Dreher N.
      • Spiera Z.
      • McAuley F.M.
      • Kuohn L.
      • Durbin J.R.
      • Marayati N.F.
      • et al.
      Policy interventions, social distancing, and SARS-CoV-2 transmission in the United States: a retrospective state-level analysis.
      • Siedner M.J.
      • Harling G.
      • Reynolds Z.
      • Gilbert R.F.
      • Haneuse S.
      • Venkataramani A.S.
      • et al.
      Social distancing to slow the US COVID-19 epidemic: longitudinal pretest-posttest comparison group study.
      but also county-level
      • Ebrahim S.
      • Ashworth H.
      • Noah C.
      • Kadambi A.
      • Toumi A.
      • Chhatwal J.
      Reduction of COVID-19 incidence and nonpharmacologic interventions: analysis using a US county-level policy data set.
      ,
      • Courtemanche C.
      • Garuccio J.
      • Le A.
      • Pinkston J.
      • Yelowitz A.
      Strong Social distancing measures in the United States reduced the COVID-19 growth rate.
      ,
      • Jalali A.M.
      • Khoury S.G.
      • See J.
      • Gulsvig A.M.
      Delayed interventions, low compliance, and health disparities amplified the early spread of COVID-19.
      analyses, three studies estimating the effects of NPIs in Europe
      • Fountoulakis K.N.
      • Fountoulakis N.K.
      • Koupidis S.A.
      • Prezerakos P.E.
      Factors determining different death rates because of the COVID-19 outbreak among countries.
      ,
      • Flaxman S.
      • Mishra S.
      • Gandy A.
      • Unwin H.J.T.
      • Mellan T.A.
      • Coupland H.
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
      ,
      • Hunter P.
      • Colón-González F.
      • Brainard J.
      Impact of non-pharmaceutical interventions against COVID-19 in Europe: a quasi-experimental study.
      and two studies in all OECD countries.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      ,
      • Piovani D.
      • Christodoulou M.N.
      • Hadjidemetriou A.
      • Pantavou K.
      • Zaza P.
      • Bagos P.G.
      • et al.
      Effect of early application of social distancing interventions on COVID-19 mortality over the first pandemic wave: an analysis of longitudinal data from 37 countries.
      Five studies selected specific countries that belong to different world regions
      • Hsiang S.
      • Allen D.
      • Annan-Phan S.
      • Bell K.
      • Bolliger I.
      • Chong T.
      • et al.
      The effect of large-scale anti-contagion policies on the COVID-19 pandemic.
      ,
      • Banholzer N.
      • van Weenen E.
      • Kratzwald B.
      Impact of non-pharmaceutical interventions on documented cases of COVID-19.
      ,
      • Banholzer N.
      • van Weenen E.
      • Lison A.
      Estimating the effects of non-pharmaceutical interventions on the number of new infections with COVID-19 during the first epidemic wave.
      ,
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      ,
      • Chaudhry R.
      • Dranitsaris G.
      • Mubashir T.
      • Bartoszko J.
      • Riazi S.
      A country level analysis measuring the impact of government actions, country preparedness and socioeconomic factors on COVID-19 mortality and related health outcomes.
      and three included both countries and US states.
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      ,
      • Esra R.
      • Jamieson L.
      • Fox M.P.
      • Letswalo D.
      Evaluating the impact of non-pharmaceutical interventions for SARS-CoV-2 on a global scale.
      ,
      • Haug N.
      • Geyrhofer L.
      • Londei A.
      • Dervic E.
      • Desvars-Larrive A.
      • Loreto V.
      • et al.
      Ranking the effectiveness of worldwide COVID-19 government interventions.

      Outcomes of interest

      The authors have reported on different epidemiological parameters to assess the effectiveness of the NPIs studied. More than half of the studies reported on the reproduction number and the case growth rate, daily or weekly (detailed in Annex 3). For all of these studies, excluding Ebrahim et al.
      • Ebrahim S.
      • Ashworth H.
      • Noah C.
      • Kadambi A.
      • Toumi A.
      • Chhatwal J.
      Reduction of COVID-19 incidence and nonpharmacologic interventions: analysis using a US county-level policy data set.
      , Li et al. (a)
      • Li Y.
      • Campbell H.
      • Kulkarni D.
      • Harpur A.
      • Nundy M.
      • Wang X.
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      and Liu et al.
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
      the outcome was estimated by the authors through calculations assuming different epidemiological characteristics of SARS-CoV-2, like time until symptom onset and the distribution of the serial interval.
      • Thompson R.N.
      • Stockwin J.E.
      • van Gaalen R.D.
      • Polonsky J.A.
      • Kamvar Z.N.
      • Demarsh P.A.
      • et al.
      Improved inference of time-varying reproduction numbers during infectious disease outbreaks.
      The number of confirmed cases, mortality or death rate, confirmed deaths and deaths growth rate come next as the most reported outcomes.

      NPIs assessed

      We analysed 16 NPIs that were consistently assessed in the studies included in the review. More than three quarters of the studies analysed the effectiveness of two NPIs: lockdowns (also called stay-at-home mandates or shelter-in-place orders) were analysed by 29 studies and school closing was analysed by 27. Half of the studies analysed in this review assessed the effectiveness of international travel or border restrictions and social gathering restrictions. Several NPIs were analysed in less than a quarter of the studies: business or venue closing, testing strategies, mask wearing requirements, social distancing, contact tracing strategies and isolation or quarantine (see Table 1).

      Statistical methodology

      Several authors used sophisticated and flexible methods, like Bayesian longitudinal models
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      ,
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      ,
      • Banholzer N.
      • van Weenen E.
      • Kratzwald B.
      Impact of non-pharmaceutical interventions on documented cases of COVID-19.
      ,
      • Banholzer N.
      • van Weenen E.
      • Lison A.
      Estimating the effects of non-pharmaceutical interventions on the number of new infections with COVID-19 during the first epidemic wave.
      ,
      • Flaxman S.
      • Mishra S.
      • Gandy A.
      • Unwin H.J.T.
      • Mellan T.A.
      • Coupland H.
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      • Olney A.M.
      • Smith J.
      • Sen S.
      • Thomas F.
      • Unwin H.J.T.
      Estimating the effect of social distancing interventions on COVID-19 in the United States.
      or event studies,
      • Hsiang S.
      • Allen D.
      • Annan-Phan S.
      • Bell K.
      • Bolliger I.
      • Chong T.
      • et al.
      The effect of large-scale anti-contagion policies on the COVID-19 pandemic.
      ,
      • Zhang X.
      • Warner M.E.
      Covid-19 policy differences across us states: shutdowns, reopening, and mask mandates.
      ,
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
      ,
      • Courtemanche C.
      • Garuccio J.
      • Le A.
      • Pinkston J.
      • Yelowitz A.
      Strong Social distancing measures in the United States reduced the COVID-19 growth rate.
      while others used linear regression
      • Jüni P.
      • Rothenbühler M.
      • Bobos P.
      • Thorpe K.E.
      • Da Costa B.R.
      • Fisman D.N.
      • et al.
      Impact of climate and public health interventions on the COVID-19 pandemic: a prospective cohort study.
      ,
      • Duhon J.
      • Bragazzi N.
      • Kong J.D.
      The impact of non-pharmaceutical interventions, demographic, social, and climatic factors on the initial growth rate of COVID-19: a cross-country study.
      ,
      • Li Y.
      • Campbell H.
      • Kulkarni D.
      • Harpur A.
      • Nundy M.
      • Wang X.
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      ,
      • Fountoulakis K.N.
      • Fountoulakis N.K.
      • Koupidis S.A.
      • Prezerakos P.E.
      Factors determining different death rates because of the COVID-19 outbreak among countries.
      ,
      • Koh W.C.
      • Naing L.
      • Wong J.
      Estimating the impact of physical distancing measures in containing COVID-19: an empirical analysis.
      ,
      • Leffler C.T.
      • Ing E.
      • Lykins J.D.
      • Hogan M.C.
      • McKeown C.A.
      • Grzybowski A.
      Association of country-wide coronavirus mortality with demographics, testing, lockdowns, and public wearing of masks.
      ,
      • Wong M.C.
      • Huang J.
      • Teoh J.
      • Wong S.H.
      Evaluation on different non-pharmaceutical interventions during COVID-19 pandemic: an analysis of 139 countries.
      ,
      • Stokes J.
      • Turner A.J.
      • Anselmi L.
      • Morciano M.
      • Hone T.
      The relative effects of non-pharmaceutical interventions on early COVID-19 mortality: natural experiment in 130 countries.
      and simple correlation coefficients.
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      ,
      • Jalali A.M.
      • Khoury S.G.
      • See J.
      • Gulsvig A.M.
      Delayed interventions, low compliance, and health disparities amplified the early spread of COVID-19.
      The quality of the statistical methodology is evaluated in the risk of bias tool in Annex 2.

      Risk of bias analysis

      The maximum score from the tool was 18, the highest ranked study had a score of 17 and the lowest had a score of 11. The studies were grouped in three categories based on their rating: lower (with a rating of 11 or 12), intermediate (with a rating of 13–15) and higher (with a rating of 16 or 17) quality. Overall, thirty studies were considered to have to have intermediate and higher quality. The detailed quality assessment tool is provided in Annex 2.

      Results of the studies

      Results on the reproduction number, epidemic growth and incidence-related outcomes

      A heatmap of the findings of all studies (excluding Hsiang et al.
      • Hsiang S.
      • Allen D.
      • Annan-Phan S.
      • Bell K.
      • Bolliger I.
      • Chong T.
      • et al.
      The effect of large-scale anti-contagion policies on the COVID-19 pandemic.
      , who compared different NPIs from different countries) studying the reproduction number, epidemic growth and incidence-related outcomes is shown in Fig. 2. Pozo-Martin et al.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      performed independent analyses of two different timeframes, each analysis is represented separately in the heatmap.
      Fig 2
      Fig. 2Heatmap of the assessment of NPIs effectiveness in the studies analysing epidemic growth and incidence-related outcomes. The color grading is given according to the effectiveness ranking of each article. Darker green means higher effectiveness among the NPIs studied in the same article. Gray means no significant association with the outcome. White means the NPI was not studied. If no ranking was established, all the NPIs studied in the same article have the same shade of green. The rating provided is the result of the risk of bias analysis.
      Rt = time varying Reproduction number; g = growth rate; IRR1 = incident risk ratio of NPIs on the number of cases; IRR2 = incidence rate ratio; Growth* = epidemic growth expressed as ratios of rate ratios; CI = cumulative incidence. Pozo-Martin et al. 1 and 2: first and second waves.

      Most effective interventions

      Overall, school closing was found to be the most effective measure: 14 out of 24 studies (58%) that analysed this NPI found an association with reduced number of cases and its implementation. It was estimated to be the most effective policy in four studies
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      ,
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
      ,
      • Hunter P.
      • Colón-González F.
      • Brainard J.
      Impact of non-pharmaceutical interventions against COVID-19 in Europe: a quasi-experimental study.
      ,
      • Haug N.
      • Geyrhofer L.
      • Londei A.
      • Dervic E.
      • Desvars-Larrive A.
      • Loreto V.
      • et al.
      Ranking the effectiveness of worldwide COVID-19 government interventions.
      and the second most effective in four studies
      • Li Y.
      • Campbell H.
      • Kulkarni D.
      • Harpur A.
      • Nundy M.
      • Wang X.
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      ,
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      ,
      • Olney A.M.
      • Smith J.
      • Sen S.
      • Thomas F.
      • Unwin H.J.T.
      Estimating the effect of social distancing interventions on COVID-19 in the United States.
      ,
      • Islam N.
      • Sharp S.J.
      • Chowell G.
      • Shabnam S.
      • Kawachi I.
      • Lacey B.
      • et al.
      Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries.
      . Brauner et al.
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      , estimated a mean reproduction number reduction of 39% after the closure of schools and universities. Haug et al.
      • Haug N.
      • Geyrhofer L.
      • Londei A.
      • Dervic E.
      • Desvars-Larrive A.
      • Loreto V.
      • et al.
      Ranking the effectiveness of worldwide COVID-19 government interventions.
      found robust evidence of a mean reduction of 73% in the reproduction number associated with school closures.
      Other NPIs that were consistently found among the most effective in reducing cases were: workplace closing, business or venue closing and public event bans. Workplace closing was associated with an improvement of the outcome in 12 out of 14 studies that analysed this NPI (86%). Among the studies that ranked the NPIs, four concluded that it was the most effective measure
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      ,
      • Koh W.C.
      • Naing L.
      • Wong J.
      Estimating the impact of physical distancing measures in containing COVID-19: an empirical analysis.
      ,
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
      ,
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      and three found it was the second most effective.
      • Li Y.
      • Campbell H.
      • Kulkarni D.
      • Harpur A.
      • Nundy M.
      • Wang X.
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      ,
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
      ,
      • Wong M.C.
      • Huang J.
      • Teoh J.
      • Wong S.H.
      Evaluation on different non-pharmaceutical interventions during COVID-19 pandemic: an analysis of 139 countries.
      Regarding business and venue closing, four out of seven studies assessing its effectiveness (57%) concluded that this measure had a significant impact on the outcome, from which two studies found it the most effective intervention
      • Banholzer N.
      • van Weenen E.
      • Kratzwald B.
      Impact of non-pharmaceutical interventions on documented cases of COVID-19.
      ,
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      and another two the second most effective.
      • Dreher N.
      • Spiera Z.
      • McAuley F.M.
      • Kuohn L.
      • Durbin J.R.
      • Marayati N.F.
      • et al.
      Policy interventions, social distancing, and SARS-CoV-2 transmission in the United States: a retrospective state-level analysis.
      ,
      • Courtemanche C.
      • Garuccio J.
      • Le A.
      • Pinkston J.
      • Yelowitz A.
      Strong Social distancing measures in the United States reduced the COVID-19 growth rate.
      For public event bans, six out of 12 studies (50%) found that it was a predictor for the outcome. Li et al. (a)
      • Li Y.
      • Campbell H.
      • Kulkarni D.
      • Harpur A.
      • Nundy M.
      • Wang X.
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      found it was the most effective measure, reaching a peak effect of 25% reduction in the reproduction number 28 days after its implementation. Islam et al.
      • Islam N.
      • Sharp S.J.
      • Chowell G.
      • Shabnam S.
      • Kawachi I.
      • Lacey B.
      • et al.
      Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries.
      found that greater reduction of the incidence was always observed when public event bans were included in the combination of measures implemented. Two studies found it the second most effective control measure.
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
      ,
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.

      Intermediate effectiveness interventions

      There are NPIs consistently found effective among the studies, which were not generally ranked as the most effective measures. These NPIs are lockdowns, movement limitations through national or international travel restrictions, social gathering bans ranging from 10 people to mass gathering bans, social distancing, public information campaigns and mask wearing requirements.
      Twenty-seven studies analysed the relation between lockdown and the reproduction number, case growth rates and cases, and 18 out them (67%) found an association with their reduction. Five studies found lockdowns to be the most effective control measure.
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      ,
      • Esra R.
      • Jamieson L.
      • Fox M.P.
      • Letswalo D.
      Evaluating the impact of non-pharmaceutical interventions for SARS-CoV-2 on a global scale.
      ,
      • Olney A.M.
      • Smith J.
      • Sen S.
      • Thomas F.
      • Unwin H.J.T.
      Estimating the effect of social distancing interventions on COVID-19 in the United States.
      ,
      • Dreher N.
      • Spiera Z.
      • McAuley F.M.
      • Kuohn L.
      • Durbin J.R.
      • Marayati N.F.
      • et al.
      Policy interventions, social distancing, and SARS-CoV-2 transmission in the United States: a retrospective state-level analysis.
      ,
      • Courtemanche C.
      • Garuccio J.
      • Le A.
      • Pinkston J.
      • Yelowitz A.
      Strong Social distancing measures in the United States reduced the COVID-19 growth rate.
      Flaxman et al.
      • Flaxman S.
      • Mishra S.
      • Gandy A.
      • Unwin H.J.T.
      • Mellan T.A.
      • Coupland H.
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
      found it to be the only measure reducing the reproduction number below one. In contrast, five studies found it to be the least effective measure studied.
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
      ,
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      ,
      • Islam N.
      • Sharp S.J.
      • Chowell G.
      • Shabnam S.
      • Kawachi I.
      • Lacey B.
      • et al.
      Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries.
      ,
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      ,
      • Haug N.
      • Geyrhofer L.
      • Londei A.
      • Dervic E.
      • Desvars-Larrive A.
      • Loreto V.
      • et al.
      Ranking the effectiveness of worldwide COVID-19 government interventions.
      This disparity could be a result of the contrasting definitions of lockdown used by different authors. While most authors studied the additional impact of lockdown after the implementation of other NPIs,
      • Banholzer N.
      • van Weenen E.
      • Kratzwald B.
      Impact of non-pharmaceutical interventions on documented cases of COVID-19.
      ,
      • Banholzer N.
      • van Weenen E.
      • Lison A.
      Estimating the effects of non-pharmaceutical interventions on the number of new infections with COVID-19 during the first epidemic wave.
      ,
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      others estimated the effect of lockdown including several other measures.
      • Hsiang S.
      • Allen D.
      • Annan-Phan S.
      • Bell K.
      • Bolliger I.
      • Chong T.
      • et al.
      The effect of large-scale anti-contagion policies on the COVID-19 pandemic.
      ,
      • Flaxman S.
      • Mishra S.
      • Gandy A.
      • Unwin H.J.T.
      • Mellan T.A.
      • Coupland H.
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
      ,
      • Koh W.C.
      • Naing L.
      • Wong J.
      Estimating the impact of physical distancing measures in containing COVID-19: an empirical analysis.
      Brauner et al.
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      suggest, that in some countries the reproduction number may have decreased below one without enforcing lockdowns by issuing other NPIs. Li et al. (b)
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
      studied lockdown for more than two months in the United States and concluded that it proved to be effective, but that its effect in reducing the growth rate of the number of cases decreased with time.
      Both domestic and international travel restrictions have been associated with a reduction in the reproduction number, growth rate or incidence-related outcomes. Five out of 12 studies (42%) assessing domestic travel restrictions and nine out of 15 (60%) studying international travel restrictions found an association. International travel restrictions were shown to be more effective than domestic or national travel restrictions in the studies analysing both policies. Haug et al.
      • Haug N.
      • Geyrhofer L.
      • Londei A.
      • Dervic E.
      • Desvars-Larrive A.
      • Loreto V.
      • et al.
      Ranking the effectiveness of worldwide COVID-19 government interventions.
      estimated that international border restrictions reduced the reproduction number by 56%, while individual movement restrictions reduced it by 42%. In contrast, Wibbens et al.
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      concluded that even recommendations of avoiding movement across regions and cities were more effective than bans on all international regions and total border closures in reducing weekly growth rates.
      The definition of social gathering restrictions ranged from mass gathering bans to banning gathering of less than ten people. While mass gathering bans were associated with a reduction of incidence-related outcomes in 7 out of 14 studies (50%), social gathering bans were associated in 11 out of 15 (73%). Consistently in higher and intermediate quality studies, restrictions of smaller social gatherings have been found more effective than mass gathering restrictions, Haug et al.
      • Haug N.
      • Geyrhofer L.
      • Londei A.
      • Dervic E.
      • Desvars-Larrive A.
      • Loreto V.
      • et al.
      Ranking the effectiveness of worldwide COVID-19 government interventions.
      , and Banholzer et al. (a)
      • Banholzer N.
      • van Weenen E.
      • Kratzwald B.
      Impact of non-pharmaceutical interventions on documented cases of COVID-19.
      reached the same conclusion.
      While some authors consider social distancing to be a combination of certain other restrictive measures
      • Jüni P.
      • Rothenbühler M.
      • Bobos P.
      • Thorpe K.E.
      • Da Costa B.R.
      • Fisman D.N.
      • et al.
      Impact of climate and public health interventions on the COVID-19 pandemic: a prospective cohort study.
      ,
      • Bo Y.
      • Guo C.
      • Lin C.
      • Zeng Y.
      • Li H.B.
      • Zhang Y.
      • et al.
      Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020.
      ,
      • Siedner M.J.
      • Harling G.
      • Reynolds Z.
      • Gilbert R.F.
      • Haneuse S.
      • Venkataramani A.S.
      • et al.
      Social distancing to slow the US COVID-19 epidemic: longitudinal pretest-posttest comparison group study.
      sometimes it is considered as the official encouragement by the government to keep social distance.
      • Flaxman S.
      • Mishra S.
      • Gandy A.
      • Unwin H.J.T.
      • Mellan T.A.
      • Coupland H.
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
      Flaxman et al.
      • Flaxman S.
      • Mishra S.
      • Gandy A.
      • Unwin H.J.T.
      • Mellan T.A.
      • Coupland H.
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
      evaluated the effectiveness of officially encouraging social distance and did not observe an impact on the reproduction number. However, several studies that defined social distancing as a mix of several policies found an association between these measures and a reduction in the reproduction number,
      • Bo Y.
      • Guo C.
      • Lin C.
      • Zeng Y.
      • Li H.B.
      • Zhang Y.
      • et al.
      Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020.
      the growth rate
      • Siedner M.J.
      • Harling G.
      • Reynolds Z.
      • Gilbert R.F.
      • Haneuse S.
      • Venkataramani A.S.
      • et al.
      Social distancing to slow the US COVID-19 epidemic: longitudinal pretest-posttest comparison group study.
      and the epidemic growth.
      • Jüni P.
      • Rothenbühler M.
      • Bobos P.
      • Thorpe K.E.
      • Da Costa B.R.
      • Fisman D.N.
      • et al.
      Impact of climate and public health interventions on the COVID-19 pandemic: a prospective cohort study.
      Two health system measures found to be effective in reducing COVID-19 cases are public information campaigns and mask wearing requirements. Six out of eight studies (75%) analysing public information campaigns and six out of seven (86%) analysing mask wearing requirements found an association with the outcomes of interest. Several studies found public information campaigns highly effective.
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      ,
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
      ,
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      ,
      • Wong M.C.
      • Huang J.
      • Teoh J.
      • Wong S.H.
      Evaluation on different non-pharmaceutical interventions during COVID-19 pandemic: an analysis of 139 countries.
      Wibbens et al.
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      found it the most impactful measure when there was a coordinated public information campaign. Li et al. (b)
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
      found that the growth rate reduction induced by public health information campaigns increased during the study period, reaching its peak two months after implementation. However, some high rated studies found public information campaigns to be among the least effective policies.
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
      ,
      • Haug N.
      • Geyrhofer L.
      • Londei A.
      • Dervic E.
      • Desvars-Larrive A.
      • Loreto V.
      • et al.
      Ranking the effectiveness of worldwide COVID-19 government interventions.
      With respect to mask wearing requirements, three intermediate and high-quality studies
      • Bo Y.
      • Guo C.
      • Lin C.
      • Zeng Y.
      • Li H.B.
      • Zhang Y.
      • et al.
      Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020.
      ,
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      ,
      • Esra R.
      • Jamieson L.
      • Fox M.P.
      • Letswalo D.
      Evaluating the impact of non-pharmaceutical interventions for SARS-CoV-2 on a global scale.
      found it to be among the most effective measures. Chernozhukov et al.
      • Chernozhukov V.
      • Kasahara H.
      • Schrimpf P.
      Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S.
      , found that the only significant measure reducing the case growth was implementing mask wearing requirements for employees in public facing businesses.

      Least effective interventions

      There was no consistent evidence on the effectiveness of public transport closure or of the following three health system NPIs: testing strategies, contact tracing strategies and isolation or quarantine strategies.
      Only one of the 12 studies that analysed its effect found an association between public transport closures and the reproduction number, growth rate or case related outcomes.
      • Deb P.
      • Furceri D.
      • Ostry J.D.
      The effect of containment measures on the COVID-19 pandemic.
      Neither of the six studies analysing the effect of testing policies found testing policies to improve the outcome. Pozo-Martin et al.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      found in an analysis of the measures between October and December that both testing symptomatic and asymptomatic individuals was a predictor for a higher growth rate. None of the three studies analysing contact tracing strategies found a significant association with COVID-19 cases. Two out of four studies analysing isolation or quarantine strategies found an association with the outcome. However, in these studies these interventions were among the least effective. The findings regarding testing strategies, contact tracing strategies and isolation or quarantine strategies were consistent across intermediate and high-quality studies as rated in our review.

      Results on mortality and death-related outcomes

      A heatmap of the findings of all studies assessing mortality and death-related outcomes is shown in Fig. 3. Note that some studies can be found in both Figs. 2 and 3, since several authors assessed more than one outcome. Twelve studies considered mortality or death-related outcomes in their analysis: four studied mortality, three death growth rate, three studied confirmed deaths, one studied the incident risk ratio on the number of deaths and one the case fatality rate. The evidence of the comparative effectiveness of NPIs with respect to mortality-related outcomes is not clear. School closing seemed to be the most effective NPI, six out of ten studies (60%) analysing this measure found it was significantly associated with the outcome. International travel restrictions are associated with a decrease in mortality in 4 out of 7 studies (57%). Mask wearing requirements, public event bans and mass gathering bans show consistent association with the outcome in all of the studies assessing these interventions. However, mask wearing requirements and social distancing were only studied in two and one article respectively. In line with studies analysing epidemic growth and incidence-related outcomes, testing strategies and contact tracing strategies show no evidence of being associated with COVID-19 deaths.
      Fig 3
      Fig. 3Heatmap of the assessment of NPIs effectiveness in the studies analysing mortality and death-related outcomes. The color grading is given according to the effectiveness ranking of each article. Darker green means higher effectiveness among the NPIs studied in the same article. Gray means no significant association with the outcome. White means the NPI was not studied. If no ranking was established all the NPIs studied in the same article have the same shade of green. The rating provided is the result of the risk of bias analysis.
      g = growth rate; IRR = incident risk ratio of NPIs on the number of deaths; CFR = case fatality rate.
      Eight studies assessed both a death-related outcome and an incidence-related outcome, while four studies focused only in mortality related outcomes.
      • Fountoulakis K.N.
      • Fountoulakis N.K.
      • Koupidis S.A.
      • Prezerakos P.E.
      Factors determining different death rates because of the COVID-19 outbreak among countries.
      ,
      • Leffler C.T.
      • Ing E.
      • Lykins J.D.
      • Hogan M.C.
      • McKeown C.A.
      • Grzybowski A.
      Association of country-wide coronavirus mortality with demographics, testing, lockdowns, and public wearing of masks.
      ,
      • Piovani D.
      • Christodoulou M.N.
      • Hadjidemetriou A.
      • Pantavou K.
      • Zaza P.
      • Bagos P.G.
      • et al.
      Effect of early application of social distancing interventions on COVID-19 mortality over the first pandemic wave: an analysis of longitudinal data from 37 countries.
      ,
      • Stokes J.
      • Turner A.J.
      • Anselmi L.
      • Morciano M.
      • Hone T.
      The relative effects of non-pharmaceutical interventions on early COVID-19 mortality: natural experiment in 130 countries.
      Several studies that explored the effectiveness of NPIs in both types of outcomes found associations between them and the number of cases, reproduction number or growth rate that were not relevant for mortality.
      • Li Y.
      • Li M.
      • Rice M.
      • Zhang H.
      • Sha D.
      • Li M.
      • et al.
      The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
      ,
      • Dreher N.
      • Spiera Z.
      • McAuley F.M.
      • Kuohn L.
      • Durbin J.R.
      • Marayati N.F.
      • et al.
      Policy interventions, social distancing, and SARS-CoV-2 transmission in the United States: a retrospective state-level analysis.
      ,
      • Hunter P.
      • Colón-González F.
      • Brainard J.
      Impact of non-pharmaceutical interventions against COVID-19 in Europe: a quasi-experimental study.
      ,
      • Chaudhry R.
      • Dranitsaris G.
      • Mubashir T.
      • Bartoszko J.
      • Riazi S.
      A country level analysis measuring the impact of government actions, country preparedness and socioeconomic factors on COVID-19 mortality and related health outcomes.
      Hunter et al.
      • Hunter P.
      • Colón-González F.
      • Brainard J.
      Impact of non-pharmaceutical interventions against COVID-19 in Europe: a quasi-experimental study.
      found that mass gathering bans were relevant for reducing cases and deaths, but business closures only for deaths. Piovani et al.
      • Piovani D.
      • Christodoulou M.N.
      • Hadjidemetriou A.
      • Pantavou K.
      • Zaza P.
      • Bagos P.G.
      • et al.
      Effect of early application of social distancing interventions on COVID-19 mortality over the first pandemic wave: an analysis of longitudinal data from 37 countries.
      reached a similar conclusion regarding mass gathering bans, and also due to school closing. Both Papadopoulos et al.
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      and Chernozhukov et al.
      • Chernozhukov V.
      • Kasahara H.
      • Schrimpf P.
      Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S.
      found school closing to be the most effective NPI in reducing the number of deaths. In contrast, two authors found that the closure of schools decreased the number of deaths, but not the number of cases.
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      ,
      • Chernozhukov V.
      • Kasahara H.
      • Schrimpf P.
      Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S.
      Leffler et al.
      • Leffler C.T.
      • Ing E.
      • Lykins J.D.
      • Hogan M.C.
      • McKeown C.A.
      • Grzybowski A.
      Association of country-wide coronavirus mortality with demographics, testing, lockdowns, and public wearing of masks.
      concluded that in countries with cultural norms or government policies supporting public mask-wearing, per-capita coronavirus mortality increased on average 16.2% each week, as compared with 61.9% each week in remaining countries.

      Dose-response effect of NPIs

      The OxCGRT stringency index is a score for each country that provides information about the overall intensity of mostly social distancing policies implemented by the Governments in a certain moment of time.
      • Hale T.
      • Angrist N.
      • Goldszmidt R.
      • Kira B.
      • Petherick A.
      • Phillips T.
      • et al.
      A global panel database of pandemic policies (Oxford COVID-19 government response tracker).
      There are contradictory results about stringency being a predictor of improved outcomes. While Leffler et al.
      • Leffler C.T.
      • Ing E.
      • Lykins J.D.
      • Hogan M.C.
      • McKeown C.A.
      • Grzybowski A.
      Association of country-wide coronavirus mortality with demographics, testing, lockdowns, and public wearing of masks.
      could not find any association between the stringency index and mortality, Deb et al.
      • Deb P.
      • Furceri D.
      • Ostry J.D.
      The effect of containment measures on the COVID-19 pandemic.
      estimated that countries that have put in place stringent measures have reduced the number of confirmed cases and deaths by more than 200 percent relative to the absence of measures.
      Different levels of business closing were studied by several authors. Hunter et al.
      • Hunter P.
      • Colón-González F.
      • Brainard J.
      Impact of non-pharmaceutical interventions against COVID-19 in Europe: a quasi-experimental study.
      observed no additional value to closing all non-essential services in comparison to only initial business closure. In contrast, Brauner et al.
      • Brauner J.M.
      • Mindermann S.
      • Sharma M.
      • Johnston D.
      • Salvatier J.
      • Gavenčiak T.
      • et al.
      Inferring the effectiveness of government interventions against COVID-19.
      estimated that closing some high-risk businesses reduced the reproduction number by 31% while closing most nonessential businesses reduced it by 40%. Brauner et al. also found a dose-response effect for gathering bans: they estimated a 36% and 21% reduction in the reproduction number when limiting gatherings to 10 people or less, and to 100 people or less, respectively. Liu et al.
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
      reached similar conclusions, adding that restrictions on gatherings of more than 1000 people were not effective.
      Pozo-Martin et al.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      in an analysis of the first COVID-19 wave and Koh et al.
      • Koh W.C.
      • Naing L.
      • Wong J.
      Estimating the impact of physical distancing measures in containing COVID-19: an empirical analysis.
      reached a similar conclusion regarding workplace closure. Recommended workplace closure or staying at home had been effective, implying that voluntary physical distancing has played an important role. However, Pozo-Martin et al.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      also observed improved outcomes, when all‑but-essential workplaces were closed. Regarding mask wearing requirements, Pozo-Martin et al.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      found that the effectiveness increased when they were mandated for all public places in all geographical areas within a country.
      With respect to lockdown measures, Koh et al.
      • Koh W.C.
      • Naing L.
      • Wong J.
      Estimating the impact of physical distancing measures in containing COVID-19: an empirical analysis.
      suggest that early on in the outbreak complete lockdowns may be unnecessary to control viral transmission, because partial lockdowns show to be equally effective. This finding is supported by the analysis of Papadopoulos et al.
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      , which concluded that the maximum stringency of individual lockdown policies was not associated with reduced case numbers or mortality.
      Wibbens et al.
      • Wibbens P.D.
      • Koo W.W.Y.
      • McGahan A.M.
      Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
      performed a detailed analysis of the individual OxCGRT intensity levels of NPIs in the United States and concluded, that, in general, the higher the policy intensity, the higher the relative impact on reducing the growth of infections. However, the difference in some cases might not be sufficiently relevant to upscale the level of the measure, taken the socioeconomic burden associated. They found that school closure and travel restrictions needed to be implemented at maximum stringency to reach a high impact and public information campaigns are most impactful at the lowest recorded level. Stokes et al.
      • Stokes J.
      • Turner A.J.
      • Anselmi L.
      • Morciano M.
      • Hone T.
      The relative effects of non-pharmaceutical interventions on early COVID-19 mortality: natural experiment in 130 countries.
      associated stricter measures with reduced mortality.

      Timeliness of implementation

      Regarding the effect of time delays in NPI implementation in the incidence and incidence-related outcomes, most studies found an association between the time delay and worse outcomes. Koh et al.
      • Koh W.C.
      • Naing L.
      • Wong J.
      Estimating the impact of physical distancing measures in containing COVID-19: an empirical analysis.
      concluded that all NPIs have to be implemented early to be effective. Chaudhry et al.
      • Chaudhry R.
      • Dranitsaris G.
      • Mubashir T.
      • Bartoszko J.
      • Riazi S.
      A country level analysis measuring the impact of government actions, country preparedness and socioeconomic factors on COVID-19 mortality and related health outcomes.
      found that days to travel restrictions was positively associated with the number of cases. Both Islam et al.
      • Islam N.
      • Sharp S.J.
      • Chowell G.
      • Shabnam S.
      • Kawachi I.
      • Lacey B.
      • et al.
      Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries.
      and Jalali et al.
      • Jalali A.M.
      • Khoury S.G.
      • See J.
      • Gulsvig A.M.
      Delayed interventions, low compliance, and health disparities amplified the early spread of COVID-19.
      concluded that earlier implementation of lockdowns was associated with a greater reduction in incidence of COVID-19 and in the latter case they also found an association with early introduction of face mask requirements. Papadopoulos et al.
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      concluded that early timing of lockdown introduction is of greater importance than its stringency. In contrast, Pozo-Martin et al.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      did not find that a delay in the response was a predictor of epidemic growth in the OECD countries.
      All the studies that analysed the effect of timeliness in mortality or mortality-related outcomes found an association between early implementation and improved outcomes.
      • Fountoulakis K.N.
      • Fountoulakis N.K.
      • Koupidis S.A.
      • Prezerakos P.E.
      Factors determining different death rates because of the COVID-19 outbreak among countries.
      ,
      • Leffler C.T.
      • Ing E.
      • Lykins J.D.
      • Hogan M.C.
      • McKeown C.A.
      • Grzybowski A.
      Association of country-wide coronavirus mortality with demographics, testing, lockdowns, and public wearing of masks.
      ,
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      ,
      • Piovani D.
      • Christodoulou M.N.
      • Hadjidemetriou A.
      • Pantavou K.
      • Zaza P.
      • Bagos P.G.
      • et al.
      Effect of early application of social distancing interventions on COVID-19 mortality over the first pandemic wave: an analysis of longitudinal data from 37 countries.
      ,
      • Stokes J.
      • Turner A.J.
      • Anselmi L.
      • Morciano M.
      • Hone T.
      The relative effects of non-pharmaceutical interventions on early COVID-19 mortality: natural experiment in 130 countries.
      Papadopoulos et al.
      • Papadopoulos D.I.
      • Donkov I.
      • Charitopoulos K.
      The impact of lockdown measures on COVID-19: a worldwide comparison.
      , concluded that early generalised school closure, early generalised workplace closure, early restriction of international travel and early public information campaigns were independently associated with reduced national COVID-19 death rate. Similarly, Leffler et al.
      • Leffler C.T.
      • Ing E.
      • Lykins J.D.
      • Hogan M.C.
      • McKeown C.A.
      • Grzybowski A.
      Association of country-wide coronavirus mortality with demographics, testing, lockdowns, and public wearing of masks.
      found an association between early international travel restrictions and a reduction of COVID-19 per-capita mortality. Piovani et al.
      • Piovani D.
      • Christodoulou M.N.
      • Hadjidemetriou A.
      • Pantavou K.
      • Zaza P.
      • Bagos P.G.
      • et al.
      Effect of early application of social distancing interventions on COVID-19 mortality over the first pandemic wave: an analysis of longitudinal data from 37 countries.
      found that the early application of mass gatherings and school closures was associated with an important reduction in COVID-19 mortality. Fountoulakis  et al.
      • Fountoulakis K.N.
      • Fountoulakis N.K.
      • Koupidis S.A.
      • Prezerakos P.E.
      Factors determining different death rates because of the COVID-19 outbreak among countries.
      found that early implementation of public events bans was a crucial factor for reducing deaths.

      Effect of number of NPIs

      Some authors reported on the effects on the outcome depending of the number of NPIs implemented. Islam et al.
      • Islam N.
      • Sharp S.J.
      • Chowell G.
      • Shabnam S.
      • Kawachi I.
      • Lacey B.
      • et al.
      Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries.
      concluded that the implementation of any physical distancing intervention was associated with an overall reduction in COVID-19 incidence of 13%. Bo et al.
      • Bo Y.
      • Guo C.
      • Lin C.
      • Zeng Y.
      • Li H.B.
      • Zhang Y.
      • et al.
      Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020.
      and Jüni et al.
      • Jüni P.
      • Rothenbühler M.
      • Bobos P.
      • Thorpe K.E.
      • Da Costa B.R.
      • Fisman D.N.
      • et al.
      Impact of climate and public health interventions on the COVID-19 pandemic: a prospective cohort study.
      determined that the implementation of two or more types of NPIs was more effective for containing the spread of COVID-19 than implementing only one type. Bo et al.
      • Bo Y.
      • Guo C.
      • Lin C.
      • Zeng Y.
      • Li H.B.
      • Zhang Y.
      • et al.
      Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020.
      also found that all NPI implementations involving social distancing were associated with a greater decrease in the reproduction number than those not involving distancing and concluded that combinations with more types of NPIs seemed to be associated with slower epidemic growth.

      Discussion

      Based on 34 ecological studies identified, this systematic review found that, among the 16 NPIs studied, school closure has been the most effective in reducing COVID-19 cases during the first wave of the pandemic. Workplace closures, business or venue closures and public event bans were also consistently considered among the most effective measures in reducing the number of cases. Public information campaigns and mask wearing requirements also proved to be effective in controlling the pandemic, while having less disruptive effects on the population than other NPIs. In contrast, public transport closure, testing strategies, contact tracing strategies and isolation or quarantine strategies showed no evidence of being effective in the studies assessed. Most of the studies assessing mortality were not able to estimate a comparative effectiveness of the interventions. While early implementation was consistently associated with a higher effectiveness in reducing COVID-19 cases and deaths, the stringency of the interventions was not. NPIs are effective in controlling the spread of COVID-19. An early response and a combination of specific social distancing measures are effective at reducing COVID-19 cases and deaths.
      We found that the most effective NPI was school closing. This NPI has been widely used since the beginning of the pandemic due to its effectiveness against influenza outbreaks.
      • Cauchemez S.
      • Ferguson N.M.
      • Wachtel C.
      • Tegnell A.
      • Saour G.
      • Duncan B.
      • et al.
      Closure of schools during an influenza pandemic.
      • Rashid H.
      • Ridda I.
      • King C.
      • Begun M.
      • Tekin H.
      • Wood J.G.
      • et al.
      Evidence compendium and advice on social distancing and other related measures for response to an influenza pandemic.
      • Ferguson N.M.
      • Cummings D.A.
      • Fraser C.
      • Cajka J.C.
      • Cooley P.C.
      • Burke D.S.
      Strategies for mitigating an influenza pandemic.
      However, school closing carries a heavy socioeconomical burden, hindering education and social interactions for children and causing additional child-care obligations for parents, linked to work absenteeism.
      • Bayham J.
      • Fenichel E.P.
      Impact of school closures for COVID-19 on the US health-care workforce and net mortality: a modeling study.
      There are conflicting results in the literature regarding the effectiveness of this NPI in mitigating the COVID-19 pandemic since it is still not clear if SARS-CoV-2 transmission occurs differently among children.
      • Mehta N.S.
      • Mytton O.T.
      • Mullins E.W.S.
      • Fowler T.A.
      • Falconer C.L.
      • Murphy O.B.
      • et al.
      SARS-CoV-2 (COVID-19): what do we know about children? A systematic review.
      Some authors have concluded that school closing is not an effective NPI and that COVID-19 control can be reached without this measure.
      • Munro A.P.S.
      • Faust S.N.
      Children are not COVID-19 super spreaders: time to go back to school.
      • Munro A.P.S.
      • Faust S.N.
      Addendum to: children are not COVID-19 super spreaders: time to go back to school.
      • Macartney K.
      • Quinn H.E.
      • Pillsbury A.J.
      • Koirala A.
      • Deng L.
      • Winkler N.
      • et al.
      Transmission of SARS-CoV-2 in Australian educational settings: a prospective cohort study.
      Viner et al.
      • Viner R.M.
      • Russell S.J.
      • Croker H.
      • Packer J.
      • Ward J.
      • Stansfield C.
      • et al.
      School closure and management practices during coronavirus outbreaks including COVID-19: a rapid systematic review.
      performed a systematic review on the effects of school closing on the spread of respiratory diseases and included four modeling studies at the beginning of the COVID-19 pandemic. They found that the effect of school closures was comparatively lower than other measures. In a recent systematic review of observational studies analysing school closing and reopening Walsh et al.
      • Walsh S.
      • Chowdury A.
      • Braithwaite V.
      • Russell S.
      • Birch J.
      • Ward J.
      • et al.
      Do school closures and school reopenings affect community transmission of COVID-19? A systematic review of observational studies.
      found that half of the studies with lower risk of bias concluded that school closing reduced community transmission while half of the studies found no effect. Other analyses
      • Auger K.A.
      • Shah S.S.
      • Richardson T.
      • Hartley D.
      • Hall M.
      • Warniment A.
      • et al.
      Association between statewide school closure and COVID-19 incidence and mortality in the US.
      Should Schools Reopen? Interim Findings and Concerns
      Draft document for public discussion.
      • Abdollahi E.
      • Haworth-Brockman M.
      • Keynan Y.
      • Langley J.M.
      • Moghadas S.M.
      Simulating the effect of school closure during COVID-19 outbreaks in Ontario.
      • Post R.A.J.
      • Regis M.
      • Zhan Z.
      • van den Heuvel E.R.
      How did governmental interventions affect the spread of COVID-19 in European countries?.
      • Klimek-Tulwin M.
      • Tulwin T.
      Early school closures can reduce the first-wave of the COVID-19 pandemic development.
      • Staguhn E.D.
      • Castillo R.C.
      • Weston-Farber E.
      The impact of statewide school closures on COVID-19 infection rates.
      suggest that school closing may have been a more important factor during the first wave of the COVID-19 epidemic than initially thought. As the epidemiological situation has improved, reopening schools has become imperative. Walsh et al.
      • Walsh S.
      • Chowdury A.
      • Braithwaite V.
      • Russell S.
      • Birch J.
      • Ward J.
      • et al.
      Do school closures and school reopenings affect community transmission of COVID-19? A systematic review of observational studies.
      found that there is no increase in community transmission of COVID-19 after reopening schools in a low transmission context with appropriate mitigation measures, as it was observed in Norway and Denmark.
      • Stage H.B.
      • Shingleton J.
      • Ghosh S.
      • Scarabel F.
      • Pellis L.
      • Finnie T.
      Shut and reopen: the role of schools in the spread of COVID-19 in Europe.
      However, in countries like Germany, that reopened schools when community transmission was still high, it can lead to an increase in the growth rate.
      • Stage H.B.
      • Shingleton J.
      • Ghosh S.
      • Scarabel F.
      • Pellis L.
      • Finnie T.
      Shut and reopen: the role of schools in the spread of COVID-19 in Europe.
      In the United Kingdom schools have been closed twice, first during the first round of restrictions starting March 2020, reopening between June and August 2020. Second, schools closed in December 2020 or January 2021 and reopened in March and April 2021.
      • Tatlow H.
      • Cameron-Blake E.
      • Grewal S.
      • Hale T.
      • Philips T.
      • Wood A.
      Variation in the response to COVID-19 across the four nations of the United Kingdom.
      Mitigation measures have been established for schools to reopen across the United Kingdom, including mask wearing and testing,

      BBC. COVID: When will the children return to school and what are the rules? [Available from: https://www.bbc.com/news/education-51643556].

      and after the reopening in both instances cases still went down.

      OurWorldInData. Daily confirmed COVID-19 cases in the UK [Available from: https://ourworldindata.org/coronavirus-data?country=~GBR.

      Indeed, several studies worldwide have found that mitigation measures allow to reopen schools safely.
      • Lessler J.
      • Grabowski M.K.
      • Grantz K.H.
      • Badillo-Goicoechea E.
      • Metcalf C.J.E.
      • Lupton-Smith C.
      • et al.
      Household COVID-19 risk and in-person schooling.
      • Krishnaratne S.
      • Pfadenhauer L.M.
      • Coenen M.
      • Geffert K.
      • Jung-Sievers C.
      • Klinger C.
      • et al.
      Measures implemented in the school setting to contain the COVID-19 pandemic: a rapid scoping review.
      • Gillespie D.L.
      • Meyers L.A.
      • Lachmann L.
      • Redd S.C.
      • Zenilman J.M.
      The experience of 2 independent schools with in-person learning during the COVID-19 pandemic.

      J. Bailey Is it safe to reopen schools? An extensive review of the research. Available at: https://www.crpe.org/sites/default/files/3-12_is_it_safe_to_reopen_schools_an_extensive_review_of_the_research_1.pdf. 2021.

      Lessler et al.
      • Lessler J.
      • Grabowski M.K.
      • Grantz K.H.
      • Badillo-Goicoechea E.
      • Metcalf C.J.E.
      • Lupton-Smith C.
      • et al.
      Household COVID-19 risk and in-person schooling.
      found that even implementing low levels of in-school mitigation measures COVID-19 outcomes were reduced. On average, each measure implemented was associated with a 9% decrease in the odds of COVID-19-like illness.
      • Lessler J.
      • Grabowski M.K.
      • Grantz K.H.
      • Badillo-Goicoechea E.
      • Metcalf C.J.E.
      • Lupton-Smith C.
      • et al.
      Household COVID-19 risk and in-person schooling.
      In conclusion, a cautious approach for reopening should be adapted to each context, with specific mitigation measures, stepwise opening and monitoring the effects of reopening for in-school and community transmission.
      We found other social distancing NPIs to be consistently effective in controlling the COVID-19 pandemic: workplace closing, business and venue closing and public event bans. As reflected in the results of this review, there are differences in the effectiveness of closing all businesses or targeted ones. Further, business and venue closures affect the economy disproportionally
      • Bartik A.W.
      • Bertrand M.
      • Cullen Z.
      • Glaeser E.L.
      • Luca M.
      • Stanton C.
      The impact of COVID-19 on small business outcomes and expectations.
      and therefore, careful consideration for implementing these measures needs to be taken. A tailored approach is necessary for each context as workplace closures can pose different social or psychological problems to workers.
      • Kniffin K.M.
      • Narayanan J.
      • Anseel F.
      • Antonakis J.
      • Ashford S.P.
      • Bakker A.B.
      • et al.
      COVID-19 and the workplace: implications, issues, and insights for future research and action.
      Nonetheless, strategies can be adopted to reduce their potential negative effects.
      • Kniffin K.M.
      • Narayanan J.
      • Anseel F.
      • Antonakis J.
      • Ashford S.P.
      • Bakker A.B.
      • et al.
      COVID-19 and the workplace: implications, issues, and insights for future research and action.
      Another widely used measure is the ban of public events. At the beginning of the pandemic several public events were considered “super spreader events” in China
      • Fan C.
      • Liu L.
      • Guo W.
      • Yang A.
      • Ye C.
      • Jilili M.
      • et al.
      Prediction of epidemic spread of the 2019 novel coronavirus driven by spring festival transportation in China: a population-based study.
      and in other countries.
      • Lau M.S.Y.
      • Grenfell B.
      • Thomas M.
      • Bryan M.
      • Nelson K.
      • Lopman B.
      Characterizing superspreading events and age-specific infectiousness of SARS-CoV-2 transmission in Georgia, USA.
      ,
      • Streeck H.
      • Schulte B.
      • Kümmerer B.M.
      • Richter E.
      • Höller T.
      • Fuhrmann C.
      • et al.
      Infection fatality rate of SARS-CoV2 in a super-spreading event in Germany.
      A super spreader event describes a situation when only one or some positive cases infect many people. Sun et al.
      • Sun K.
      • Wang W.
      • Gao L.
      • Wang Y.
      • Luo K.
      • Ren L.
      • et al.
      Transmission heterogeneities, kinetics, and controllability of SARS-CoV-2.
      estimated that 15% of the people accounted for 80% of the infections in the Hunan province in China in early 2020. Estimations for local transmission in Hong Kong
      • Adam D.C.
      • Wu P.
      • Wong J.Y.
      • Lau E.H.Y.
      • Tsang T.K.
      • Cauchemez S.
      • et al.
      Clustering and superspreading potential of SARS-CoV-2 infections in Hong Kong.
      and other locations
      • Liu Y.
      • Eggo R.M.
      • Kucharski A.J.
      Secondary attack rate and superspreading events for SARS-CoV-2.
      back up these findings. The existence of super spreaders is considered to be a common characteristic of coronaviruses, and it is related with several factors, like prolonged indoor gatherings with poor ventilation.
      • Lewis D.
      Superspreading drives the COVID pandemic - and could help to tame it.
      However, the exact reasons why some individuals are able to infect many people and other individuals only a few remain unclear. Note that super-spreader events are not avoidable only via public event bans, but also through venue closures and gathering restrictions, which were found to be effective NPIs in our analysis. In a mathematical model developed by Chang et al.
      • Chang S.
      • Pierson E.
      • Koh P.W.
      • Gerardin J.
      • Redbird B.
      • Grusky D.
      • et al.
      Mobility network models of COVID-19 explain inequities and inform reopening.
      using mobility data, they observed that restaurants, cafes and gyms could account for most COVID-19 infections in US cities, with 80% of the predicted transmission being linked to 10% of the locations.
      Our results regarding several health system interventions need to be cautiously interpreted. For instance, testing and contact tracing policies and isolation or quarantining, which are standard public health activities, were not found to be effective in controlling the COVID-19 pandemic. This can be attributed to several factors. First, if the outcome of the study is related to the number of cases, when case detection improves through more efficient testing and tracing the case number reported will rise, without representing a real rise of cases. Second, countries that have implemented successful control strategies strongly relying on these interventions, like China,
      • Tang B.
      • Xia F.
      • Tang S.
      • Bragazzi N.L.
      • Li Q.
      • Sun X.
      • et al.
      The effectiveness of quarantine and isolation determine the trend of the COVID-19 epidemics in the final phase of the current outbreak in China.
      ,
      • Zu J.
      • Li M.L.
      • Li Z.F.
      • Shen M.W.
      • Xiao Y.N.
      • Ji F.P.
      Transmission patterns of COVID-19 in the mainland of China and the efficacy of different control strategies: a data- and model-driven study.
      South Korea,
      • Jeong G.H.
      • Lee H.J.
      • Lee J.
      • Lee J.Y.
      • Lee K.H.
      • Han Y.J.
      • et al.
      Effective Control of COVID-19 in South Korea: cross-sectional study of epidemiological data.
      Singapore
      • Lee V.J.
      • Chiew C.J.
      • Khong W.X.
      Interrupting transmission of COVID-19: lessons from containment efforts in Singapore.
      and New Zealand
      • Baker M.G.
      • Wilson N.
      • Anglemyer A.
      Successful elimination of COVID-19 transmission in New Zealand.
      have not been specifically addressed among the studies included in this review. In our analysis, testing and contact tracing policies and isolation or quarantining were only assessed in few studies among those that met the inclusion criteria. However, the findings were consistent across intermediate and high-quality studies. In the existing literature, testing and contact tracing strategies, followed by quarantining or isolation, have been considered essential in controlling COVID-19 spread. It has been observed that higher testing volume or testing coverage are correlated with improved control of the pandemic.
      • Pozo-Martin F.
      • Weishaar H.
      • Cristea F.
      • Hanefeld J.
      • Bahr T.
      • Schaade L.
      • et al.
      The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
      ,
      • Vicentini C.
      • Bazzolo S.
      • Gamba D.
      • Zotti C.M.
      Analysis of the fatality rate in relation to testing capacity during the first 50 days of the COVID-19 epidemic in Italy.
      • Saglietto A.
      • Moirano G.
      • Anselmino M.
      • De Ferrari G.M.
      Higher testing coverage is associated with lower COVID-19 mortality rate: insights from Italian regions.
      • Wei C.
      • Lee C.C.
      • Hsu T.C.
      • Hsu W.T.
      • Chan C.C.
      • Chen S.C.
      • et al.
      Correlation of population mortality of COVID-19 and testing coverage: a comparison among 36 OECD countries.
      However, in this review we assessed the impact of different testing strategies, meaning which individuals get tested and how, not the number of tests performed. Several studies have highlighted the importance of a comprehensive test, trace and quarantine approach in different contexts.
      • Salathé M.
      • Althaus C.L.
      • Neher R.
      • Stringhini S.
      • Hodcroft E.
      • Fellay J.
      • et al.
      COVID-19 epidemic in Switzerland: on the importance of testing, contact tracing and isolation.
      ,
      • Steinbrook R.
      Contact tracing, testing, and control of COVID-19-learning from Taiwan.
      Hellewell et al.
      • Hellewell J.
      • Abbott S.
      • Gimma A.
      • Bosse N.I.
      • Jarvis C.I.
      • Russell T.W.
      • et al.
      Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts.
      found through a modeling study that the pandemic could be controlled under certain testing and tracing strategies and concluded that the most important factor in determining whether an outbreak was controllable or not was the delay between symptom onset and isolation. A recent systematic review comparing mass testing and contact tracing with conventional test and trace strategies concluded that mass testing could be more effective in controlling the pandemic.
      • Mbwogge M.
      Mass testing with contact tracing compared to test and trace for the effective suppression of COVID-19 in the United Kingdom: systematic review.
      Interestingly, another effective health system measure is the adoption of public information campaigns by the governments. Although this intervention is not among the most effective measures in our analysis, it is consistently associated with a reduction of COVID-19 cases. Mask wearing requirements were also consistently effective in reducing COVID-19 cases in our review. The use of masks can be associated to individual discomfort, but it does not present such an important disruption for daily activities as most of the other measures analysed in this review. Much research has been produced around the use of face masks and its effectiveness, mostly confirming its positive impact in controlling the virus spread.
      • Howard J.
      • Huang A.
      • Li Z.
      • Tufekci Z.
      • Zdimal V.
      • van der Westhuizen H.-.M.
      • et al.
      An evidence review of face masks against COVID-19.
      ,
      • Cheng V.C.
      • Wong S.C.
      • Chuang V.W.
      • So S.Y.
      • Chen J.H.
      • Sridhar S.
      • et al.
      The role of community-wide wearing of face mask for control of coronavirus disease 2019 (COVID-19) epidemic due to SARS-CoV-2.
      Through mask wearing requirements workplaces, schools and businesses have been allowed to open. Considering the disruptive effects of most NPIs and their high societal and economical cost, the implementation of effective public information campaigns and mask requirements can present large benefits with less efforts and socioeconomical consequences than other NPIs.
      Similarly, due to the socio-economic burden associated with scaling up some NPIs, the specific context needs to be considered. As reflected in the results of this review, upscaling the level of some measures does not always imply improved outcomes. Besides, the difference in the reduction of cases or deaths might not be sufficiently relevant to implement more stringent measures, considering its socioeconomical impact.
      Our findings rely mostly on the analysis of studies published based on data from the first wave of the COVID-19 pandemic. However, several studies analysing the effectiveness of NPIs during the second wave have been released as preprints recently. Sharma et al.
      • Sharma M.
      • Mindermann S.
      • Rogers-Smith C.
      • Leech G.
      • Snodin B.
      • Ahuja J.
      • et al.
      Understanding the effectiveness of government interventions in Europe’s second wave of COVID-19.
      studied the implementation effects of 17 NPIs in 114 subnational areas from 7 European countries. They found that the combined effect of general NPI implementation was smaller during the second wave, which can be attributed, among other factors, to the influence of maintained individual protective behaviours after the first wave. In line with the results of our review, they concluded that in the second wave closing specific businesses was highly effective, together with strict small gathering restrictions. In contrast, they estimate that school closing was not as effective during the second wave in comparison to the first, which could be linked to the control measures adopted in schools after reopening. In another preprint, Ge et al.
      • Ge Y.
      • Zhang W.B.
      • Liu H.
      • Ruktanonchai C.W.
      • Hu M.
      • Wu X.
      • et al.
      Effects of worldwide interventions and vaccination on COVID-19 between waves and countries.
      analysed the effect of NPIs from the first wave until March 2021 in 133 countries, assessing vaccine rollouts as well. Consistent with Sharma et al.
      • Sharma M.
      • Mindermann S.
      • Rogers-Smith C.
      • Leech G.
      • Snodin B.
      • Ahuja J.
      • et al.
      Understanding the effectiveness of government interventions in Europe’s second wave of COVID-19.
      , Ge et al.
      • Ge Y.
      • Zhang W.B.
      • Liu H.
      • Ruktanonchai C.W.
      • Hu M.
      • Wu X.
      • et al.
      Effects of worldwide interventions and vaccination on COVID-19 between waves and countries.
      observed differences in the effectiveness of NPIs during the different waves; they found that school closing was the most effective measure during the first wave, but not among the most relevant in the second. Gathering restrictions and facial covering requirements were consistently considered effective among waves, whereas international travel restrictions played a more important role in the control of the second wave. Regarding vaccine rollout, they considered that vaccination was increasingly contributing to the pandemic control, despite its effect having a significantly lower impact than the NPIs by the time of the study.
      As in every evidence review, the comparability of the studies analysed depends on their design and methodological heterogeneity. First, different outcomes were studied to assess the effectiveness of NPIs among the studies. These were mainly the reproduction number, the growth rate, the number of cases and the number of deaths. Mortality data gives only information about severe cases, but is less influenced by testing strategies and testing capacity than case counts.
      • Adam D.
      A guide to R-the pandemic’s misunderstood metric.
      ,
      • Anderson R.
      • Donnelly C.
      • Hollingsworth D.
      Reproduction number (R) and growth rate (r) of the COVID-19 epidemic in the UK: methods of estimation, data sources, causes of heterogeneity, and use as a guide in policy formulation.
      Testing and contact tracing capacity has been an essential constraint during the pandemic, especially at the beginning. Growth rate is more easily calculated than the reproduction number and avoids many inferential difficulties in estimating the latter.
      • Anderson R.
      • Donnelly C.
      • Hollingsworth D.
      Reproduction number (R) and growth rate (r) of the COVID-19 epidemic in the UK: methods of estimation, data sources, causes of heterogeneity, and use as a guide in policy formulation.
      For calculating the reproduction number several epidemiological assumptions, like time until symptom onset, time until death or serial interval, need to be estimated.
      • Adam D.
      A guide to R-the pandemic’s misunderstood metric.
      ,
      • Anderson R.
      • Donnelly C.
      • Hollingsworth D.
      Reproduction number (R) and growth rate (r) of the COVID-19 epidemic in the UK: methods of estimation, data sources, causes of heterogeneity, and use as a guide in policy formulation.
      However, the reproduction number provides more information than the growth rate on the impact of control measures given the non-linear epidemic curve of COVID-19.
      • Anderson R.
      • Donnelly C.
      • Hollingsworth D.
      Reproduction number (R) and growth rate (r) of the COVID-19 epidemic in the UK: methods of estimation, data sources, causes of heterogeneity, and use as a guide in policy formulation.
      If the proportion of cases that are unreported remains constant throughout the study time, estimates of the reproduction number are unaffected by underreporting.
      • Thompson R.N.
      • Stockwin J.E.
      • van Gaalen R.D.
      • Polonsky J.A.
      • Kamvar Z.N.
      • Demarsh P.A.
      • et al.
      Improved inference of time-varying reproduction numbers during infectious disease outbreaks.
      Second, the number of NPIs analysed and their definitions differ among the studies included in our review. Some homogeneity is expected for studies with the same NPIs data source, like the 14 studies using the OxCGRT dataset (details in Annex 3). However, there are some differing definitions of specific NPIs among the studies. For instance, school closure can include or exclude secondary schools and/or universities, the amount of people assessed for social gathering restrictions varies and there are differences in the definition of lockdown, business closure or social distancing requirements. Furthermore, some authors consider the onset of a policy when it is officially recommended and others when it is enforced. Several studies consider both options by analysing different stringency levels instead of applying a binary approach.
      There are some limitations in the body of evidence of this review. First, all the studies use retrospective and observational data to draw inferences about the effectiveness of NPIs. Conclusions from these studies are limited to the specific time and places studied and may be affected by confounding effects from unobserved factors. However, the broad geographical scope of the studies and the very different methodological approaches used to answer the study questions increase the robustness of our findings. Second, only studies that established a direct link between the implementation of NPIs and an outcome were included in the review. Therefore, it is assumed by all studies that the effect on the outcome depends only on the implementation of the measures. Nonetheless, individual behavior, even before the implementation of measures,
      • Chernozhukov V.
      • Kasahara H.
      • Schrimpf P.
      Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S.
      ,
      • Teslya A.
      • Pham T.M.
      • Godijk N.G.
      • Kretzschmar M.E.
      • Bootsma M.C.J.
      • Rozhnova G.
      Impact of self-imposed prevention measures and short-term government-imposed social distancing on mitigating and delaying a COVID-19 epidemic: a modeling study.
      and the compliance with the NPIs
      • Kasting M.L.
      • Head K.J.
      • Hartsock J.A.
      • Sturm L.
      • Zimet G.D.
      Public perceptions of the effectiveness of recommended non-pharmaceutical intervention behaviors to mitigate the spread of SARS-CoV-2.
      ,
      • Kantor B.N.
      • Kantor J.
      Non-pharmaceutical interventions for pandemic COVID-19: a cross-sectional investigation of US general public beliefs, attitudes, and actions.
      have played an essential role in controlling the pandemic. Third, all studies included in this review assess the effectiveness of NPIs during the first wave of the pandemic. This improves comparability among the studies: during the first months of the pandemic people were still adapting to protective behaviours, almost all the population was susceptible to the virus and no vaccine immunization had started, so that the effect of NPIs implementation could be more directly linked to the outcomes. However, these studies are less relevant to understand the effect of “controlled openings”, for instance businesses opening with improved hygiene conditions, adapted workplaces and schools with social distancing protocols and openings relying on testing. The worldwide availably of personal protective equipment, hospital equipment and testing material has also remarkably improved since the beginning of the pandemic. Finally, there is an overrepresentation of high-income countries in the region-targeted studies, mainly the USA and Europe, but also most OECD countries. There are neither studies focused on low- or middle-income countries nor on Asia or Oceania, even though some of these countries are included in the worldwide studies. Comparing measures within Europe and the USA leads to analysing territories with more similarities in pandemic control approaches. Many Asian countries have had recent experience with pandemics before COVID-19
      • Chan E.Y.
      • Cheng C.K.
      • Tam G.
      • Huang Z.
      • Lee P.
      Knowledge, attitudes, and practices of Hong Kong population towards human A/H7N9 influenza pandemic preparedness, China, 2014.
      ,
      • Lam W.K.
      • Zhong N.S.
      • Tan W.C.
      Overview on SARS in Asia and the world.
      and, as stated before, some of them relied strongly on health system interventions. Regarding low- and middle-income countries, fewer economic resources translate to less surveillance and testing capacity, which increases underreporting and hinders the assessment of effectiveness of NPIs. However, in the first pandemic wave many Sub-Saharan African countries implemented social distancing measures before the first detected case,
      • Liu Y.
      • Morgenstern C.
      • Kelly J.
      • Lowe R.
      • Jit M.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
      which may have played a role in the low burden of disease encountered in the region during that time.
      • Salyer S.J.
      • Maeda J.
      • Sembuche S.
      • Kebede Y.
      • Tshangela A.
      • Moussif M.
      • et al.
      The first and second waves of the COVID-19 pandemic in Africa: a cross-sectional study.
      Nonetheless, in Africa, the second wave has been more severe, and despite this situation the stringency of the measures implemented in the continent is decreasing.
      • Salyer S.J.
      • Maeda J.
      • Sembuche S.
      • Kebede Y.
      • Tshangela A.
      • Moussif M.
      • et al.
      The first and second waves of the COVID-19 pandemic in Africa: a cross-sectional study.
      The knowledge of NPIs effectiveness for specific regions should not only be limited to high-income counties in order to allow policy makers worldwide to make tailored decisions.
      This systematic review has certain limitations. First, we did not include in our review the evidence coming from mathematical studies simulating the impact of NPIs on epidemic control. While they generate high quality evidence, these studies rely on assumptions about the type and intensity of the NPIs being implemented rather than on the actual policy implementation. In contrast, this review provides information from data-driven studies that have explored the real-life impact of NPI implementation. Second, due to the novelty of the topic and the urgency to answer the study question we included preprints in the review. Even though these studies have not yet undergone peer review, a risk of bias tool was used to assess their quality. All of the preprints included were considered to have sufficient quality and add relevant evidence to the review. Regarding the quality assessment, there is no consolidated risk of bias tool for evaluating ecological studies. Nonetheless, we used a published tool that has been validated in several systematic reviews of ecological studies before.
      • Cortes-Ramirez J.
      • Naish S.
      • Sly P.D.
      • Jagals P.
      Mortality and morbidity in populations in the vicinity of coal mining: a systematic review.
      ,
      • Betran A.P.
      • Torloni M.R.
      • Zhang J.
      • Ye J.
      • Mikolajczyk R.
      • Deneux-Tharaux C.
      • et al.
      What is the optimal rate of caesarean section at population level? A systematic review of ecologic studies.
      To the best of our knowledge, at the time of writing there is no other review of empirical studies assessing the comparative effectiveness of more than two NPIs against COVID-19. We found that school closing, followed by workplace closing, business and venue closing and public event bans were the most effective NPIs in controlling the spread of COVID-19. Public information campaigns and mask wearing requirements, less disruptive to the population than other NPIs, were also effective measures. An early response and a combination of specific social distancing measures are effective at reducing COVID-19 cases and deaths. Since scientific knowledge, individual behavior and resources keep on changing and adapting throughout the pandemic, more research needs to be targeted to understand changes in the effectiveness of NPIs and whether controlled openings and lifting of restrictions are compatible with epidemic control.

      Declaration of Competing Interest

      None.

      Acknowledgments

      Funding sources: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

      Appendix. Supplementary materials

      References

      1. Stop the Wuhan virus.
        Nature. 2020; 577: 450https://doi.org/10.1038/d41586-020-00153-x
      2. Worldometers. COVID-19 Coronavirus Pandemic [Available from: https://www.worldometers.info/coronavirus/?utm_campaign=homeAdvegas1?]

        • Lai S.
        • Ruktanonchai N.W.
        • Zhou L.
        • Prosper O.
        • Luo W.
        • Floyd J.R.
        • et al.
        Effect of non-pharmaceutical interventions to contain COVID-19 in China.
        Nature. 2020; 585: 410-413https://doi.org/10.1038/s41586-020-2293-x
        • Thu T.P.B.
        • Ngoc P.N.H.
        • Hai N.M.
        • Tuan L.A.
        Effect of the social distancing measures on the spread of COVID-19 in 10 highly infected countries.
        Sci Total Environ. 2020; : 742https://doi.org/10.1016/j.scitotenv.2020.140430
        • Sebastiani G.
        • Massa M.
        • Riboli E.
        Covid-19 epidemic in Italy: evolution, projections and impact of government measures.
        Eur J Epidemiol. 2020; 35: 341-345https://doi.org/10.1007/s10654-020-00631-6
        • Askitas N.
        • Tatsiramos K.
        • Verheyden B.
        Estimating worldwide effects of non-pharmaceutical interventions on COVID-19 incidence and population mobility patterns using a multiple-event study.
        Sci Rep. 2021; 11: 1972https://doi.org/10.1038/s41598-021-81442-x
        • Nicola M.
        • Alsafi Z.
        • Sohrabi C.
        • Kerwan A.
        • Al-Jabir A.
        • Iosifidis C.
        • et al.
        The socio-economic implications of the coronavirus pandemic (COVID-19): a review.
        Int J Surg. 2020; 78: 185-193https://doi.org/10.1016/j.ijsu.2020.04.018
        • Trougakos J.P.
        • Chawla N.
        • McCarthy J.M.
        Working in a pandemic: exploring the impact of COVID-19 health anxiety on work, family, and health outcomes.
        J Appl Psychol. 2020; 105: 1234-1245https://doi.org/10.1037/apl0000739
        • Peak C.M.
        • Childs L.M.
        • Grad Y.H.
        • Buckee C.O.
        Comparing nonpharmaceutical interventions for containing emerging epidemics.
        Proc Natl Acad Sci. 2017; 114 (U S A.): 4023-4028https://doi.org/10.1073/pnas.1616438114
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        PLoS Med. 2009; 6e1000097https://doi.org/10.1371/journal.pmed.1000097
        • Hale T.
        • Angrist N.
        • Goldszmidt R.
        • Kira B.
        • Petherick A.
        • Phillips T.
        • et al.
        A global panel database of pandemic policies (Oxford COVID-19 government response tracker).
        Nat Hum Behav. 2021; 5: 529-538https://doi.org/10.1038/s41562-021-01079-8
        • Dufault B.
        • Klar N.
        The quality of modern cross-sectional ecologic studies: a bibliometric review.
        Am J Epidemiol. 2011; 174: 1101-1107https://doi.org/10.1093/aje/kwr241
        • Cortes-Ramirez J.
        • Naish S.
        • Sly P.D.
        • Jagals P.
        Mortality and morbidity in populations in the vicinity of coal mining: a systematic review.
        BMC Public Health. 2018; 18: 721https://doi.org/10.1186/s12889-018-5505-7
        • Betran A.P.
        • Torloni M.R.
        • Zhang J.
        • Ye J.
        • Mikolajczyk R.
        • Deneux-Tharaux C.
        • et al.
        What is the optimal rate of caesarean section at population level? A systematic review of ecologic studies.
        Reprod Health. 2015; 12: 57https://doi.org/10.1186/s12978-015-0043-6
        • Hsiang S.
        • Allen D.
        • Annan-Phan S.
        • Bell K.
        • Bolliger I.
        • Chong T.
        • et al.
        The effect of large-scale anti-contagion policies on the COVID-19 pandemic.
        Nature. 2020; 584: 262-267https://doi.org/10.1038/s41586-020-2404-8
        • Jüni P.
        • Rothenbühler M.
        • Bobos P.
        • Thorpe K.E.
        • Da Costa B.R.
        • Fisman D.N.
        • et al.
        Impact of climate and public health interventions on the COVID-19 pandemic: a prospective cohort study.
        CMAJ. 2020; 192: E566-EE73https://doi.org/10.1503/cmaj.200920
        • Bo Y.
        • Guo C.
        • Lin C.
        • Zeng Y.
        • Li H.B.
        • Zhang Y.
        • et al.
        Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020.
        Int J Infect Dis. 2021; 102: 247-253https://doi.org/10.1016/j.ijid.2020.10.066
        • Wibbens P.D.
        • Koo W.W.Y.
        • McGahan A.M.
        Which COVID policies are most effective? A Bayesian analysis of COVID-19 by jurisdiction.
        PLoS ONE. 2020; 15: e0244177https://doi.org/10.1371/journal.pone.0244177
        • Pozo-Martin F.
        • Weishaar H.
        • Cristea F.
        • Hanefeld J.
        • Bahr T.
        • Schaade L.
        • et al.
        The impact of non-pharmaceutical interventions on COVID-19 epidemic growth in the 37 OECD member states.
        Eur J Epidemiol. 2021; https://doi.org/10.1007/s10654-021-00766-0
        • Zhang X.
        • Warner M.E.
        Covid-19 policy differences across us states: shutdowns, reopening, and mask mandates.
        Int J Environ Res Public Health. 2020; 17: 9520https://doi.org/10.3390/ijerph17249520
        • Duhon J.
        • Bragazzi N.
        • Kong J.D.
        The impact of non-pharmaceutical interventions, demographic, social, and climatic factors on the initial growth rate of COVID-19: a cross-country study.
        Sci Total Environ. 2021; 760https://doi.org/10.1016/j.scitotenv.2020.144325
        • Ebrahim S.
        • Ashworth H.
        • Noah C.
        • Kadambi A.
        • Toumi A.
        • Chhatwal J.
        Reduction of COVID-19 incidence and nonpharmacologic interventions: analysis using a US county-level policy data set.
        J Med Internet Res. 2020; 22
        • Li Y.
        • Campbell H.
        • Kulkarni D.
        • Harpur A.
        • Nundy M.
        • Wang X.
        • et al.
        The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
        Lancet Infect Dis. 2021; 21: 193-202https://doi.org/10.1016/S1473-3099(20)30785-4
        • Banholzer N.
        • van Weenen E.
        • Kratzwald B.
        Impact of non-pharmaceutical interventions on documented cases of COVID-19.
        MedRxiv. 2020; https://doi.org/10.1101/2020.04.16.20062141
        • Banholzer N.
        • van Weenen E.
        • Lison A.
        Estimating the effects of non-pharmaceutical interventions on the number of new infections with COVID-19 during the first epidemic wave.
        PLoS ONE. 2021; 16https://doi.org/10.1371/journal.pone.0252827
        • Li Y.
        • Li M.
        • Rice M.
        • Zhang H.
        • Sha D.
        • Li M.
        • et al.
        The impact of policy measures on human mobility, COVID-19 cases, and mortality in the US: a spatiotemporal perspective.
        Int J Environ Res Public Health. 2021; 18: 996https://doi.org/10.3390/ijerph18030996
        • Esra R.
        • Jamieson L.
        • Fox M.P.
        • Letswalo D.
        Evaluating the impact of non-pharmaceutical interventions for SARS-CoV-2 on a global scale.
        MedRxiv. 2020; https://doi.org/10.1101/2020.07.30.20164939