Let’s start here: I have no idea whether mask mandates work, and you don’t either.
The reason that neither of us know is because it’s literally unknowable.
When no one has done the studies to answer a question, any hypothesis about the answer is just that: a guess.
Someday, I hope that we’ll look back on the COVID culture wars and remember how we used the societal conflict about masking as the impetus to create a national clinical trial network for the rapid implementation of cluster randomized trials to quantify the impact of non-pharmacologic interventions during a pandemic.
Unfortunately, I fear that we’ll just lapse into what we’ve been doing: shouting at each other as we cherry pick subpar data to make the case that our preferred policy is the right one.
The problem with our current situation is that you can’t credibly answer a simple question like “do mask mandates work in the US to prevent COVID transmission?” with any degree of certainty, because we don’t have the data to make a judgment.
That hasn’t stopped people from having really strong opinions, and it hasn’t stopped prestigious journals from publishing articles on the topic, articles that I would argue do essentially nothing to further our understanding of a really important question.
The New England Journal of Medicine just published a new article looking at the impact of lifting mask mandates in schools
You can read the article here: Lifting Universal Masking in Schools - Covid-19 Incidence among Students and Staff.
The authors looked at school districts throughout the greater Boston area during the 2021-2022 school year, covering 72 school districts and a total of 294,084 students and 46,530 staff.
It’s a big sample size over a significant time period.
Mask mandates were lifted in Massachusetts in February 2022, but different schools rescinded their own mandates at different times, allowing for researchers to look at differences in COVID cases among the different districts and how they changed around the time before and after mask mandates were lifted.
70 of the districts dropped the mandate at varying times and 2 districts kept the mandate in place throughout the entirety of the school year.
The authors in this article used a technique called “difference in difference analysis,” which is a statistical technique used to infer causality when randomization is not possible.
Keep that sentence in mind, because we’re going to come back to it later.
They found an association between lifting mask mandates and rising COVID cases:
Their estimate is that a third of COVID cases post mask mandate were due to the fact that the mandate was lifted:
“The lifting of masking requirements was associated with an additional 44.9 Covid-19 cases per 1000 students and staff (95% CI, 32.6 to 57.1) during the 15 weeks after the statewide masking policy was rescinded (Table 1). This estimate corresponded to an additional 11,901 Covid-19 cases (95% CI, 8651 to 15,151), which accounted for 33.4% of the cases (95% CI, 24.3 to 42.5) in school districts that lifted masking requirements and for 29.4% of the cases (95% CI, 21.4 to 37.5) in all school districts during that period.”
If you want to dive deep into the strengths of this article, you can read the editorial in the New England Journal or this Twitter thread.
But you can’t just blindly accept the authors conclusions here, you need to remember that their methodology is fatally flawed
You can spend a lot of time going through the flaws of this trial, and I would highly recommend this article discussing them.
The weaknesses here are numerous - lack of a dose response, the fact that some of the districts had already lifted the mask mandate prior to the state lifting - but I’m not going to spend much time going through them.
That’s because this trial has the same fatal flaw that all of the US mask trials share: it was not designed in a way that makes it capable of testing cause and effect.
The decision of a district to remove the mask mandate doesn’t occur in a vacuum. No public policy choice does.
There are always going to be important differences between areas that keep mandates in place compared to those who remove them. These differences are called confounders, and they are the reason why this trial doesn’t add anything important to inform our discussions or policymaking.
Much of the social media commentary has focused on why the known confounders either strengthen or weaken the authors conclusions - socioeconomic differences, vaccination differences, testing differences, community spread differences.
But the more important point is not about the known confounders - it’s about the unknown confounders.
You don’t know what you don’t know, and that’s the reason that you need to randomize rather than just observe.
Our biggest ongoing COVID failure - the greatest medical invention in the 20th century is the randomized controlled trial, and no one who sets policy seems to care
When I describe the randomized trial as the greatest medical invention of the 20th century, I’m not exaggerating.
I’ll never understand why the people who have been in real positions of public health power during the entirety of the pandemic - that’s a lot of bureaucrats across two different administrations - don’t want to get the answers to these important questions before thrusting their policy preferences on us.
Our failure to implement randomized trials during this pandemic is why questions that should be straightforward don’t have any answers 3 years into the pandemic.
No one knows the answer to things like:
Do mask mandates work?
Do cloth masks work at all? (They don’t seem to do anything in Bangladesh)
Do the new boosters prevent COVID transmission and severe illness or do they just briefly raise antibody levels?
Does better ventilation reduce virus transmission?
And even if the answer to every single one of these questions is “yes,” we have no idea about the effect size, because we haven’t done the research.
You can’t make a cost-benefit analysis when you don’t have a sense of the size of the impact. And as a consequence, most of us are flying blind when it comes to making these decisions that impact our health and the health of those around us.
You might feel differently about how important it is to wear a mask or get a booster if there was data to demonstrate the degree of effectiveness.
I am fully convinced that our unwillingness or inability to generate the data that we need has directly led to increased conflict, polarization, and scientific mistrust.
And so a critique of the absence of data should not be interpreted as an endorsement of any specific public policy position - as I said above, I have no idea whether mask mandates work and you don’t know either.
Bu I am pretty tired of people suggesting that they know the answers to questions that they obviously can’t know the answers to.