Masks in schools: how convincing is the government's evidence?

8 January 2022

6:00 PM

8 January 2022

6:00 PM

Why has the government changed its mind and asked children to wear masks in school? When Plan B was announced last month, there was no requirement. But that has changed.

Nadhim Zahawi, the Education Secretary, was asked why on Monday. He replied thatwe conducted a small observational study with 123 schools who had followed mask-wearing in classrooms before and saw that they made a difference’. He suggested it was quite a significant study. ‘If you just think it through, with a respiratory disease that is aerosol-transmitted, if you are asymptomatic but wearing a mask, you’re much, much less likely to infect other people.’

The government has now published an ‘Evidence Summary’ for the use of face coverings in education settings. The study compared schools where pupils worse masks with those that did not, and found a difference in Covid absences: of just 0.6 percentage points. Even the official report accepts that this is not a statistically significant difference.

As practitioners of evidence-based medicine, we are used to assessing such research — and asking not just about the conclusions but about the quality of the study. If you’re making a huge policy change — like ordering secondary school kids to mask up — then you really ought to have strong evidence. Does it exist this time?

The problems face masks pose for children

The Evidence Summary from the Department for Education (DfE) accepts the harm inflicted by face masks in class. It cites research showing that:

  • ‘80 per cent of pupils reported that wearing a face covering made it difficult to communicate, and more than half (55 per cent) felt wearing one made learning more difficult’
  • ‘Face coverings may have physical side effects and impair face identification, verbal and non-verbal communication between teacher and learner.’
  • ‘Almost all secondary leaders and teachers (94 per cent) thought that wearing face coverings has made communication between teachers and students more difficult, with 59 per cent saying it has made it a lot more difficult.’

All pretty bleak. The question is whether the positives outweigh the negatives. You’d expect any study of this kind to be high-quality and able to demonstrate a sizable net effect on children’s wellbeing. But the evidence published falls very short of this.

Big claims, weak evidence

The DfE report offers two forms of evidence in support of masks in class: the UK Health Security Agency (UKHSA) review, updated last November (which cites a variety of evidence), and the newly-released study of 123 schools that Nadhim Zahawi referred to.

Let’s take these in turn. The DfE talks about the UKHSA review as if it were a slam-dunk which ‘strengthens the evidence for the effectiveness of face coverings in reducing the spread of COVID-19 in the community, through source control, wearer protection, and universal masking.’

But on further inspection, the claims have a rather weak evidence base. Take, for example, the statement saying: ‘The virus can remain airborne for many hours in closed indoor spaces such as classrooms’. For ‘many hours’ in classroom air? How do they know? We are pointed to the opinion of a single author and a reference back to one of the government subcommittee’s works. Perhaps this was done by finding intact particles, or by some of the virus fragments identified by PCR. If the latter, it has little significance without other information. If the former, had it sufficient concentration to cause infection, and was the chain of transmission identified?

The jury is still out

The UKHSA ‘rapid review’ cites four observational studies set in schools and a summer camp and reports all four ‘were low quality’ — one was so bad it was ‘unable to provide evidence due to its design’. It also reports ‘mixed results for the effectiveness of face coverings, with some studies suggesting face coverings were associated with reduced COVID-19 transmission and others suggesting no statistically significant effect’. The DfE also cited two recent US studies of low quality (see here and here).

It concludes ‘in all studies, there is a high risk that factors other than face-covering use affected the results (in either direction) as people who are more likely to wear face coverings are likely different in many ways than people that are not, for example, they may engage more or less with other behaviours that are likely to reduce transmission of COVID-19, such as hand washing and social distancing.’ So after all these studies: has the efficacy of masks been scientifically proven? No: the jury is still out.

The 123-school study does not include anything on Omicron

The DfE document states the effectiveness of masks stems mostly from reducing the emission of virus-carrying particles, and that preliminary Sage advice suggests Omicron might show more airborne transmission.

However, the most interesting and perhaps nonsensical part of the document comes at the end under ‘Caveats’. Here we are told that ‘this (document) only uses data from the first half term of 2021/22 so does not include any Omicron data.’ Therefore the vaunted survey of 123 schools does not apply to the Omicron variant as it was carried out before its appearance and circulation.

The school masks difference: just 0.6 per cent

The DfE study of 123 schools says that in schools with masks, Covid absences fell to 3 per cent, from 5.3 per cent — a 2.3 per cent drop. In schools that did not have masks — 1,192 of them — there was a 1.7 per cent drop: a small difference of 0.6 percentage points.

As the DfE accepts, the 123-school analysis is ‘non-peer reviewed’ and ‘shows a non-statistical and unknown clinical significant reduction in infection in a short follow-up period’. So the results are within the margin of error, in a study that has not been properly assessed in the standard academic way.

Is this really a good basis on which to make a decision that will affect so many schoolchildren?

Zahawi says the study ‘saw that [masks] made a difference’. In fact, the DfE states that this analysis is preliminary and experimental. It needs a robust external peer review on a longer timescale (we’d like to offer ourselves up for that). But the DfE does not seek to conceal the flaws in its study. In fact, it lists several:

  • The analysis does not differentiate between the classroom and other areas (including outdoors).
  • It doesn’t say what the Covid rates were in the area these schools were in.
  • We don’t have any other data — like regional stats and differential uses of face coverings — that might suggest an increase in school absences.
  • We don’t know the characteristics of these schools, like age and class sizes, which might account for higher absences.

In search of better evidence

On p12 of the document, we are told that ‘the most reliable way to measure the impact of the use of face coverings would be to do randomised control trials (RCTs).’ This is correct. The adverse effects associated with mask-wearing in schools further strengthens the need for a proper test — an RCT — to assess whether the benefits of masks outweigh the harms.

But the DfE says such trials are ‘not possible’ due to ‘the speed and ever-changing nature of the pandemic’. Really? Given how many people’s lives are being affected by these government rules, commissioning a proper study to see if they work ought to be a priority — because it is certainly possible.

Over the last two decades, 15 studies have been carried out testing masks against transmission of flu-like illnesses and two against Covid. One was done during the 2009 ‘swine flu’ pandemic. Even during the Covid pandemic, an RCT has been done in Bangladesh. You’d think that the UK has the wherewithal to pull of a study on these shores. In the face of such uncertainty — and the known harms of masks in classrooms — any reluctance to do them is hard to justify. Surely if we are to ask millions of children to comply, we can fund a proper study to see if this policy works.

Without such evidence, ministers are in the dark — as the DfE acknowledges. Due to a lack of randomised control trials, it says ‘it should be noted that we cannot assess causality, i.e. whether the use of face coverings resulted in X% absence rate.’ In layman’s terms: we can’t really tell how well they work, and over what period of time. But you should still mask up.

The DfE document has all the hallmarks of retrofitting evidence to justify the policy. The strong language from Zahawi is simply not matched by the strength of the evidence. Far too much of the figures produced to back masks in schools are based on opinions, low-quality studies (with mixed results), and a single survey in 123 British schools over three weeks in October 2021 that is not peer-reviewed, and severely limited. And, on top of that, it simply doesn’t report a statistically significant benefit for masks in schools.

So the Education Secretary has been poorly briefed. There are clear concerns amongst students and teachers that wearing masks impacts learning. Yet the DfE document does not meet the requirements for the high-quality evidence we would need before justifying the policy. It’s time for the Department for Education to go back to school.

Got something to add? Join the discussion and comment below.

Show comments