Covid-19: On the Waning Effectiveness of Vaccines

Data presented at recent Downing Street press conferences appears to understate the effectiveness of vaccination at preventing hospitalisation with Covid-19. It may also give a misleading impression of the extent of waning vaccine effectiveness. Even with the best data, though, it is difficult to accurately estimate how much vaccine effectiveness is waning. However, since boosters are very effective in increasing immunity, they might be thought of as necessary not only to protect against possibly waning immunity but also because two vaccine doses were never enough in the first place!

Here are a couple of interesting bar-charts:

I’ve tried to adjust the sizing to compensate for the fact that the charts are formatted slightly differently, not 100% successfully. Nevertheless, I’m sure you’ll agree that the orange bars are getting taller, relative to the blue bars. It looks like the efficacy of the vaccines is waning dramatically. Scary.

But there are some serious problems with this data:

1. Unknown number of unvaccinated

It turns out that PHE (now UKSA) have overestimated the size of the population, so – because they know how many people have been vaccinated – they have massively overestimated the size of the unvaccinated population, as explained by David Spiegelhalter and Anthony Masters:

“PHE uses National Immunisation Management System (Nims) estimates of 8.1 million adults aged 40 to 49 in England. These numbers depend on GP records and tend to overcount. The Office for National Statistics (ONS) also produces population estimates, based on the 2011 census, migration and death registrations and estimates 7.1 million adults in that age group, a million fewer than Nims.”

This is particularly problematic in older age groups in which vaccination rates are very high.

Older people are therefore much better protected by vaccination than Figure 1 and 2 imply. I’ll tell my mum she can go clubbing again!

2. The plot thickens – NIMS vs ONS data

Oliver Barnes and John Burn-Murdoch, writing in the FT on 11th October also explain the problem:

But they suggest the figures based on NIMS data should be accurate for over 80s, i.e. that Figures 1 and 2 are (approximately) correct for that age group. Yet comparing the age-groups in Figures `1 and 2 shows a consistent pattern of (apparently) waning immunity – there doesn’t seem to be a sudden deterioration in the over 80s, though the younger you are the better, of course.

Time for a sanity-check. The left-hand chart in Figure 3 shows infection rates in the unvaccinated over 80s are still less than for vaccinated over 80s, even using the supposedly “more likely to be accurate” NIMS data. It seems the NIMS data still overestimates the over-80s population, just not by as much as the ONS data does. Both the NIMS and ONS data is also suspect for the 70-79 age group.

3. So how effective are the vaccines really?

This is all very exasperating, as, with my parents in the at-risk demographic, I’d really like to know how effective the vaccines are. Is it even plausible that, pre-boosters, they give only the level of protection against hospital admission as suggested by Figure 2?

Well, in a word, no.

My gut feel for the numbers tells me that the NHS wouldn’t be just stressed at the moment, but broken, were the vaccines now as ineffective as implied by the PHE bar-charts. But let’s see if we can do a little better than that,

I realised eventually that it’s possible to ballpark vaccine effectiveness independently of the population statistics, by comparing infection survey data with hospital admissions:

Comparing Figures 4 and 5, we see that the level of infection in over 70s in England was roughly 0.9% in the week to 30th Jan 2021, during the Alpha wave, before even the first dose had kicked in for a significant proportion of the population, and roughly 0.8% in the week to 22nd October 2021. It may have peaked at around 1.3% at the turn of the year. Note that Figures 4 and 5 have different scales, because of the current off-the-scale level of infections among young teenagers.

Can we compare hospital admissions for older age groups between these periods? Yes, we can:

Well, that’s interesting, isn’t it? It looks like rates of hospital admission for Covid for the older age-groups, 55+, are at least 80% less now than at the peak of the Alpha wave and the reduction is quite consistent across those age-groups.

Those going into hospital on 17th Jan might have been infected around the turn of the year, when infection levels in the over-70s were around 1.3% (see Figure 4).

Now, here we’re comparing infection levels with hospital admissions, whereas Figures 1 and 2 compare hospital admissions in vaccinated and unvaccinated individuals. We know the vaccines prevent infection, so need to allow for that. If the protection against infection is about 50%, as often suggested, then the protection the vaccinated population has now, compared to that of the unvaccinated population in January, has to be doubled. Even if we pessimistically assume that 0.8% infected now is equivalent to 1.3% at the turn of the year, comparing hospital admissions at the start and end of the charts in Figure 6 gives a reasonable ballpark idea of the level of protection the vaccines are giving against hospitalisation for the over 70s at least.

And, of course, I haven’t allowed for the unvaccinated included in the recent hospital admission data.

It’s also worth noting that we now have the Delta variant which evades some of the protection given by the vaccines, compared to the Alpha variant.

4. The unvaccinated gain immunity through infection

But you’d still expect the proportions of vaccinated and unvaccinated hospital admissions to be constant over time, if vaccine efficacy weren’t waning. Do Figures 1 and 2 give an idea of how much vaccine efficacy is waning?

The trouble is, over time, both vaccinated (through “breakthrough” infections) and unvaccinated people become infected and recover, acquiring immunity (or fail to recover, of course). They’re very unlikely to become infected again, at least for a few months. They disappear from the pool of people vulnerable to the virus, either because the vaccine isn’t 100% effective, or because they haven’t had the vaccine.

But, because there are far fewer unvaccinated people than assumed in the bar-charts in Figures 1 and 2, that population acquires immunity through infection much faster than the vaccinated population does. For example, to guess at the approximate numbers, if the number of unvaccinated people is a third of the number assumed and 3% of them were infected between September and October, there’d be 9% fewer of the inflated number of unvaccinated to be infected by the time of the second snapshot in Figure 2, above.

Because of this overestimation of the unvaccinated population, you’d expect the proportion of hospital admissions of the unvaccinated to decline over time in the PHE/UKSA graphs, making it appear as if vaccine effectiveness is waning more than it actually is.

Curiously, a similar number of vaccinated individuals in the older age groups were infected in the periods covered by Figures 1 and 2 (which have different scales). This might be what you’d expect given the fairly constant case numbers in the country as a whole (Figure 7) and unchanged level of restrictions over that period.

5. The unvaccinated gain more immunity through infection

Then, as I explained in the post On Herd Immunity, an infection in the unvaccinated population turns one (in general) fully susceptible individual into one fully immune individual, but an infection in the vaccinated population may turn someone already 50% protected from infection into someone fully immune. The net effect is that the unvaccinated population acquires immunity through infection faster than does the vaccinated population. This could make it seem as if immunity through vaccination is waning more than it actually is.

6. Who hasn’t been vaccinated?

Furthermore, we need to ask ourselves why people in the older age groups haven’t been vaccinated. This could affect the chances of them becoming infected. For example, the unvaccinated group might include some so frail that vaccination is contra-indicated. You’d expect these people to be shielded, so less likely to become infected than the vaccinated, who might gradually feel safer following vaccination. In this case, vaccinated people might be progressively more likely to become infected than the unvaccinated, but not because the vaccine efficacy is waning.

The unvaccinated population may. though, also include Covid denialists. As well as refusing vaccination, these people might not take other precautions against infection. As they get infected, the numbers of susceptible individuals in this group would decline steadily but faster than the number of partially-susceptible vaccinated people become fully immune due to infection, again making it look as if vaccine effectiveness was declining.

Conclusion

Determining the extent of waning of Covid vaccine effectiveness is a non-trivial exercise even in the UK. If not corrected for, the statistical problems with the exercise tend to bias towards a overestimation of waning. Nevertheless, it may be that while waning is the reason being given for rolling out booster jabs, two doses were never in fact enough in the first place! I’ll certainly be having mine when eligible.