In a daring yet transparent move, a new study from Cleveland Clinic — a non-profit academic medical center and one of America’s best hospitals — produced a disturbing figure showing that the more vaccine shots you get, the higher your risk of Covid (Figure 1):
As usual, people have exploited this graph as proof that vaccines weaken your immunity against Covid, which is indeed very convincing. But there’s more to this plot than meets the eye. And a few factors must be considered before drawing any conclusions from this plot.
What the study did and found
The study, “Effectiveness of the Coronavirus Disease 2019 (COVID-19) Bivalent Vaccine,” is recently released as a preprint. Although it’s not peer-reviewed yet, the authors Shrestha et al. are credible scientists from Cleveland Clinic, the U.S. with a good track record of academic publishing.
Shrestha et al. analyzed the effectiveness of the bivalent mRNA vaccine, which targets both Omicron and original Wuhan variants of SARS-CoV-2, against Covid. As mRNA vaccines were efficacious in prior clinical trials, bivalent mRNA vaccines were approved for use without demonstration of efficacy in clinical trials. Only safety data was required for approval.
They recruited 51,011 employers (median age of 42 years, 41% had prior Covid, and 87% had at least one Covid vaccine dose) at Cleveland Clinic. The employers were offered the bivalent mRNA vaccine on 12 September 2022, which is the study’s start date and ended on 12 December 2022. By the end of the study, 21% (n = 10,804) got the bivalent mRNA vaccine (89% from Pfizer and 11% from Moderna), and 5% (n = 2,452) got Covid.
After adjusting for age, gender, job category, number of Covid vaccine doses received, and pandemic phase when the last Covid episode occurred, the bivalent mRNA vaccine reduced the risk of Covid by 30%.
But further analysis showed that, in the authors’ words, “the more recent the last prior COVID-19 episode was, the lower the risk of COVID-19, and that the greater the number of vaccine doses previously received, the higher the risk of COVID-19.”
Specifically, for the former statement, those who got Covid 6–9 and 9–12 months previously had a 2-times and 3.5-times increased risk of Covid compared to those who got Covid within the last 3 months.
But Shrestha et al. did not elaborate much for the later statement on prior vaccine doses, besides the controversial figure they produced (Figure 1), showing that the more Covid vaccine doses you received, the higher the cumulative incidence of Covid. Because the error bars don’t overlap, the differences seen are likely statistically significant.
How to make sense of this study
First, this study is observational, which by default cannot establish cause-and-effect due to inherent biases, such as differences in behavior.
For example, the authors admitted that “Those who chose to receive the bivalent vaccine might have been more worried about infection and might have been more likely to get tested when they had symptoms, thereby disproportionately detecting more incident infections among those who received the bivalent vaccine.”
But at the same time, the authors acknowledged that “If individuals received the bivalent vaccine thinking it would reduce their risk of infection, they would have been less inclined to get tested for the same symptoms after getting the vaccine (bivalent vaccinated state) than before getting the vaccine (non-bivalent vaccinated state), providing greater opportunity to detect infection in the non-boosted than the boosted state, thereby having the effect of overestimating vaccine effectiveness.”
Therein lies an inherent limitation of observational studies — no randomization to nullify the behavioral factors — as well as genetic, environmental, and countless other factors — between groups. That’s why observational studies can’t prove causation, they can only support causation in light of what randomized clinical trials found.
Second, the number of infections may not be a good indicator of vaccine effectiveness, given that the vaccine’s main value lies in its ability to prevent severe Covid. But unfortunately, the study has too few cases of severe Covid to analyze if vaccines decreased the risk of severe Covid.
This might be a sign that the vaccine is working well in this study, but it could also be that Omicron caused less severe Covid or the employers were rather young and not immunocompromised. Another reason is that immunity from prior undetected asymptomatic infection may have masked vaccine effectiveness or decreased the incidence of severe Covid.
Third, association or correlation does not mean causation. For example, severe Covid is associated with corticosteroid use. But this is because corticosteroids are used to treat severe Covid, not because they cause severe Covid. Another example is that ice cream sales correlated with shark attacks, because people are more likely to have ice cream outside and swim in the ocean during hot weather.
“The association of increased risk of COVID-19 with higher numbers of prior vaccine doses in our study, was unexpected,” Shrestha et al. do admit. “A simplistic explanation might be that those who received more doses were more likely to be individuals at higher risk of COVID-19.”
Comments / 752