PHOTO-ILLUSTRATION: SAM WHITNEY; GETTY IMAGES
THE MAKERS OF a third coronavirus vaccine announced positive results in clinical trials on Monday, setting off yet another round of excited news reports. This one, produced by a partnership between a University of Oxford research institute, its spinout company Vaccitech, and the pharmaceutical company AstraZeneca, does not need to be stored at freezing temperatures and would be cheaper and easier to produce than the high-efficacy vaccines produced by BioNTech-Pfizer and Moderna. Indeed, according to an initial write-up in The New York Times, Oxford-AstraZeneca’s is “expected to be relied upon heavily across the globe, to help curb a pandemic that has killed more than 1.3 million people.”
Sounds like great news, right? Monday’s press release from AstraZeneca presents “convincing evidence that [the vaccine] works,” said Science. But not everyone has been convinced. The price of AstraZeneca’s shares actually dropped on the news, and an analysis from an investment bank concluded, “We believe that this product will never be licensed in the US.” Over at STAT News, Anthony Fauci cautioned that we’ll need to see more data before coming to a conclusion. The skeptics have strong reasons to be concerned: This week’s “promising” results are nothing like the others that we’ve been hearing about in November—and the claims that have been drawn from them are based on very shaky science.
The problems start with the fact that Monday’s announcement did not present results from a single, large-scale, Phase 3 clinical trial, as was the case for earlier bulletins about the BNT-Pfizer and Moderna vaccines. Instead, Oxford-AstraZeneca’s data came out of two separate studies: one in the UK that began in May, and another in Brazil, which got started at the end of June. These two studies were substantially different from one another: They didn’t have standardized dosing schemes across the trials, for one thing, nor did they provide the same “control” injections to volunteers who were not getting the experimental Covid vaccine. The fact that they may have had to combine data from two trials in order to get a strong enough result raises the first red flag.
Consider that leading vaccine makers—including AstraZeneca—issued a scientific-rigor-and-integrity pledge back in September, in which they promised to submit their products for approval or emergency use authorization only “after demonstrating safety and efficacy through a Phase 3 clinical study that is designed and conducted to meet requirements of expert regulatory authorities such as FDA.” Note the wording here: These companies did not suggest that they might claim to have demonstrated efficacy through multiple, distinct clinical studies, combined together to get enough data. They said they would use a Phase 3 study—as in, one big one. Yet AstraZeneca has already applied on the basis of this data for approval in Canada, and has plans to do the same in Britain, Europe and Brazil. The company also says it will use the data to apply for emergency use authorization in the US.
The Food and Drug Administration’s guidance for Covid-19 vaccines does allow for emergency use authorization based on interim analyses, but the same document says this must be supported by a minimum level of vaccine efficacy “for a placebo-controlled efficacy trial.” Again: it refers to a trial. That is exactly what BNT-Pfizer and Moderna did. Both released the FDA-approved blueprints for their trials—called trial protocols—weeks ahead of time, with details of the calculations and statistical rules that they’d use to determine when to perform an interim analysis and how much certainty could be attached to those results. When BNT-Pfizer’s discussions with the FDA led to changes in this plan, BNT-Pfizer explained why, and released an updated protocol. That’s scientific rigor, and it matters a lot. When a vaccine-maker specifies the rules of the game before the results start coming in, we can check their work and be confident in what they tell us at the end. We can make sure they haven’t cherry-picked the data.
How did Oxford-AstraZeneca end up with this patched-together analysis instead of data from a single, large trial?
The Oxford-AstraZeneca story is very different, though. Presumably, neither of the two trials from which they combined data could have provided a clear answer on the vaccine’s efficacy on its own. To make things worse, Oxford-AstraZeneca reported only the results for certain subgroups of people within each one. (For perspective on this: The two subgroups chosen leave out perhaps half the people in the Brazilian trial.) Meanwhile, one of their key claims is that giving half a dose of the vaccine on the first injection, followed by a standard dose on the second one, led to better outcomes—but neither of these trials had been designed to test this hypothesis. In fact, it’s since emerged that the half-dose/full-dose option started out as a mistake, and one that was only caught when some people in the study didn’t have the usual high rate of adverse effects.
There were other dosing issues, too, that haven’t been explained even though dosing is the centerpiece of the press release. There are many different regimens in these trials—the UK study has more than two dozen arms, meaning the volunteers were divided into that many groups according to age and how much of the vaccine would be administered and when. The doses are measured by the number of altered viral particles they contain, and the developers decided that the standard dose would be 5 x 1010 viral particles. But for many of those arms in the UK trial—as well as everyone who got the vaccine in the Brazilian trial—publicly available trial information shows that the standard dose could be between 3.5 and 6.5 × 1010 viral particles. The lower end of that range isn’t far off from a half-dose.
How did Oxford-AstraZeneca end up with this patched-together analysis instead of data from a single, large trial? After all, this vaccine went into Phase 3 testing before either BNT-PFizer’s or Moderna’s did. But in the UK, where that testing started, the Covid-19 outbreak happened to be receding. That meant results would be coming in very slowly.
A month later, a second Phase 3 trial for the vaccine started in Brazil. That one was for healthcare workers, for whom the risk of being exposed to Covid was far higher than it was for the people in the UK trial. But the two trials had other substantive differences. In the UK, for example, the volunteers who did not get the experimental Covid vaccine were injected with meningococcal vaccine; in Brazil, those in the comparison group were given a saline injection as a placebo.
Meanwhile, BNT-Pfizer and Moderna began Phase 3 trials for their coronavirus vaccines on the same day in July: Both planned to include 30,000 volunteers at the time, and both trial plans were approved by the FDA. Oxford-AstraZeneca then announced they, too, would run a 30,000-person trial in the US.
But that research on the Oxford-AstraZeneca vaccine quickly fell behind the others’. The US trial was approved by the FDA, but it didn’t start recruiting people until the end of August; and just a week later, it was put on hold so the FDA could investigate a serious adverse event in the UK trial. It wasn’t clear what caused the volunteer to get sick, but the FDA did not give the all-clear for Oxford-AstraZeneca’s US trial to resume until Oct. 23. By then the protocol for the trial had been publicly released. It says the plan is to inject the vaccine in two standard doses, a month apart; and two people will be vaccinated for every one who gets a placebo saline injection.
So here we are at the end of November. BNT-Pfizer and Moderna have offered up a masterclass in how to do major vaccine trials quickly in a pandemic, while Oxford-AstraZeneca has, for the moment, only an assortment of smaller ones ready to look at.
But wait, more red flags! Last week, Oxford-AstraZeneca published some results from earlier in the development of the UK trial. That paper included a trial protocol for the UK study, attached as an appendix. Deep in that document, and apparently overlooked by reporters and commentators, was an eyebrow-raising suggestion: Under a section marked “Interim and primary analyses of the primary outcome,” the trialists outline a plan to combine and analyze data from four clinical trials (only half of which are Phase 3), carried out in different ways on three different continents. The plan, they wrote, was to pull out results only for the people across these four trials who had gotten "two standard-dose vaccines," and then pool those together for what's called a meta-analysis.
The appendix doesn’t say when this became the plan. We don’t even know if the Oxford-AstraZeneca team followed it. In fact, it’s impossible to know, at this point, just how many analyses these researchers have run, and on which data. That’s a scientific red flag with flashing lights. (Again it’s useful to compare this work to the BNT-Pfizer and Moderna trials, where the analyses were clearly spelled out ahead of time for everyone to see.) All we know for sure is that on Monday, Oxford-AstraZeneca announced results of a different interim analysis that included only volunteers from the two trials in the UK and Brazil. As we’ve seen, this analysis was not limited to the people who got two standard doses of the vaccine.
Everything You Need to Know About the Coronavirus
Here's all the WIRED coverage in one place, from how to keep your children entertained to how this outbreak is affecting the economy.
There are other problems, too. In the press release, Oxford-AstraZeneca reports that two of the dosing regimens “demonstrated efficacy.” Presumably, none of the others did, but they didn’t give specifics. Of the only two regimens they reported, one (the mistaken first half-dose, followed by a full dose at least a month later) came in at 90 percent, and the other (two standard doses at least a month apart) achieved only 62 percent efficacy. You’ll see reports that the vaccine had 70 percent efficacy, on average; but that’s un-knowable, because we only have numbers on these two regimens, as opposed to everyone in the trials—and how they arrived at those percentages isn’t explained. As far as we know, some of this analysis could hinge on data from just a few sick people. That means the findings could be a coincidence, or they could be biased by other factors. For example, it has since been revealed that the people who received an initial half-dose—and for whom the vaccine was said to have 90-percent efficacy—included no one over the age of 55. That was not the case for the standard-dosing group, however, where the reported efficacy was 62 percent. This demographic difference could be more important than the change to the size of the first dose.
That’s not the end of the problems. Overall, the Oxford-AstraZeneca trials appear to include relatively few participants over the age of 55, even though this group is especially vulnerable to Covid-19. (People over 55 were not originally eligible to join the Brazilian trial at all.) Compare that to BNT-Pfizer’s trial, where 41 percent of the volunteers were over 55. The Oxford-AstraZeneca vaccine also seems to produce relatively high rates of adverse events. If you want to dig further into this vaccine’s story and issues, I’ve laid out a more detailed rundown of the Oxford-AstraZeneca trials and sources here.
Now it’s over to drug regulatory agencies around the world. They have to make a decision about this vaccine that was once ahead of the pack, but for which there are still no reliable and rigorous results from a single, large phase 3 trial. If anything short of that standard is accepted for this vaccine, it will be easy to stoke already widespread fears about corners being cut. A loss of trust would affect more than this one vaccine.
Hilda Bastian (@hildabast) has a PhD in health science, and expertise in assessing the validity of evidence for health claims.
SEE
No comments:
Post a Comment