Yusuff Adebayo Adebisi knows that a vaccine that offers 70 per cent protection against Coovid-19 could be a valuable tool against the coronavirus pandemic in Nigeria – especially if that vaccine is cheap and doesn’t have to be stored at extremely cold temperatures. But what if another vaccine — one that is more expensive to buy and to store – was 95 per cent effective?

“Should we send the less-effective vaccine to Africa? Or should we look for a way to strengthen the cold storage?” asks Adebisi, director for research at African Young Leaders for Global Health, a non-profit organisation based in Abuja.

These are the kinds of question facing researchers and government leaders worldwide, as they take stock of the emerging selection of coronavirus vaccines and try to decide which will be most useful in putting an end to a pandemic that has already taken nearly 2.5 million lives.

It is a decision shaped by limited supplies and hampered by limited data, says Cristina Possas, a public-health researcher at the Oswaldo Cruz Foundation in Rio de Janeiro, Brazil. “It is not possible to compare these vaccines at this point,” she says.

In Bangladesh, health economist Shafiun Shimul at the University of Dhaka worries about the risks if governments delay vaccinations for months to build cold-chain infrastructure. “If you want to control infection, you have to rely on something that is contextually doable for you – it’s not only about effectiveness,” he says. “If they wait for perfection, I think it will be a long wait.”

Given the demand for speed amid limited supplies, any effort to rank the vaccines must take into account not only their reported effectiveness, but also supplies, costs, the logistics of deploying them, the durability of the protection they offer and their ability to fend off emerging viral variants.

Even so, many people might find it hard to look away from clinical-trial results that suggest an efficacy gap. So far, more than 200 million doses of coronavirus vaccines have been delivered and data have been rolling in from clinical trials in several countries.

The top-line results from those studies suggest a range of protection: from 95 per cent efficacy for a vaccine made by Pfizer of New York City and BioNTech of Mainz, Germany, to about 70 per cent suggested by initial results on a vaccine made by AstraZeneca of Cambridge and the University of Oxford, both in the United Kingdom.

It might be tempting, but it simply isn’t possible to directly compare the effectiveness of vaccines on the basis of those results alone, cautions David Kennedy, who studies the ecology and evolution of infectious diseases at Pennsylvania State University in University Park.

Each measure of efficacy comes with a degree of uncertainty and trials might have differing definitions of important criteria, such as what constitutes a ‘severe’ bout of Covid-19 compared to a ‘moderate’ one.

Added to this are the demographics of each trial: in the case of the Oxford-AstraZeneca vaccine, for example, the developers collected few data about the vaccine’s efficacy in people over 65. This led Germany to authorize the vaccine only for those under 65, even though the European Medicines Agency recommends it for all adults.

And the vaccines were studied at different times in various countries. Each trial can only offer a snapshot of protection against the viral variants that were dominant in that time or place, says Kennedy. “That number relates to a particular point in time,” he says. “How that translates into protection over one to two years is not the same.”

This point is particularly relevant as the world grapples with emerging coronavirus variants, some of which seem to evade aspects of the immune responses stimulated by vaccines. Researchers first spotted one such variant, called 501Y.V2 or B.1.351, in December in South Africa, where it now accounts for the majority of new coronavirus infections.

That variant has since been identified in countries around the world, and the extent to which it might reduce vaccine efficacy remains unclear. Laboratory studies and clinical-trial data suggest that most vaccines will still provide significant protection. But the AstraZeneca vaccine faltered badly: in an analysis1 of about 2,000 people in South Africa, it did not protect against mild or moderate Covid-19 due to the variant.

Faced with those results, the government of South Africa announced on 7 February that it would place the roll-out of the AstraZeneca vaccine on hold, despite the shot being significantly cheaper than Pfizer’s vaccine, and easier to store.

AstraZeneca and the University of Oxford have granted permission to generics makers such as the Serum Institute of India in Pune to crank out doses as quickly as possible and the vaccine had been considered the best hope for Africa, says Joia Mukherjee, chief medical officer of Partners in Health, a charity based in Boston, Massachusetts, that operates in 11 countries.

“But if it isn’t effective against the South African variant, we’re going to have to switch tactics,” she says. “To march forward and use it when we know that the variant is spreading through Africa, particularly southern Africa, is malpractice on a global scale.

However, some regions in Africa could still benefit, says infectious-disease specialist Loice Achieng at the University of Nairobi. The 501Y.V2 variant has not yet become dominant in Kenya, she says, and it’s still possible that the AstraZeneca vaccine could protect against severe COVID-19 caused by it. “I think it’s probably something that shouldn’t be written off,” she says.

There are hopes that more suitable vaccines will become available to fill some of the gaps. Johnson & Johnson in New Brunswick, New Jersey, for example, is developing a single-shot vaccine that would dramatically simplify vaccine roll-outs. But it completed clinical trials only in late January and it is not yet clear how quickly or smoothly the company will be able to start producing millions of doses, says Jerome Kim, director-general of the International Vaccine Institute in Seoul.

The world is still waiting for crucial data about the vaccines that are being rolled out now, says Kim. Medicines do not always perform as well in the real world as they do within the strict confines of a clinical trial. Early data from Israel’s massive vaccination campaign suggest that the Pfizer vaccine results are holding up, but it will take months to collect similar data about other vaccines.

Researchers are also starting to test a range of doses, schedules and combinations of vaccines. They still do not know how long vaccine-mediated immunity will last, or how well the various vaccines reduce coronavirus spread – all factors that could shape which is considered the ‘best’. “It’s not just a matter of getting them out as fast as possible,” says Mark Jit, a vaccine epidemiologist at the London School of Hygiene & Tropical Medicine.

“It’s making sure that as we get them out, we are putting in place the surveillance studies to see how well the vaccines are doing in different situations.”

Eventually, it might be possible to be more strategic about which vaccines to use in which settings, says Kim. But for now, the data simply aren’t there. “You’re watching these things change in real time,” he says. “In the next month, we could think something quite different.”

•  A Nature magazine report