The quest to arrive at the pandemic's true death toll.

With the onslaught of COVID-19 having definitely eased in recent months, now is a good time to take stock of the pandemic, and one of the things that will be looked at most intently is its true death toll. But this is not without its complications.

On March 30, Devex reported that the World Health Organisation was set to release its estimates of excess deaths caused by Covid-19 in early April. The estimates were produced as a collaboration between the United Nations Department of Economic and Social Affairs (Ecosoc or UN DESA), the WHO, and a WHO technical advisory group specifically set up for the purpose. But this has since been held up, and the New York Times reported earlier this month that it was mainly due to objections from India, where the WHO has estimated a death toll close to 8 times higher than the official figure (more than 4 million, against the Modi government's 520,000).

Even globally, the death toll is estimated to have been close to three times the collated official figure (that most recently crossed 6 million), in the WHO's report. The WHO has not released it, but the NYT reports it to be 15 million. This doesn't compare too unfavourably with other independent studies that attempted to do the same thing. Probably the most influential of these studies, published in the Lancet, estimates there were 18.2 million excess deaths globally between 1 January 2020 and 31 December 2021 whereas the official death toll from Covid-19 during that period was 5.9 million.

Counting the dead

The researchers behind the study published in The Lancet, a team led by Dr Haidong Wang at the Institute for Health Metrics and Evaluation (IHME) in Seattle, Washington, said the mortality impact from the Covid-19 pandemic has been "more devastating" than the situation documented by official statistics, which provide only a "partial picture" of the true burden of mortality. Evidence from initial studies suggest a significant proportion of excess deaths are a direct result of Covid-19, the authors said, while conceding that more research is needed in this regard.

To estimate COVID-19 deaths, the IHME study uses the measure called excess mortality, which is held to be a convenient tool to overcome variations in the ways that countries identify and record deaths, in this case due to Covid-19. Researchers estimate excess deaths by comparing the total deaths reported in a region or country, from all causes, with how many deaths would be expected given trends in recent years.

All cause mortality reports were collected for 74 countries and territories and 266 states or provinces through searches of government websites, the World Mortality Database, Human Mortality Database, and the European Statistical Office. The data was corrected for lags in reporting and for under-registration of deaths, with weeks with heat waves excluded. An ensemble of six models was used to predict the expected mortality rate in the absence of Covid-19. The model was also used to predict excess mortality rate for locations where all-cause mortality data were not available.

Globally, the excess death rate in the IHME paper is estimated to be 120 deaths per 100 000 of population with 21 countries estimated to have rates over 300 deaths per 100 000. The highest estimated excess death rates were in Andean Latin America (512 per 100 000 population), Eastern Europe (345 deaths per 100 000), Central Europe (316 deaths per 100 000), Southern sub-Saharan Africa (309 deaths per 100 000), and Central Latin America (274 deaths per 100 000). But there were also high rates outside these regions including in Lebanon, Armenia, Tunisia, Libya, several regions in Italy, and several southern US states.

At country level, excess mortality rates were highest in Bolivia (734.9 per 100 000), Bulgaria (647.3), and Eswatini (634.9). In Russia there were an estimated 1.1 million excess deaths; a rate of 374.6 deaths per 100 000 population. The US also had an estimated 1.1 million excess deaths with a rate of 179.3 per 100 000 population. An estimated 792 000 excess deaths occurred in Brazil; a rate of 186.9 per 100 000. Because of its large population, India with its estimated 4.1 million excess deaths, accounted for an estimated 22% of the global total deaths. It meant whereas the USA has the highest death toll in the world from Covid-19, going by the official figures, the excess mortality method catapulted India to the top, and by some distance.

In Europe, the UK had an estimated excess mortality rate of 126.8 per 100 000, which was lower than Spain (186.7), Italy (227), and Belgium (146.6) and closer to that of France (124) and Germany (120.5). Some countries were estimated to have had fewer deaths than expected, including Iceland (48 fewer deaths per 100 000), Australia (38 fewer deaths per 100 000), and Singapore (16 fewer deaths per 100 000).

In Bangladesh, the true death toll of Covid-19 was almost 15 times higher than the officially reported figure, if the IHME's study is to be believed. It is important to note though, that Bangladesh was one of the countries for which the team was forced to rely largely on modelling, for lack of reliable figures covering the entire period. The cumulative excess mortality number in Bangladesh was 413,000 by the end of 2021, according to the study while the official fatality figure from Covid-19 was 28,100.

The reliance on modelling meant a lower confidence level too, when it comes to the Bangladesh number. This is reflected in the very wide range around their central estimate, stretching from 347,000 to 504,000.

In terms of absolute figures, the highest number of excess deaths were estimated to be in India with 4.07 million, followed by the USA with 1.13 million, Russia with 1.07 million, Mexico with 798,000, Brazil with 792,000, Indonesia with 736,000 and Pakistan with 664,000.

The IHME's central estimate is similar to that of The Economist, which has maintained a running count of excess deaths on its website that arrived at 18 million by the end of 2021. But the error bars on the IHME's analysis are notably narrower: The Economist has a 95% uncertainty, or confidence interval for an estimate of 12.6 million to 21.0 million; the IHME's is much narrower, just 17.1 million to 19.6 million. Plus, of course, it is peer reviewed.

It is important to note that under the excess mortality method, you're not in fact claiming that all the excess deaths would have been directly attributable to a Covid-19 infection. As the authors state, the difference between excess mortality and reported Covid-19 deaths might be the result of underdiagnosis because of insufficient testing, or higher than expected mortality from other diseases because of behavioural change, or even reduced access to healthcare or other essential services. Remember when at the peak of the pandemic, it was becoming difficult to get doctors appointments to have other ailments checked? Lockdowns had an effect on hospital visits too, especially for people without cars.

The study also highlights that high burden countries are distributed across all regions of the world, reinforcing the evidence for the truly global nature of the pandemic. Finding ways to strengthen death reporting systems and mitigating political barriers to accurate reporting will be important for tracking this and future pandemics, the authors said.

How to take it

Many governments can be expected to resent the new information being thrown up by excess mortality studies, seeing it as either blaming them for incompetence or for obfuscation of data. And they are not the only ones. Experts in the field have their own issues with each individual study, if not their view of EM as "a more comprehensive and robust indicator" of the pandemic's true death toll.

The IHME model for example, contains some "bizarre features", according to Jonathan Wakefield, a statistician at the University of Washington in Seattle who leads the WHO technical advisory group mentioned earlier. The IHME's approach leads him to doubt the validity of its uncertainty intervals and other statistical features of the modelling, reported Nature magazine.

Ariel Karlinsky, an economist at the Hebrew University of Jerusalem who maintains the World Mortality Database and has worked on EM estimates, says the new study's central estimate of 18 million is reasonable, but that some of the IHME's numbers for excess deaths in individual countries are significantly out-of-step with other estimates.

"They still have their ludicrous estimate for Japan at over 100,000 excess deaths, which is over six times the reported deaths. I really don't know how they are getting that," he says.

Karlinsky reviewed an earlier version of the IHME's estimate of excess deaths, from May 2021, on his own website. There he wrote: "The issue with the IHME report is that it uses extremely partial data when much more encompassing (such as World Mortality) exists, the issue is that the country-level estimates they showed publicly are incredibly different than known ones (mostly higher) and that they purport to accurately estimate excess deaths where data simply does not exist - this undermines a tremendous effort currently underway to improve and collect vital data in many countries."

A more stinging rebuke, and one that resonates closely with our own take on the issue, comes from Stéphane Helleringer, professor of demographics and epidemics at New York University's Abu Dhabi campus, who wrote on Twitter:

"I do worry a lot though about (the) false impression of knowledge and confidence that is conveyed by their estimates; especially the detailed global maps like the ones they just produced for COVID death toll and MANY other health indicators for which few or no data are available. The risk is that IHME figures, with their apparent precision, will distract some funders and governments from (the) goal of universal death registration in low to middle Incomes countries. From their standpoint, if IHME readily estimates mortality, why invest in complex systems to register each death?"

What Dr Helleringer is saying, essentially, is that these studies are all well and good, but just as we shouldn't assume the official death toll to be the real figure, we should regard each of these individual studies with a fair amount of salt as well - if possible, the right amount of salt for each. Going by the excess mortality method, we should not expect to arrive at any exact figure. Rather, we should satisfy ourselves with more of a range that is derived from collating the different death tolls they produce, attaching greater credence (i.e. less salt) to the ones that are able to utilise the most reliable and fullest sets of input data.

Dr Wang has stated how studies from several countries, including Sweden and the Netherlands, suggest Covid-19 was the direct cause of most excess deaths, but at the same time, he did acknowledge that they currently don't have enough evidence for most locations. That is why further research is indeed still needed to reveal how many deaths were caused directly by Covid-19, and how many occurred as an indirect result of the pandemic.

The fact that The Economist's machine-learning model and the IHME arrived at roughly the same figure may be a positive reinforcement in favour of the 18 million death toll, but the WHO's figure is lower - remember, 15 million as reported in the New York Times - and that is why it is important that its study is released. That 3 million difference may make a difference at the country level for a number of nations.

Among other things, the WHO would crucially have access to more information from more territories around the world. The IHME and the Economist estimates represent comprehensive and rigorous attempts to understand how mortality has changed during the pandemic at the global level. But they do come with a great deal of uncertainty given the large amount of data that is missing and the known shortcomings even for data that is available.

This all makes it even more imperative that the WHO's study is allowed to see the light of day. Sensitive governments should not be allowed to hold it back.

Pushing back

Apart from the WHO's report, the Indian government has also dismissed the IHME's study and one that appeared in the journal Science earlier that concentrated on India as 'speculative'. In the WHO's report, more than a third of the additional nine million deaths are estimated to have occurred in India, according to NYT, which was unable to learn the estimates for other countries.

In a response to the NYT article (provocatively titled "India Is Stalling the W.H.O.'s Efforts to Make Global Covid Death Toll Public"), the Indian government released a statement through its Press Information Bureau that stated, among other things: "India has been in regular and in-depth technical exchange with WHO on the issue. The analysis, which uses mortality figures directly obtained from Tier-I set of countries, uses a mathematical modelling process for Tier II countries (which includes India). India's basic objection has not been with the result (whatever they might have been) but rather the methodology adopted for the same."

The parentheses were part of the statement. DC has corrected some of the obvious grammatical mistakes in the statement for clarity. India has not submitted its total mortality data to the WHO for the past two years, but the researchers used numbers gathered from at least 12 states, including Andhra Pradesh, Chhattisgarh and Karnataka, to arrive at their figure for the country.

To produce mortality estimates for countries with partial or no death data (the Tier-II countries referred to in India's statement), the experts in the advisory group used statistical models and made predictions based on country-specific information such as containment measures, historical rates of disease, temperature and demographics to assemble national figures and, from there, regional and global estimates.

Other countries too will be found to have made significant undercounts, especially in South Asia and Sub-Saharan Africa, two regions plagued by poverty that have very informal systems for dealing with mortality and health records. The Russian national statistics agency found excess mortality of more than one million in its jurisdiction - a figure that is reportedly close to the one in the WHO draft, as well as the one estimated by IHME. Russia has objected to the number, but it has made no effort to stall the release of the data. For most people, it is difficult to understand how one government is able to exercise the sort of influence necessary to hold back a report of global significance.

That probably reflects the severely weakened stance of the WHO as a global authority over the course of the pandemic, for a variety of reasons - both deserved, and undeserved. For one thing, the world has never before seen a pandemic that has been so politicised - from the naming of the disease to all the vaccine nationalism. Be that as it may, in recent weeks, WHO spokespersons have come out with bullish statements that they do still intend to publish their study, and in fact will do so no matter what. They have pledged to include in their report any objections that governments may raise. It has also been reported that a few members of the technical advisory group have warned that if the WHO does not release the figures, they would be forced to go ahead and do it themselves.

Either way, it is bound to be upsetting.