Why comparing coronavirus outbreaks in different countries can be misleading — and even dangerous

Why is the death rate so low in Germany? How come the UK has comparatively fewer cases than elsewhere? And what’s going on in Russia?

An online map of global coronavirus cases, compiled by Johns Hopkins University (JHU), is, according to the university, being viewed more than 1 billion times every day by people who want to see the figures for themselves. Questions once reserved for epidemiologists are now being discussed over dinner tables and around (virtual) water coolers across the world.

But what if the numbers aren’t painting the full picture?

For people who are comparing the data and attempting to divine how different countries’ epidemics stack up against each other, experts have a warning: nations have different reporting standards, different approaches to testing, and different approaches to tracing cases, all of which makes such comparisons dangerously misleading.

Sheila Bird, a renowned British biostatistician and professor at Cambridge University, says testing and reporting have been inconsistent even within individual countries, let alone internationally. Rules on who gets tested — and where and when — can evolve, and these changes and time lapses are not necessarily captured in the data published by different governments.

Bird is one of a number of high-profile scientists who have been calling on the UK government to run more coronavirus tests and provide more nuanced data. The UK government says the health system doesn’t currently have the capacity to test everyone who has Covid-19 symptoms. Only patients sick enough to require hospital treatment get tested, according to the government. The country had more than 9,962 cases and 467 deaths as of Thursday, according to JHU.

Meanwhile in Germany, anyone who shows flu-like symptoms and has in the past 14 days come into contact with a confirmed case, or traveled to a high-risk region, gets tested, according to official guidelines published by the health ministry. According to JHU, more than 41,519 people there have tested positive as of Thursday, with just 239 deaths.

And in South Korea, there is free and easy access to testing for anyone whom a doctor deems needs it, and authorities have been tracing the contacts of infected patients. As of Wednesday, the country had 9,241 cases and 131 deaths, according to JHU.

The relatively low number of tests done in the UK helps explain why the case numbers appear so much lower in the JHU map compared to other European countries. It doesn’t mean fewer people are sick, necessarily — just that fewer people are being tested.

“The danger is relying on the figures and assuming that they give us an accurate count of the total number of people who have been infected,” said Mike Tildesley, an associate professor at Warwick University.

The lack of testing could also make the situation look worse than it is. Based on the number of confirmed cases and reported deaths, the UK appears to have a relatively high Covid-19 mortality rate. But that’s not necessarily the case, because in the UK, only those who are very ill get tested. People with mild symptoms, who are very likely to recover, are being told to stay home without being tested — meaning they’re not being captured by the data.

Internationally, the comparisons rely on a piecemeal of different sources. The JHU map, which has become the go-to resource for many media organizations and experts, pulls in data from the World Health Organization, the US Centers for Disease Control and Prevention, the European Center for Disease Prevention and Control and China’s national health commission, in addition to local media reports, health departments, and the online community for health care professionals known as DXY.

Bird said comparing such numbers could be dangerous. “Lacking assured like-with-like comparators in other countries, we risk making heroic, and perhaps heroically wrong, inferences country-to-country,” she told CNN in an email.

Tildesley, who specializes in developing mathematical models to simulate the spread of diseases, said people need to be mindful of the bigger picture. “The fear is that people may look at their own risk, consider it’s low and then not take the measures that are being advised,” he said. His advice: Even though the likelihood of any one individual in the UK being infected is very low at the moment, everyone should behave as if they were.

Rather than focus too much on the actual numbers, scientists spend a lot of time looking at the epidemic curve — the shape of the graph that captures the total number of new cases in each country day by day.

“If different nations have different standards and conditions, they at least generate a consistent curve if those standards and conditions are stable across time,” said Lisa Gitelman, an NYU professor and the editor of a book called ‘”Raw Data’ Is an Oxymoron.”

So, if Italy keeps testing people at similar rate and its daily rate of increase in new cases starts going down — the curve starts flattening — that’s good news. But the testing rate needs to stay the same. If the UK suddenly starts testing many more people and sees a big jump in new cases, it doesn’t necessarily mean the epidemic is spreading faster. Similarly, if a country runs out of tests, it may suddenly report a misleading drop in new cases.

Accurate information is important, because to defeat the virus, people need to be willing to comply with strict restrictions. Scientists say the only way to defeat the virus is through social distancing, which requires citizens to drastically alter their way of life.

Martin Hibberd, a professor of emerging infectious disease at London School of Hygiene and Tropical Medicine, said the UK government’s figures “may make some people think there are not many cases.” He said that’s why the associated messaging is important. “Gaining compliance will then depend on convincing people that their actions impact the numbers going forward,” he said.

“When the containment restrictions are increased, we should expect to see an impact on the numbers — the forward predictions should change — and people can be convinced that their actions matter,” he added.

The reproduction number — showing the average number of people each infected person passes the virus on to — is a key metric when looking at the success of the restrictive measures. Early data indicates that social distancing measures really help to curb the disease from spreading, according to a study published in The Lancet on Wednesday. The Czech Ministry of Health released preliminary data on Tuesday that suggest that while the initial reproduction number in the Czech Republic stood at 2.64, this went down to 1.84 after March 12, when movement restrictions came into place.

To convince people to obey the rules, politicians need to make a convincing case that the situation is serious enough to merit such sacrifices.

“Unfortunately there is no one metric that enables us to compare situations in different countries,” Tildesley said. Looking at the number of deaths gives some indication about the severity of the situation in each country, but international comparisons are again complicated. No two countries are alike when it comes to access to healthcare and medical resources, underlying health conditions and even the age structure of the population.

“The number of deaths may be higher in different countries if there are not the same level of resources for treatment as in other countries, so it doesn’t necessarily mean that the disease is spreading in a different way or the controls are less effective,” Tildesley added.

Having more complete data on the number of all recoveries would be helpful. Scientists are hoping that a widespread testing for antibodies, which should become available at least in some countries in the coming days, would reveal who has had the illness in the past and help paint a more accurate picture of the epidemic.

But even when the numbers are accurate, they may be hard to comprehend. “Data is useful, but I think the most important thing is looking at the experience in other countries,” said Nick Chater, a professor of behavioral science at Warwick Business School.

“In the UK, the government’s projections are that in a good scenario, 20,000 people die … but that’s very hard to make sense of until you realize what’s happening in Italy now,” he said. “There are 6,000 deaths, and if you look at what it’s like to be dealing with the healthcare disaster there, it’s clearly absolutely terrible, and anything that’s much worse than that is clearly really really terrible. It’s these comparisons that bring the data alive.”