key: cord-0930464-ujxtr5va
authors: Koch, Christoffer; Okamura, Ken
title: Benford’s Law and COVID-19 reporting()
date: 2020-09-14
journal: Econ Lett
DOI: 10.1016/j.econlet.2020.109573
sha: 347d94b42dc9a03859ba8ac45a72d3528a026fed
doc_id: 930464
cord_uid: ujxtr5va

Trust in the reported data of contagious diseases in real time is important for policy makers. Media and politicians have cast doubt on Chinese reported data on COVID-19 cases. We find Chinese confirmed infections match the distribution expected in Benford’s Law and are similar to that seen in the U.S. and Italy. We identify a more likely candidate for problems in the policy making process: Poor multilateral data sharing on testing and sampling.

Contrary to popular speculation, we find no evidence that the Chinese massaged their COVID-19 statistics. We use a statistical fraud detection technique, Benford's (1938) Law, to assess the veracity of the statistics. This empirical finding is important because China was affected first.

Policies to combat the global pandemic are informed by its response. Skepticism about the Chinese data may result -and may indeed already have resulted -in poor policy choices.

Data sharing practices at the early stages of the pandemic were inadequate and led to costly policy errors.

The media frequently claim the Chinese government has understated the numbers of those affected. 1 Politicians echo these claims with President Trump declaring the reported death toll and infections seemed "a little bit on the light side". Much of the concern about Chinese data manipulation can be attributed to geopolitical tensions and foreign governments' need for a scapegoat. 2 The on-going doubts over the credibility of its published data are problematic as it impacts subsequent policy choices by countries that saw epidemics later. Papers that rely on Chinese data for calibration and analysis include: Models of economic activity and the trade-off with deaths such as Atkeson (2020), Jones, Philippon, and Venkateswaran (2020) and Alvarez, Argente, and Lippi (2020) ; Fang, Wang, and Yang (2020) predict the effect of movement restrictions on the spread of the disease. 3 Since countries patterned their social distancing and lockdown policies on the choices made by China, 4 policy makers need to know the data is reliable.

Lack of confidence in Chinese data may have contributed to a slower response in Europe to the emergent pandemic. Chinese provinces neighboring Hubei province, the Chinese epicentre, imposed movement controls, quarantines and checks on January 23 rd at a time when the number of confirmed cases in Hubei was 444 and the number of deaths was 17. 5 In comparison Italy, Europe's initial pandemic hotspot, reached 445 cases on February 26 th and 17 deaths the following day. It took until March 9 th for a national lockdown. Similarly, restrictions on international travel were too late and too mild. By February 26 th , Hubei had seen cases rise to 65,187 and deaths to 2,615. Skepticism about politically motivated manipulation of Chinese state statistics is deeply rooted. Anecdotal and academic evidence point to lower level officials manipulating data to meet targets. In 2007, Chinese Premier Li Kejiang called all GDP measures "man-made and therefore not reliable" when discussing data on Liaoning province. 6 Lyu, Wang, Zhang, and Zhang (2018) find evidence that regional growth rates are manipulated to meet growth targets.

In the case of the SARS outbreak in 2002-3, criticism of the Chinese response surfaced. The World Health Organization (WHO) suspected that China underreported the number of cases (see Parry, 2003 

Data manipulation took place early in the epidemic. 8 The number of cases reported by the Wuhan authorities was "frozen" at 41 during the Hubei provincial Chinese People's Political Consultative Conference and the Wuhan People's Congress (Lianghui) between January 12 th and 17 th , 2020. A member of the WHO emergency committee, John Mackenzie, told the Financial Times on February 5 th that China must have been withholding information on new cases. 9 2 Benford's Law Benford' s Law is used to detect fraud or flaws in data collection based on the distribution of the first digits of observed data. A Benford distribution of first digits arises naturally for exponential processes with multiple changes of magnitude, Michalski and Stoltz (2013) . The spread of COVID-19 demonstrates exponential growth and changes of magnitude.

The frequency with which the first digit is "1" is 30.1%, the first digit is "2" is 17.6% etc, declining to the first digit being "9" only 4.6% of the time. Since it takes a 100% increase to go from "1" to "2" and a mere 11.1% increase to go from "9" to "1", this logarithmic distribution makes sense. See Table (2011), Holz (2014) and Nigrini (1996) .

We compile data from the Johns Hopkins University Corona Virus Research Center, for China and the Centers for Disease Control for the U.S. For Italy our data comes from the daily Dipartimento della Protezione Civile bulletins. The time period for each country matches the period when the pandemic goes through its exponential growth phase and then declines as measures to combat the infection such as quarantines and lockdowns are instituted as in ing capacity was also a problem in both Italy and the U.S. As long as the choice of sampling methodology does not change, the number of confirmed cases will also follow an exponential path and thus Benford's Law.

As sampling changes, through testing different groups or changing definitions, we would expect to find a drift away from Benford's Law. For example, if a country only tests those who present at hospital with symptoms, this sample will grow exponentially along with the overall number of infected. If the country then increases testing to those who are symptomatic, but not hospitalized, this sample will also grow exponentially, but will not be the same series as the hospital sample. Frequent sampling changes, in terms of the populations, definitions, or accuracy will lead to deviation from Benford's Law.

The sample periods match the exponential growth phase of the pandemic and the subsequent deceleration. We expect the period pre-lockdown to follow a Benford distribution, the period post-lockdown is a treatment period that should disrupt the Benford distribution.

We follow Kissler, Tedijanto, Lipsitch, and Grad (2020) and assume 9 days between infection and hospitalization and assume infections are detected with hospitalization. We use the dates 4 J o u r n a l P r e -p r o o f given in Fang, Wang, and Yang (2020) as proxies for Chinese provincial lockdowns. For Italy we use the national lockdown on March 9 th for all regions. For U.S. states, we use the date of a"stay-at-home order". Tests of significance for Benford's Law require that the "true" distribution should follow the Benford distribution. Our null hypothesis is that the observed distribution follows the theoretical (Benford) distribution. The most common test is the Chi-Square test of Goodness of Fit:

Where n denotes the number of observations, h is the observed frequencies of the digits and p is the Benford's Law distribution. We also use a Kuiper test (a modified Kolmogorov-Smirnov 5 J o u r n a l P r e -p r o o f Journal Pre-proof test).

and H d and P d represent the cumulative frequencies of the first digit d in the observed data and the Benford distribution. We also calculate the m (max) statistic where m = max d=1,...,9 |h d − p d | and the d (distance) statistic Notes: * * * , * * , and * denotes statistical significance at the 1%, 5%, and 10% level. We find in Table 3 , as expected, pre-lockdown matches Benford far more than the overall period for Italy and China. The U.S. distribution is close to Benford for the entire period and does not appear to change significantly before and after the lockdowns are announced. (Dave, Friedson, Matsuzawa, and Sabia, 2020) .

Given the doubts about the reliability of Chinese data, why did the Italians delay? The likely reason is in the sampling of infections. The Italian government believed they were detecting a far higher proportion of infected than the Chinese had managed at the same point in the pandemic. On February 27 th , "Public health officials have said that Italy contributed to fears of an epidemic in Europe with its zealousness in testing" 10 and Dr. Walter Ricciardi, an Italian government adviser and World Health Organization (WHO) official, was quoted as saying there was "too much testing". 11 In retrospect, the Italians had not "tested too much" and their rate of virus detection was no better than that of the Chinese.

Information on the extent of testing early in the pandemic is lacking in both China and Europe. Hubei authorities were able to undertake 4,000 tests per day by February 4 th . Even

Germany, lauded for its testing regimen, had not published data on the number of tests in late February, leading to Dr. Ricciardi claiming as late as March 7 th "from an epidemiologic point of view, it is not plausible that Italy ... accounts for more cases than Germany and France." 12

It is possible to create data series that fit Benford's Law (Diekmann, 2007) . To manipulate the Chinese data requires coordination of daily announcements across all provinces while accurately forecasting future infection rates. This is improbable. With the benefit of hindsight, we now know that the Italians had a similar sampling method to China. Both countries' distributions of first digits for confirmed cases pre-lockdown follow Benford's Law. A key insight from our analysis is a focus on the underlying data sampling processes. The ongoing geopolitical dynamics and escalation of policy rhetoric between the U.S., European countries, and China have obscured one of the causes of poor policy responses. We conclude a refinement of the who:2008 International Health Regulations Article VI §2 to share timely, accurate and sufficiently detailed information on the extent and reliability of testing is necessary.

A Simple Planning Problem for COVID-19 Lockdown

What Will Be the Economic Impact of COVID-19 in the US? Rough Estimates of Disease Scenarios

The Law of Anomalous Numbers

When Do Shelter-in-Place Orders Fight COVID-19 Best? Policy Heterogeneity Across States and Adoption Time

Not the First Digit! Using Benford's Law to Detect Fraudulent Scientific Data

The Macroeconomics of Epidemics. Working Paper 26882

Human Mobility Restrictions and the Spread of the Novel Coronavirus (2019-nCoV) in China

Benford's Law and Macroeconomic Data Quality

The Quality of China's GDP Statistics

Optimal Mitigation Policies in a Pandemic: Social Distancing and Working from Home

Social distancing strategies for curbing the COVID-19 epidemic

GDP Management to Meet or Beat Growth Targets

Do Countries Falsify Economic Data Strategically? Some Evidence That They Might

A Taxpayer Compliance Application of Benford's Law". The Journal of the American Taxation Association 18.1

The Political Economy of Numbers: On the Application of Benford's Law to International Macroeconomic Statistics

WHO is worried that China is under-reporting SARS

Impacts of Social and Economic Factors on the Transmission of Coronavirus Disease 2019 (COVID-19) in China