COVID-19 World Survey Data API

Important Notice 12-6-2020

Summary
The definition of Covid-like illness (CLI) and influenza-like illness (ILI) have been inadvertently switched in our code for the World COVID-19 Symptoms Survey since the survey began running.

As of today (December 6), we have released an update to our API that fixes the issue for both historic and future data. Below is a detailed summary of the issue and fix. Please reach out with any questions.

Overview
The World COVID-19 Symptom Survey asks respondents to self-report whether they have experienced any of a list of symptoms over the past 24 hours. We aggregate responses containing specific combinations of symptoms to estimate Covid-like illness (CLI) and influenza-like illness (ILI) prevalence within regions and countries or territories.

The intended definitions are as follows:

• Covid-like illness (CLI): fever along with cough, shortness of breath, or difficulty breathing
• Influenza-like illness (ILI): fever, along with cough or sore throat

When the definitions were initially coded, they were inadvertently switched. That is, the coding of CLI was implemented using the influenza-like symptoms and the coding of ILI was implemented using the Covid-like symptoms.

Detection
This issue was discovered when we were testing a new set of aggregate signals that further restricted the definition of CLI to include anosmia. When these aggregate estimates were compared to the API’s CLI and ILI estimates, we found that the “restricted” CLI estimates were bounded not by the CLI estimates as was expected, but instead by the ILI estimates, which led to further investigation.

Scope
This issue affects estimates reported in our aggregate data API since the survey began fielding in April. Consequently, it also affects estimates provided in our COVID-19 World Symptom Map and Facebook’s COVID-19 Interactive Map & Dashboard that both rely on the aggregates API as the underlying data source.

This issue does not affect individual-level survey responses that are provided to entities who have signed a microdata use agreement. It also does not affect the US data which has been collected and distributed by Carnegie Mellon University. US estimates in the UMD and Facebook maps are also unaffected.

Impact
The definitions of ILI and CLI are overlapping, that is, both require the presence of fever and at least one other of a group of symptoms—this group includes cough for both ILI and CLI. Because of this, the two signals are highly correlated. Overall, from May to present, the correlation between smoothed_pct_cli and smoothed_pct_ili is 0.977. At the country-level, only 8 countries (Cameroon, Guinea, Laos, Mauritania, Mozambique, Oman, Qatar, Sudan) had a correlation between CLI and ILI of between 0.8 and 0.9; the remaining 123 countries had a correlation above 0.9. Based on this, the impact of this issue to modeling and decision-making is expected to have been small.

Mitigation
We have released an update to our API that fixes the issue for both historic and future data. Users who have downloaded historic data will need to re-download the data to ensure that they have up-to-date estimates. Our map has also been updated to reflect this change, and Facebook will be incorporating the updated API data into their map.