COVID-19 Models: Can They Tell Us What We Want to Know?
Josh Michaud, Jennifer Kates, and Larry Levitt
Published:
Models have taken center stage in many key policy discussions surrounding COVID-19, largely due to the unprecedented nature of the situation, the many uncertainties about the disease and the way ahead, and the need to make informed policy decisions now on how best to manage that path forward. The White House has used models to initially estimate somewhere between 100,000 and 240,000 Americans may die from coronavirus (more recently the estimates have been revised downward). States and cities are using models to shape their health system responses as the virus spreads in their communities. Researchers are using models to estimate important epidemiological characteristics of the disease such as the incubation period, transmissibility, asymptomaticity, and severity, as well as the likely impacts of different public health interventions such as social distancing, airport screening, travel restrictions, and contact tracing.
While clearly models can be important tools for understanding the disease and policy responses, their approaches and assumptions vary widely, and can give widely divergent results. To their credit, many modelers are transparent about the variables and assumptions used.
In this post, we look at the primary uses for disease models, basic model approaches, and a number of key existing models in the context of COVID-19. We build on many excellent news articles and modeling overviews that have already been published.
Uses of Models
Models can be helpful as tools to make informed guesses about the disease, its future spread, and effects of different actions and interventions. Models are particularly useful in situations where many data elements are not available or not possible to collect, which is the case now with COVID-19. Some of the information gaps that COVID-19 models can help address include:
- describing characteristics of the virus/disease for which there may be a lack information. For example, estimating incubation period or transmissibility (the R0);
- forecasting how many cases, deaths, hospitalizations, or other outcomes are likely to occur in a given location over a given time frame; and
- Understanding the potential effects of interventions and policies by looking at projections and different scenarios.
Below, we take a closer look at models that try to forecast or make projections about the future.
Model Approaches for Projections and Forecasts
Three key modeling approaches being used for forecasting and projections are:
- SEIR/SIR models are a common epidemiological modeling technique that divides an estimated population into different groups (“compartments”) such as “susceptible”, “exposed”, “infected”, and “removed/recovered”, and then applies a set of mathematical rules about how people move from one compartment to another, using assumptions about the disease process, social mixing, public health policies, and other aspects.
- Agent-based models create a simulated community and follow the interactions and resulting spread of disease among individuals (“agents”) in that community, based on assumptions and rules about things such as the individuals’ movement and mixing patterns, other behaviors and risks, and the health interventions and policies in place.
- Curve-fitting/extrapolation models infer trends about an epidemic in a given location by looking at the current status and then applying a mathematical approximation of the likely future epidemic path, which is drawn from experiences in other locations and/or assumptions about the population, transmission, and public health policies in place.
| Table 1: Example Covid-19 Forecast and Projection Models for the U.S. | |||
| Model and Organization(s) Responsible | Primary Approach | Outcomes Estimated and Timeframe | Selected Model Findings/Notes |
| Imperial College “non-pharmaceutical intervention” (NPI) Model | SEIR | Projected U.S. cases, deaths across a range of different mitigation and suppression scenarios, over the next year (to April 2021). | Projected 2.2 million U.S. deaths might occur in an “unmitigated” scenario |
| Institute for Health Metrics and Evaluation (IHME) Covid-19 Model | Curve-fitting/ extrapolation | Forecasts number of hospitalizations and deaths in the U.S. and by state, along with the timing of in the peak of hospitalizations and deaths, through August 2020. | Initially, the model forecast 81,000 deaths in the US by July. Results are updated daily, and as of Apr 12, that deaths estimate has been revised downward, to 61,545 by August 4. |
| Covid-19 Model from Northeastern University, Fogarty International Center, Fred Hutchison Cancer Center, University of Florida and others | Agent-based | Projects cases and deaths in the U.S. and by state, under no mitigation vs. “stay-at-home” scenario, through April 30, 2020. | As of April 4, model projected U.S. deaths would peak on April 8, and there would be approximately 52,575 COVID-19 deaths (range: 35,381 to 88,269) by April 30, 2020 |
| Columbia University Severe Covid-19 Risk Model (& mapping tool) | SEIR | Provides projections on number of severe cases, hospitalizations, critical care, ICU use, and deaths under different social distancing scenarios, for 3-week and 6-week periods starting April 2. | In different regions of the U.S. anywhere from 33,986 and 185,192 deaths could be averted through social distancing. |
| Los Alamos National Laboratory Confirmed and Forecasted Case Data Model | Curve-Fitting/ Extrapolation | Forecasts cases and deaths by U.S. state using assumptions about the growth rate in cases and deaths and the presence of social distancing interventions through May 20. | As an example, model best guess forecast for California as of April 8 is that there would be 138,100 cases and 4,082 deaths. |
| University of Pennsylvania Covid-19 Hospital Impact Model for Epidemics (CHIME) | SIR | Model allows users to set inputs and assumptions, then provides forecasts on expected number of hospitalizations, ICU bed demand, ventilator demand, and number of days these demands would exceed capacity at hospitals in a given area based on those inputs, over the next three months. | Using inputs for three University of Pennsylvania Health System hospitals, the model projected best- and worst-case scenarios for total hospital bed capacity needed would reach 3131 – 12,650, including 338 – 1,608 ICU beds and 118 to 599 ventilators. |
Limitations, Assumptions, and Uncertainties of Models
All models are going to be simplifications of complex biological and social processes. Outputs, projections, and forecasts can differ significantly depending on the modeling approach used, the assumptions implicit in the model, and the values of the input parameters. The same model can even give very different results if assumptions or input values are changed even slightly.
In some cases, model results may differ bases on assumptions about unknown parameters, like how transmissible the virus is or what share of people who get infected will die. In other cases, models make assumptions about what policy changes political leaders will make. In particular, many Covid-19 models are very sensitive to the degree of social distancing assumed and how long distancing will last. For example, the IHME model assumes stringent social distancing will be in place until deaths drop to below 0.3 per million per capita, which they presently estimate will occur in early May. The model also expects zero deaths in July and August of this year because authors assume “appropriate measures are put in place to guard against the reintroduction of COVID-19.” If either of these assumptions are too optimistic – social distancing is relaxed earlier, or re-introductions of the virus occur, the model will underestimate the burden of disease in the coming months.
Models often present “best guess” or median forecasts/projections, along with a range of uncertainty. Sometimes, these uncertainty ranges can be very large. Looking at the IHME model again, on April 13, the model projected that there would be a 1,648 deaths from COVID-19 in the U.S. on April 20, but that the number of deaths could range from 362 to 4,989.
Conclusion
For these reasons, it is best not to depend on a specific forecast or exact projection coming from a single model as being authoritative. Using multiple models, updating inputs and approaches given new information, and checking models against what real-time information is available can help diminish some of the limitations inherent in modeling. For example, many states, from North Carolina to Illinois to California, are using several different models to inform their decision-making. Most importantly, understanding a model’s assumptions is key. If a model assumes strict social distancing measures will stay in place, and those measures are loosened, you’re going to need a new model.
No model, or set of models, can serve as a crystal ball to predict what will happen in the future, but they can shed light and provide much needed perspective on aspects of the epidemic that might be otherwise unknowable.