For months, scientists and public officials have relied upon computer-based models to try to predict the trajectory or the coronavirus outbreak. But models are not crystal balls, and all of them involve human assumptions. PBS NewsHour science correspondent Miles O’Brien reports on how the efforts that go into making these models — and their ultimate purpose — are more complicated than many of us realize.
TRANSCRIPT
Judy Woodruff:
Even as more states are trying to reopen their economy, a new “PBS NewsHour”/NPR/Marist poll found that 77 percent of Americans worry about a second wave of infections yet to come.
This comes as computer-based models suggest that the U.S. will pass its own grim milestone by June, 100,000-plus deaths. That higher projection is arriving even sooner than some of the models estimated just weeks ago.
But models are not crystal balls. The work that goes into making them and their ultimate purpose is more complicated than you might be able to tell from the headlines.
Miles O’Brien explains in his latest report for our series the Leading Edge.
Miles O’Brien:
We live in a complicated world, filled with more data than insight.
Finding a path to clarity is not easy, even on a good day. And these are not good days. So, how can we take a huge amount of data and make it understandable, so we can see the future?
Betz Halloran:
You can’t believe every number that comes out. But if we don’t try to formulate our thinking about a complex process, then we will be running blind.
Miles O’Brien:
Betz Halloran is an infectious disease modeler. She writes mathematical formulas that define the chaotic, exponential spread of infection.
A biostatistician at Seattle’s Fred Hutchinson Cancer Research Center, she’s part of the team that curates the Global Epidemic and Mobility Model, or GLEAM.
Betz Halloran:
The GLEAM model is a big mobility model that can answer global questions.
Miles O’Brien:
GLEAM begins with the first infection in China and travels down the many paths of exponential growth, constantly calculating who is susceptible, exposed, infectious, and recovered, S-E-I-R, or SEIR.
Betz Halloran:
You can structure it in many different ways. But, usually, when we talk about infectious disease modeling, that’s the basic sort of meat and potatoes of what’s going to be in a model.
Miles O’Brien:
But the model does not stop there. It factors in the entire global transportation network, including airline schedules and capacity.
Betz Halloran:
So, the question we were asking way back then was, where is it going to spread? If it gets into the United States, where would it go first?
And once it gets in, then we could use GLEAM to look at the question of, how much is it going to spread in the different places? Where is it going to go first? And then we predicted that pretty well.
Miles O’Brien:
Halloran and her team did accurately predict where COVID-19 would first surge in the United States.
But, as the pandemic wore on, the limitations of the models became more evident. After all, no one really knows how the virus is transmitted, who’s likely to get sick and who won’t, who’s likely to die, who might have immunity.
All those questions won’t be answered until there is widespread testing. So, in the meantime, the models muddle on, with sometimes dizzyingly confusing results.
One of them, from Britain’s Imperial College, predicted two million COVID-19 deaths in the United States. But that assumed no human response, no social distancing.
Betz Halloran:
All models are wrong, but some models are helpful, and I think it’s important to remember that.
Miles O’Brien:
Nearby, at the University of Washington’s Institute for Health Metrics and Evaluation, they built a much simpler model that started with a specific question in mind: Did the health care system have the capacity to treat a surge of COVID-19 patients?
Chris Murray is the director. He and his team wrote a model that, unlike many others at the time, factored in the human response to the pandemic.
Christopher Murray:
If you ignore the behavioral response, you’re going to massively overshoot.
And so I think it is a reasonable strategy to try to look at models, like the economists do, which build in how individuals, local government, state government, are going to respond to the problems as they unfold.
Deborah Birx:
So I’m sure you’re interested in seeing all the states.
Miles O’Brien:
Producing speedy state-by-state results, with consistently lower projections, the University of Washington model was frequently cited by the White House in daily coronavirus briefings.
Deborah Birx:
And I think, if you ask Chris Murray, he would say…
Miles O’Brien:
But the model initially assumed there would be widespread adoption of social distancing restrictions in the U.S.
Once it became clear that wasn’t happening, the modeling team went back to the drawing board, releasing a new version on May 4. It now uses mobility data gleaned from cell phone usage to better understand how well people are complying with the expert advice.
As a result, that model’s projection for the total U.S. death toll by August 4 from COVID-19 instantly went from about 72,000 to 134,000.
Christopher Murray:
It’s sensible to try to look at a wide array of models and try to look at how — do they tell you the same story? Are they converging?
It’s very confusing, I think, for many decision-makers to navigate through some of the models.
Man:
We’re going to start off with this weekend.
Miles O’Brien:
Weather forecasters are some of the most adept at navigating the inherent uncertainties of modeling.
Man:
Going to have some travel problems if…
Miles O’Brien:
After all, it’s been 70 years since they first ran a model through a computer to create a forecast. It’s been steady improvement ever since. It’s now possible to reliably forecast seven days in advance with 80 percent accuracy.
But, with a novel virus, there are so many unknowns. And weather models do not have to account for human behavior.
Marshall Shepherd is director of the Atmospheric Sciences Program at the University of Georgia.
Marshall Shepherd:
It’s very important, when consuming these coronavirus models and weather models, to consume the uncertainty that we know is inherent.
But we have a way to get around that in weather called ensemble modeling.
Miles O’Brien:
Ensemble modeling, meaning combining the predictions of many different models, it’s a crucial tool that has greatly improved forecasting the weather and, in the past three years, seasonal influenza as well.
Nick Reich is an associate professor of biostatistics at the University of Massachusetts-Amherst. Working with the Centers for Disease Control and Prevention, he leads a team that builds ensemble models to improve predictions of the spread of the flu.
Nick Reich:
I don’t think any one model should be viewed as gospel truth.
When you just use one model, you end up with a too strong reliance on one particular set of assumptions and one particular viewpoint. And this is why it’s really critical to consider multiple models together.
Miles O’Brien:
The influenza models are informed by up to 20 years of experience with the viruses and the accuracy of the models.
Reich and his team have now built a COVID-19 ensemble model. But it, of course, does not have the benefits of a long backstory.
Nick Reich:
We do have hundreds of years of theory about how to build mathematical models of infectious disease, but have they ever been tested in real time in this way, with all of the data sources that are available to us? No.
We’re building this car as it’s careening down the highway, and we’re learning about these models as we go.
Miles O’Brien:
Infectious disease modelers are scrambling to figure out where we are headed, depending on the decisions we make.
If we take the time to better understand what the models can and cannot do, maybe we will do the same as we search for the path back to normalcy.
For the “PBS NewsHour,” I’m Miles O’Brien.
Banner image credit: PBS NewsHour.
