Flying Blind on Coronavirus: Why Random Testing is so Important.
The coronavirus crisis is one of the most disruptive events to hit the U.S. in a very long time.
Major sectors of the economy are being shuttered, people lives are being altered in the most profound ways, and the nation is facing extreme stress, the implications of which we do not understand.
Extraordinarily serious decisions are being made without key information: how many individuals have active infections? How many have had the virus and now have immunity? What percentage of infected individuals have few or no symptoms? Who is currently infected and needs to be quarantined? Is the current reduction in cases in Washington and elsewhere mainly the result of social distancing or the herd immunity of an increasing number of individuals that have had the virus?
For all these questions, we do not know the answer.
Our best medical scientists and epidemiologists, including a highly respected group at the University of Washington, are making projections of the future progression of the pandemic, but without sufficient information for initializing their models, resulting in a rapid decline of accuracy with time and large uncertainty in the projections.
We are flying blind.
Lack of information undermines our ability to manage the crisis. And it doesn't have to be this way.
The bottom line of this blog is that we must immediately begin random sampling of the population using both PCR testing that tells up about active infections and antibody tests that informs about who has been infected int the past.
Random sampling of populations is an essential tool for the social, biological, and physical sciences.
Political pollsters randomly sample potential voters to predict the outcomes of elections. They don't provide election projections by counting the number of avowed Democrats or Republicans who come knocking on their door. In a variety of fields, random sampling of populations is a key tool for decision making. But in the coronavirus situation, we are content not to use this powerful tool, even when we make the gravest decisions. It doesn't make sense.
Here in Washington State, virtually all the testing is being used to determine whether individuals suffering from a respiratory illness ailments have active coronavirus infections. There is, of course, good reason to test sick individuals: their treatment plan can be enhanced with such knowledge and their care-givers need to know whether they require protection (PPEs-- personal protection equipment). We know how many folks are getting tested (and not everyone that is sick is getting tested) and the percentage of those tests that are positive for active COVID infections. That is not enough.
The current testing regime leaves decision makers poorly informed. How many individuals currently have active infections, with or without symptoms? How many people have had the virus and are now potentially immune? We don't know. And without such information, it is nearly impossible to project the future.
Some of the best information we have on infections locally comes from the testing done by the UW Virology Department. As of yesterday, they had tested about 51,000 individuals, with roughly 10% testing positive. Surely, the percentage of the total population that is currently infected with COVID-19 must be less than this percentage derived from sick folks (who can be ill for a number of reasons)--but how much less? Testing ramped up in March went goes up and down.
Below is a figure that has never been shown in the media: the ratio of positive to negative results from the UW Virology testing. The ratio substantially increased between mid-March to the end of the month (from roughly 6% to 17%), suggesting that a higher proportion of tested sick people were suffering from COVID-19. Importantly, that number has been declining rapidly in April, suggesting major progress.
The Washington State Department of Health provides statewide totals of confirmed cases (see below), which have increased substantially since early March. But how much of the increase in the number of confirmed infections is the result of vastly increased testing during the past several weeks (as shown by the second figure) and how much by increased presence of COVID-19 in the population? We simply don't have that information.
Epidemiological projection, like numerical weather prediction, is an initial value problem. You start with an estimate of the initial state and your model, which contains information about the processes of the phenomena in question, attempts to project the initial state into the future. If your initial state is uncertain, so is your forecast.
There is an intrepid group at the University of Washington (IHME) attempting to do such projections using a statistical model, with COVID-19 forecasts for both individual states and the nation. Decision makers are making heavy use of their estimates. Here is a sample of their latest forecasts for Washington State deaths from the virus. The observed COVID deaths are shown by the solid red line. The dashed line shows their best estimate of the future and the shading indicates the uncertainty in the projections. HUGE uncertainties, in part because of the uncertainties in our knowledge of past and current active infections and the "herd immunity" of folks that have recovered from the virus.
But poor initial data (lack of knowledge of infections in the total populations) has resulted in their projections changing substantially in time, undermining their value to decision makers.
Below are their projections starting on March 26, April 1, and April 5. Huge differences are apparent. Their projection made on March 26th (when they started this valuable service) are shown below. The light gray line is their best projection (actually the median of their distribution) and the uncertainty is noted by light gray shading. This projection shows a peak in late April of around 27 deaths a day, and high numbers continuing into May.
The April 5th projection (solid red line, again the median of their forecast distribution) is much more benign, showing the epidemic collapsing by the end of April with far less than half the deaths, and the peak being in early April. According to their latest projections, we are now probably past the peak, something consistent with the drop in hospitalizations in our state due to the virus (see below). I suspect that the next project will be even lower.
Washington State is rapidly getting out of this terrible situation due to the combination of social distancing and the building immunity of the population. But we are pretty much ignorant about the magnitude of the herd immunity because we do not know how many have had and currently have the virus.
Some researchers are trying to indirectly secure an idea of what has happened using a combination of genetic testing of viral samples coupled with epidemiological modeling. A recent paper by Bedford et al (2020, not yet peer reviewed) suggests that the virus reached Washington State during mid to late January and then spread throughout the local population, asymptomatic for many.
Since random sampling was not available, they used mathematical models to estimate spread (see below) using a large number of simulations informed by their genetics testing. The red line is the median of these simulations (you might consider that to be their best estimate). They suggest that by mid-March roughly 2000 Washington citizens were infected as the infection increased exponentially in the population. Without any influence of social distancing, their work suggests the potential for roughly 10,000 cases by April 1 (simply by extrapolating the exponential rise of the red line).
Unfortunately, we lack the information to know what has happened here in Washington. But is clear there this virus has been spreading among us for at least two months, with many individuals unaware of being infected. There is a substantial evidence for this asymptomatic spread such as the example of the Diamond Princess cruise ship, a vessel in which the disease went rampant and nearly everyone was tested. Roughly 50% of the sick, in a population heavily skewed towards older folks, had no symptoms. Could it be even higher in a younger, healthier population? We don't know.
The wake up call for our State was when the spreading virus randomly hit a nursing home full of elderly, sick individuals in Kirkland, producing dozens of deaths and serving as a focus for spread into the community. What would have happened without that alarming situation?
We need random sampling of the population.
This situation of not knowing what is happening in the general population is crippling society's ability to deal with the virus in an informed, smart way. The capability to test is increasing now and our State and nation must give random testing of the population very high priority immediately, securing a significant percentage of the rapidly increasing testing capability to random sampling.
Such sampling offers enormous benefits. First, it will greatly improve the quality of our projections of the disease. Second, it will tell us why the situation is improving (social distancing versus herd immunity) and guide governmental actions. Third, it will enable us to squash the disease spread, by quarantining every individual that tests positive for an active infection, and to trace (and quarantine) his/her contacts. South Korea has shown the huge value of the widespread testing/quarantining approach.
It is the only way we can stay on top of the virus when we inevitably have to loosen the restrictions on our population, which we will have to do to prevent economic collapse and social disorder.
Here in Seattle, we have some of the most sophisticated mathematical modelers, statistical analysts, medical experts, and private sector analytic experts in the world, yet we are content to cripple our economy and deal with such an historic event in nearly complete ignorance of the situation. We, of all places in the world, can do much better. It is time for random sampling of our population for COVID-19 looking for both active and past infections.
When the story of this event is written, the inability to rapidly initiate and actively use large scale random testing back in February will be seen as a terrible failure of the Centers for Disease Control, state health organizations, and our Federal/state leadership that did not demand it. And this failure was compounded by other issues, such as not encouraging all Americans to wear masks early in the event and not moving immediately to secure necessary PPE and other supplies.
But we are where we are and it is time now to move from a reactive to more reasoned approach, with random testing of the population being a measure that needs immediate priority.
Major sectors of the economy are being shuttered, people lives are being altered in the most profound ways, and the nation is facing extreme stress, the implications of which we do not understand.
Extraordinarily serious decisions are being made without key information: how many individuals have active infections? How many have had the virus and now have immunity? What percentage of infected individuals have few or no symptoms? Who is currently infected and needs to be quarantined? Is the current reduction in cases in Washington and elsewhere mainly the result of social distancing or the herd immunity of an increasing number of individuals that have had the virus?
For all these questions, we do not know the answer.
Our best medical scientists and epidemiologists, including a highly respected group at the University of Washington, are making projections of the future progression of the pandemic, but without sufficient information for initializing their models, resulting in a rapid decline of accuracy with time and large uncertainty in the projections.
We are flying blind.
Lack of information undermines our ability to manage the crisis. And it doesn't have to be this way.
The bottom line of this blog is that we must immediately begin random sampling of the population using both PCR testing that tells up about active infections and antibody tests that informs about who has been infected int the past.
Random sampling of populations is an essential tool for the social, biological, and physical sciences.
Political pollsters randomly sample potential voters to predict the outcomes of elections. They don't provide election projections by counting the number of avowed Democrats or Republicans who come knocking on their door. In a variety of fields, random sampling of populations is a key tool for decision making. But in the coronavirus situation, we are content not to use this powerful tool, even when we make the gravest decisions. It doesn't make sense.
Here in Washington State, virtually all the testing is being used to determine whether individuals suffering from a respiratory illness ailments have active coronavirus infections. There is, of course, good reason to test sick individuals: their treatment plan can be enhanced with such knowledge and their care-givers need to know whether they require protection (PPEs-- personal protection equipment). We know how many folks are getting tested (and not everyone that is sick is getting tested) and the percentage of those tests that are positive for active COVID infections. That is not enough.
The current testing regime leaves decision makers poorly informed. How many individuals currently have active infections, with or without symptoms? How many people have had the virus and are now potentially immune? We don't know. And without such information, it is nearly impossible to project the future.
Some of the best information we have on infections locally comes from the testing done by the UW Virology Department. As of yesterday, they had tested about 51,000 individuals, with roughly 10% testing positive. Surely, the percentage of the total population that is currently infected with COVID-19 must be less than this percentage derived from sick folks (who can be ill for a number of reasons)--but how much less? Testing ramped up in March went goes up and down.
Below is a figure that has never been shown in the media: the ratio of positive to negative results from the UW Virology testing. The ratio substantially increased between mid-March to the end of the month (from roughly 6% to 17%), suggesting that a higher proportion of tested sick people were suffering from COVID-19. Importantly, that number has been declining rapidly in April, suggesting major progress.
The Washington State Department of Health provides statewide totals of confirmed cases (see below), which have increased substantially since early March. But how much of the increase in the number of confirmed infections is the result of vastly increased testing during the past several weeks (as shown by the second figure) and how much by increased presence of COVID-19 in the population? We simply don't have that information.
Epidemiological projection, like numerical weather prediction, is an initial value problem. You start with an estimate of the initial state and your model, which contains information about the processes of the phenomena in question, attempts to project the initial state into the future. If your initial state is uncertain, so is your forecast.
There is an intrepid group at the University of Washington (IHME) attempting to do such projections using a statistical model, with COVID-19 forecasts for both individual states and the nation. Decision makers are making heavy use of their estimates. Here is a sample of their latest forecasts for Washington State deaths from the virus. The observed COVID deaths are shown by the solid red line. The dashed line shows their best estimate of the future and the shading indicates the uncertainty in the projections. HUGE uncertainties, in part because of the uncertainties in our knowledge of past and current active infections and the "herd immunity" of folks that have recovered from the virus.
Below are their projections starting on March 26, April 1, and April 5. Huge differences are apparent. Their projection made on March 26th (when they started this valuable service) are shown below. The light gray line is their best projection (actually the median of their distribution) and the uncertainty is noted by light gray shading. This projection shows a peak in late April of around 27 deaths a day, and high numbers continuing into May.
The April 5th projection (solid red line, again the median of their forecast distribution) is much more benign, showing the epidemic collapsing by the end of April with far less than half the deaths, and the peak being in early April. According to their latest projections, we are now probably past the peak, something consistent with the drop in hospitalizations in our state due to the virus (see below). I suspect that the next project will be even lower.
Washington State is rapidly getting out of this terrible situation due to the combination of social distancing and the building immunity of the population. But we are pretty much ignorant about the magnitude of the herd immunity because we do not know how many have had and currently have the virus.
Some researchers are trying to indirectly secure an idea of what has happened using a combination of genetic testing of viral samples coupled with epidemiological modeling. A recent paper by Bedford et al (2020, not yet peer reviewed) suggests that the virus reached Washington State during mid to late January and then spread throughout the local population, asymptomatic for many.
Since random sampling was not available, they used mathematical models to estimate spread (see below) using a large number of simulations informed by their genetics testing. The red line is the median of these simulations (you might consider that to be their best estimate). They suggest that by mid-March roughly 2000 Washington citizens were infected as the infection increased exponentially in the population. Without any influence of social distancing, their work suggests the potential for roughly 10,000 cases by April 1 (simply by extrapolating the exponential rise of the red line).
The wake up call for our State was when the spreading virus randomly hit a nursing home full of elderly, sick individuals in Kirkland, producing dozens of deaths and serving as a focus for spread into the community. What would have happened without that alarming situation?
We need random sampling of the population.
This situation of not knowing what is happening in the general population is crippling society's ability to deal with the virus in an informed, smart way. The capability to test is increasing now and our State and nation must give random testing of the population very high priority immediately, securing a significant percentage of the rapidly increasing testing capability to random sampling.
Such sampling offers enormous benefits. First, it will greatly improve the quality of our projections of the disease. Second, it will tell us why the situation is improving (social distancing versus herd immunity) and guide governmental actions. Third, it will enable us to squash the disease spread, by quarantining every individual that tests positive for an active infection, and to trace (and quarantine) his/her contacts. South Korea has shown the huge value of the widespread testing/quarantining approach.
It is the only way we can stay on top of the virus when we inevitably have to loosen the restrictions on our population, which we will have to do to prevent economic collapse and social disorder.
Here in Seattle, we have some of the most sophisticated mathematical modelers, statistical analysts, medical experts, and private sector analytic experts in the world, yet we are content to cripple our economy and deal with such an historic event in nearly complete ignorance of the situation. We, of all places in the world, can do much better. It is time for random sampling of our population for COVID-19 looking for both active and past infections.
When the story of this event is written, the inability to rapidly initiate and actively use large scale random testing back in February will be seen as a terrible failure of the Centers for Disease Control, state health organizations, and our Federal/state leadership that did not demand it. And this failure was compounded by other issues, such as not encouraging all Americans to wear masks early in the event and not moving immediately to secure necessary PPE and other supplies.
But we are where we are and it is time now to move from a reactive to more reasoned approach, with random testing of the population being a measure that needs immediate priority.
Comments
Post a Comment