The lay public doesn’t realize how often medical experts get things wrong. And they don’t want to consider such a possibility because the consequences of medical error are too great to grapple with. The reason medical experts are often wrong is intrinsic to the properties of the profession. Medicine requires that important decisions be made on the basis of inadequate information. The COVID-19 epidemic and our response to it is a perfect example of medical advice that has effects far beyond the domain of medical practice and which is founded on limited data.
The oldest principle of medicine is “First, do no harm.” When a medical expert, such as Drs. Fauci or Birx, tells the leader of the country to shut down the economy, this physician can be certain that a lot of harm will follow. Thus, violating “Primum non nocere” should require a rigorous cost-benefit analysis. The problem is that a doctor has no expertise relevant to the non-medical consequences of his advice. He lacks the expertise to do the rigorous analysis that the problem requires.
The do-no-harm directive is two and a half millennia old. Evidence based medicine is a fairly recent phenomenon. That medical practice should be based on solid evidence seems so obvious that one wonders why it’s necessary to say so. Despite declaring allegiance to evidence-based medicine, the profession regularly disregards its requirements. A few examples of non-evidence-based medicine are informative.
Consider a test that has been prescribed for decades. Screening for prostate cancer using blood levels of prostatic specific antigen (PSA) began to be used before any controlled studies of its utility were performed. From the start, a minority of doctors pointed out that this type of screening had not been validated. After decades of controversy, it became clear that the test was of limited value. As many as 90% of cases of prostate cancer are not lethal. These men died with prostate cancer, but not from it. Thus, diagnosing the disease subjected them to all the side effects of treatment without any benefit, side effects that were frequent and often debilitating. About 9% of cases were so aggressive that treatment didn’t work no matter how early the disease was diagnosed. This left about 1% of patients with the disease who might benefit from early detection. Despite lack of validation it was widely used in most men once they reached middle age. After many years of gradually increasing protests by more and more physicians its use was changed to the current recommendation from the National Cancer Institute about PSA screening:
Until about 2008, some doctors and professional organizations encouraged yearly PSA screening for men beginning at age 50. Some organizations recommended that men who are at higher risk of prostate cancer, including African American men and men whose father or brother had prostate cancer, begin screening at age 40 or 45. However, as more was learned about both the benefits and harms of prostate cancer screening, a number of organizations began to caution against routine population screening. Most organizations recommend that men who are considering PSA screening first discuss the risks and benefits with their doctors.
Notice the end where it says that patients should first discuss the risks and benefits of PSA screening with their doctors. That doctors have to be told to discuss diagnosis and treatment with their patients says volumes about medical practice. All of this paragraph was blindingly obvious almost half a century ago. The profession goes on diagnostic and therapeutic wild goose chases with distressing frequency.
Other examples of leaping before we looked are mammogram screening and cholesterol monitoring. The use of both procedures has undergone significant modification since they were first used. They were widely employed before evidence was available as to how they were best applied.
Medical standards of best practice often change rapidly. Frequently, such change results from the realization that some of our diagnostic and therapeutic procedures were poorly based from the onset. Medicine includes many physicians whose zeal to make things better interferes with rigorous scientific analysis. It often takes the rest of the profession years to overcome the initial zeal of those who started practices not consistent with evidence-based medicine.
The radical mastectomy introduced at the end of the nineteenth century by the great Johns Hopkins surgeon William Halsted, was standard of care for the treatment of breast cancer for more than half a century, even though it was based on faulty logic apparent when the surgery was introduced. In the 1950s two surgeons, Pack and Ariel,1 began to treat the disease with simple removal of the tumor, now called lumpectomy. They were called murderers for withholding the mutilating radical mastectomy. The science was on their side, but decades had to pass before surgeons accepted the validity of their treatment.
The lesson is that doctors frequently get things wrong even when the evidence, if rigorously examined, shows the error of their thinking. It commonly takes a lot of time to sort things out. When a new, unexpected, and urgent medical event appears, a decision has to be rapidly made. This is the circumstance most likely to result in medical errors.
Now to COVID-19. When the infection was first recognized we knew little about it save that it was a nasty respiratory virus, related to SARS and MERS, that had the potential to cause lethal pneumonia. How lethal we didn’t know. Various models were used to estimate how much damage the virus could cause. The problem with different models is that only one of them may prove true or all of them will be false. They might help in responding to a viral epidemic, but they are of little use when a new pathogen is identified as we don’t have enough evidence to formulate a response that is more than a guess.
An additional word about models. They all rely on assumptions. These assumptions are always inadequate because they can never include everything and the neglect of even the tiniest variable can cause them to deviate from what actually will happen. They also cannot predict events subsequent to their construction which will drastically change what really happens and consequently require constant revision. Models of events far in the future, such as those predicting the climate decades hence, are sure to be wrong as innumerable future events that are unknowable will inevitably muddle the model. Consider the annual hurricane models that follow every tropical storm that develops off the west coast of Africa. They differ widely and often are all wrong, even though their time span is days not years. They only approach reality when the storm they model is just a day or so from hitting land. Mamas, don’t let your babies grow up to be modelers.
We did know that the virus was highly contagious. It didn’t take long to realize that person to person spread via droplets was responsible for most cases. Thus, people in close contact were most likely to contract the disease. New York City with its extensive network of crowded buses and subways, unique in America, was designed to foster an outbreak of COVID-19 infection. Our next biggest metropolitan area, Los Angeles, with its widespread use of single occupant car travel, was designed to be spared New York’s fate.
The federal government stepped into the COVID-19 mess in early March. Testing had become available and President Trump relied on the expertise of Drs. Fauci and Birx. Initially, they used models which predicted appalling numbers of deaths and they recommended locking down the country. The president took their advice. Yet he had possessed other options. He could have asked for a second or third medical opinion. Perhaps he did, but there is no record of additional medical consultation. There were, and still are, epidemiologists who thought locking the economy was not necessary. I’ll discuss below what I thought should have been done.
Birx and Fauci also failed to explain how to interpret screening tests. This failure caused the public to be alarmed beyond reason as positive tests for COVID-19 were returned with distressing frequency. Here’s how to interpret a screening test. This is the information Fauci and Birx still haven't offered the public. Most doctors are not fluent in the details of screening and thus are not reliable sources about screening during an epidemic, or under any circumstances.
The details of screening are given in a short PowerPoint presentation linked below.2 Briefly, a screening test has a sensitivity and specificity. Sensitivity measures likelihood of getting a positive test when applied to patients who actually have the disease. It also defines the percent of false positives that will be observed. The positive predictive value (PPV) is the percent of patients who test positive and actually have the disease. The lower a test’s sensitivity, the lower the PPV. Also, the lower the prevalence of the disease in the test population, the lower the PPV. Specificity determines the likelihood of a negative test being a true negative. See the slide show below for more details.
Drs. Fauci and Birx, as well as the Johns Hopkins website devoted to the pandemic, have never explained the issue of false positive tests to the public. Rather the doctors and the Hopkins site have ignored the issue of false positive tests and assumed every patient with a positive test to be a confirmed case.
The admittedly arcane details of screening also explain why there are so many false positive mammograms, as the test has a sensitivity of no more than 60%.
As of this writing we have no idea how many false positive tests we are getting because we don’t know the prevalence of the disease in the whole population or subdivisions of it. Additionally, most of the tests used don’t tell us their sensitivity. Many patients who test positive for the virus are said to have little or no symptoms. Part of the reason for a lot of asymptomatic patients could be that false positive tests are misidentified as true positives.
The University of California at San Francisco (UCSF) CRISR test claims a sensitivity of 93%. The specificity of the UCSF test is said to be 100%. If this is true, there will be no false negatives when their test is used.
One of President Trump’s COVID-19 team recommended that everyone get vaccinated for the flu to prevent the combination of seasonal flu with a simultaneous reemergence of COVID-19. A study published last month in the Annals of Internal Medicine on flu vaccination in the elderly in the UK concluded “that no evidence indicated that vaccination reduced hospitalizations or mortality among elderly persons.”3 Since this is the population most likely to be adversely affected by COVID-19, we should not take a lot solace from vaccinating everyone in sight against the flu.
What should we have done? I do not think that the “sometimes you have to destroy a village to save it” approach should have been taken. The deleterious effects on social and economic life in the United States are likely to far outweigh the benefits from sheltering in place for two months or more.
We knew right from the start who the vulnerable were — the old and infirm. They should have been warned to self-isolate, particularly those in nursing homes. They should have continued this practice until the epidemic abated. Instead we told everyone to stay home to “flatten the curve”—the most overused and misunderstood term of recent years. We wanted to protect our hospitals from being overwhelmed by patients with viral pneumonia. Instead we kept everyone else who needed hospital care away and have nearly bankrupted much of medical system. We also put a dagger to the heart of our economy.
How do viral epidemics end? There are only two ways. First, everyone who was susceptible (excluding the elderly and infirm) to the virus could have been allowed exposure to it and herd immunity acquired. This form of immunity usually requires 60-90% of the population to contract the virus. Or, we could develop an effective vaccine. Getting a workable vaccine against a virus is not always easy. Shutting down the economy does not end an epidemic; if anything, it prolongs it.
We have the most advanced biomedical establishment in the world; if getting a workable vaccine is doable, our scientists will do it. But we still don’t have a vaccine against HIV more than 35 years after the virus was identified. And even if a safe and effective vaccine against this virus is developed, it will take time. It’s possible that the virus will be gone before the vaccine arrives. We have no idea if it will soon fade or when and if it will return.
It seems to me that we took the wrong path to defeat a really nasty virus. We could have successfully managed the disease without wrecking civilization. Medical advice should always be tempered with healthy doses of skepticism and restraint. The image of the blind men and the elephant should always be with us. Trump’s doctors saw just part of the beast. Our response to this infestation should have been more nuanced and included a rigorous cost-benefit analysis. It should have been serious, but proportionate. It was excessive in the extreme.
As mentioned above, we likely have overestimated the true number of cases by using screening tests of varied sensitivities. We will know how many people were infected when we measure antibody levels to the virus in large populations.
Deaths caused by the virus are also hard to know as very loose standards of mortality are currently employed. We seem to be counting death from the virus as a death in anyone suspected of having the virus irrespective of underlying morbidity. Using loose mortality criteria will result in an overestimate of deaths from COVID-19. A little fewer than 8,000 people die every day in the United States. Mortality in excess of this amount may give a true estimate of deaths caused by the virus.
At present the number of patients hospitalized because of COVID-19 is the best measure of the infection’s severity. While also subject to error, it’s probably the best we can do right now.
The epidemic will kill many more people before it subsides, which is an awful consequence, but preferable to killing the country. Our task is to treat the disease as effectively as possible while allowing the country to function effectively. We should use approaches that are germane to each part of our large and diverse land. The cessation of all social and economic life is not a rational plan. Social and economic policy based solely on medical advice is a prescription for disaster—especially as our medically directed response to this disease has been ill conceived and poorly executed. It also has been contaminated by politics. I hope it is not too late to chart a more reasonable course. Every medical student is taught how to triage. When faced with a disease that threatens both individuals and society, difficult decisions have to be made. We cannot kill the country in order to save everyone’s life. People die even when a epidemic is perfectly managed.
Finally, if you think the deleterious effects of the shutdown on America are bad, consider India. The country is poor and has nine times the population density of the America. Our behavior sets policy for the rest of the world. Social distancing in India is impossible. India has survived epidemics for thousands of years. Its population is always on the brink of starvation. Following our lead in shutting down the sub-continent may cause millions starve to death.
Neil A. Kurtzman, MD is Grover E. Murray Professor Emeritus and University Distinguished Professor Emeritus at Texas Tech University HSC.
Image: Public Domain
