Deconstructing the CDC's superbug death estimates

ATLANTA, Ga. – In 2013, the U.S. Centers for Disease Control and Prevention released estimates of how many people in the country die every year from antibiotic resistant infections: 23,000. The agency estimates that an additional 15,000 die annually from Clostridium difficile, an infection linked to long-term antibiotic use.

The estimates, the agency said at the time, provided the "first snapshot of the burden and threats posed by antibiotic-resistant germs having the most impact on human health."

Since then, the estimates have been cited by untold numbers of media outlets and scholarly reports. A Google search for the estimates can yield nearly 100,000 hits. Reuters took a close look at how the agency arrived at its numbers and made a surprising discovery: They are based on so little hard data that they could be off by more than 30 percent - more than 10,000 people - in either direction.

The statistical uncertainty is a byproduct of the nation's lack of a unified surveillance system. No one at the state or federal level knows how many people are dying from drug-resistant infections.

Absent hard numbers, the agency turned to studies that rely on statistical sampling, whereby a subset of the population is studied and the results are extrapolated to cover the entire country.

Statistical sampling is common. Precision depends on the size of the sample. The larger the sample, the greater the precision. Reuters found that the CDC estimates are undermined by small sample sizes, old data and information from a few geographic areas.

For example, the CDC estimate of 15,000 deaths annually from C. difficile was based on monitoring of 88 inpatient and 33 outpatient laboratories in 10 areas across the country in 2011.

The 10 areas, part of the agency's Emerging Infections Program (EIP), represented about 3.6 percent of the nation's population at the time. Under the EIP, a handful of states are given federal dollars to closely monitor infectious diseases, generally in a few counties. They do not include population centers like New York City, Southern California, Chicago or Boston, or any locations in Texas or Florida.

Infectious diseases can occur at substantially different rates in different areas. For example, the study on which the estimates are based, titled "Burden of Clostridium difficile Infection in the United States" and published last year in the New England Journal of Medicine, found that the incidence of hospital-acquired C. difficile was 47.3 per 100,000 people in Klamath County, Oregon, while it was 159.1 per 100,000 in New Haven, Connecticut.

Because of statistical uncertainty, the estimate of deaths from C. difficile could be anywhere from 7,600 to 20,000, Reuters found.

In many components of the CDC estimates, the agency relied on even less information. One example is the deadly carbapenem-resistant Enterobacteriaceae (CRE), which public health officials describe as a nightmare bug with high mortality.

Not a single actual CRE death went into the estimates.

Instead, for CRE and five other infections, the agency turned to a study in which researchers surveyed 183 hospitals within the EIP program. The survey identified 504 hospital-acquired infections during a single year, 2011.

Based on those 504 infections, researchers extrapolated that there were 721,800 such infections nationally. But because of the small sample size, the estimate could be anywhere from 214,700 to 1.4 million.

CDC researchers then used the study, "Multistate Point-Prevalence Survey of Health Care-Associated Infections," published in the New England Journal of Medicine, to estimate how many of the 721,800 infections were Klebsiella species or Escherichia coli, two of the more common types of bacteria that can become resistant to the carbapenem class of antibiotics.

Then the CDC used data reported by about 2,000 hospitals to its National Hospital Safety Network during 2009-2010 to determine what percentage of the infections were drug-resistant.

The agency estimated 6.5 percent of people with those drug-resistant infections die, based on a single study of a Chicago teaching hospital in 2000 in which 34 people died. Applying that percentage, they came up with their estimate of 600 deaths annually from CRE. Reuters found that using the CDC's methodology, it could be twice that - 1,200 - or as few as 180.