Economists Claim Medicaid's Health Outcomes are Great. Or Do They?

The big news last week in the health policy world was a study published by three Harvard economists, arguing that Medicaid expansions were "significantly associated with reduced mortality." Their paper comes in the wake of dozens of clinical studies showing that Medicaid beneficiaries suffer from very poor health outcomes relative to those with private insurance, and in some cases fare worse than those with no insurance at all. Now, some media outlets are using the Harvard paper to browbeat states into expanding their Medicaid programs. But the Harvard paper, while interesting, has many flaws, flaws that call its expansive conclusions into question.

First, let’s talk about the good things in the Harvard paper, which appeared in last week’s issue of the New England Journal of Medicine. It was authored by three credentialed economists at the Harvard School of Public Health: Ben Sommers, Kate Baicker, and Arnold Epstein. (Sommers has taken leave from Harvard to work for the Obama administration.)

I respect them all, and have cited their work in the past. Sommers and Epstein, for example, have done some solid research on the Medicaid woodwork problem that poses hidden costs to states under Obamacare. And their latest examination of Medicaid’s performance is a useful contribution to the literature.

"Our results offer new evidence that the expansion of Medicaid coverage may reduce mortality among adults," the authors conclude. "Policymakers should be aware that major changes in Medicaid—either expansions or reductions in coverage—may have significant effects on the health of vulnerable populations."

An accompanying editorial from two liberal law professors is less subtle: "The question of whether the states will expand Medicaid, therefore, is not just a question of politics; it is a question of life, health, and death." The New York Times went even further, declaring it a "canard" that people question Medicaid’s health outcomes.

The Harvard authors are clearly aware that their work has policy implications, and welcome its use to advance the expansion of Medicaid. So, let’s examine how well their analysis backs that up.

Medicaid mortality increased in one state and decreased in two states

The authors looked at three states that expanded their Medicaid programs to childless adults—Maine, Arizona, and New York—and compared them to four neighboring states that did not—New Hampsire, Nevada and New Mexico, and Pennsylvania, over the years 1997 to 2007.

The authors found that Maine’s mortality rate increased relative to New Hampshire’s (by 13.4 deaths per 100,000); Arizona’s decreased relative to Nevada and New Mexico (by 10.2/100,000); and New York’s mortality rate declined relative to Pennsylvania (by 22.2/100,000).

In other words, the authors found that Medicaid was associated with an increase in mortality in Maine, but a decrease in mortality in Arizona and New York. Indeed, according to the authors, "single-state analyses showed [statistically] significant effects only in the largest state, New York." Based on these three conflicting results, we are supposed to declare the debate over, and definitively conclude that Medicaid improves health outcomes?

Comparing New York to Pennsylvania suffers from many confounding factors

Also, it must be noted that no two states are alike. In particular, New York—the state that appeared to show the largest benefit from expanding Medicaid—was compared to Pennsylvania, a neighboring state with a substantially different population and health-care system. Pennsylvania’s poverty rate of 11.5 percent is meaningfully lower than New York’s 14.1 percent.

Most notably, New York has a much larger immigrant population. In 2000, 38 percent of New Yorkers were from ethnic or racial minorities, compared to only 16 percent of Pennsylvanians. Hispanics and Asians comprised 21 percent of the New York population, compared to 5 percent of Pennsylvanians. This last bit is especially relevant, because recent immigrants and non-citizens are far more likely to be uninsured than U.S. citizens of any race. According to the U.S. Census, 45 percent of non-citizens are uninsured, compared to 20 percent of foreign-born naturalized citizens, and 14 percent of native-born residents.

Why are immigrants far more likely to be uninsured than native-born Americans, to a degree far greater than would be expected by differences in income? It may be that first-generation immigrants are less accustomed to Western, insurance-based health care systems.

Indeed, the Harvard authors found a profound difference between Medicaid’s performance in white and non-white populations. Based on their methodology, Medicaid improved mortality in the white population by 14 deaths per 100,000, compared to 41 per 100,000 in the non-white population.

In other words, if the authors had been able to find a more comparable state to New York, they might not have seen a significant improvement in mortality in New York relative to the comparator. And if that had been true, the study may have shown no significant effect of Medicaid on mortality rates.

But there was no suitable comparator to New York. Connecticut, New Jersey, and Vermont all had poverty rates in the 8 to 9 percent range, far lower than New York’s 14 percent. New York’s neighbors all have smaller minority populations than New York does.

In other words, the comparison that drove the authors’ key statistical finding, New York, was the one with the least suitable comparator state, and the most confounding problems.

A more scientific approach would have been to eliminate the New York results entirely, and focus only on Maine and Arizona. But if the Harvard authors had done that, they would have ended up with a completely different result: that Medicaid had no impact on mortality, compared to having no insurance at all. And that wouldn’t have been as satisfying.

There are other important differences between states

There are other confounding factors that could explain why the authors found differences between the states’ Medicaid programs. Were there differences in the baseline health statuses of the populations of the various states? How different were their crime rates? What about other changes to Medicaid unrelated to eligibility (such as cost-sharing and reimbursement)? Did certain hospitals in certain states improve their clinical practices in substantial ways? None of these questions appear to have been raised by the Harvard authors.

There is another Medicaid outcomes study being conducted in Oregon that addresses some of these inter-state issues. I will have more to say about that study in a future post.

Economists look at county-level, not individual patient data

One thing that’s remarkable about the health outcome studies done by economists is that they almost never look at the outcomes of actual individual patients. The Harvard study used county-level data from the Centers for Disease Control, "totaling 68,012 observations specific to an age group, race, sex, year, and county."

Contrast this approach to that of the landmark Medicaid outcomes study from the University of Virginia, which examined 893,658 individual surgical cases from the Nationwide Inpatient Sample database, and found that Medicaid’s health outcomes were worse than those of the uninsured. Which of these datasets would you consider to be more credible?

The study doesn’t compare Medicaid to private insurance

Most importantly of all, the Harvard study doesn’t answer the real policy question that divides conservatives and liberals: should we shove the poor into government-run health care programs, or instead give them access to high-quality private insurance? John McCain, you’ll recall, proposed universal private health insurance in 2008, and was criticized for doing so by Barack Obama.

Whatever you believe about the relative benefits of Medicaid to having no insurance at all, there is no serious debate about the superiority of private insurance to Medicaid. "For certain populations and particularly in certain states, [Medicaid] is unambiguously inferior to private insurance and to Medicare," wrote Jonathan Cohn, one of Obamacare’s most influential advocates, in the New Republic in 2011.

Given the fact that private insurance’s superiority in health outcomes is undisputed, and given that the debate in Washington is about whether or not Medicaid should be government-run or privatized, you’d think that academic economists would be racing to do studies comparing Medicaid to private plans. I’m not sure why, but such studies have been few and far between. Indeed, the Harvard study makes no mention of a significant problem with Medicaid expansions: how they crowd out private insurers who previously covered the poor.

Interesting but flawed papers, like last week’s, are a useful contribution to the debate. But even if we ignore those flaws, and grant the Harvard authors the benefit of the doubt—that Medicaid provides, at great taxpayer cost, a marginal benefit—we ought not to rest until the poorest Americans have access to the same, high-quality private insurance that the rest of us enjoy.

Obamacare doesn’t give Medicaid enrollees access to high-quality, private insurance. Instead, the law doubles down on the existing, broken system, aiming to add 17 million more Americans to Medicaid’s ragged rolls. We can do better.

This piece originally appeared in National Review Online

This piece originally appeared in National Review Online