Oregon Study: Medicaid 'Had No Significant Effect' On Health Outcomes vs. Being Uninsured

For three years, an incredibly nerdy—but consequential—debate has raged among health policy researchers regarding Medicaid, America’s government-run health-care program for the poor. Piles of studies have shown that people on Medicaid have health outcomes that are no better, and often worse, than those with no insurance at all. But supporters of Obamacare were cheered in 2011 when a lone study, out of Oregon, purported to show that Medicaid was better than being uninsured. Yesterday, however, the authors of the Oregon study published their updated, two-year results, finding that Medicaid "generated no significant improvement in measured physical health outcomes." The result calls into question the $450 billion a year we spend on Medicaid, and the fact that Obamacare throws 11 million more Americans into this broken program.

For several reasons, I am especially interested in the topic of Medicaid’s poor health outcomes. For one thing, I believe that the moral legitimacy of any approach to health reform comes from whether or not it truly helps the poor, in a manner that is also respectful to the middle-income taxpayers who are struggling to pay their own bills.

Another reason is that I happen to have a lot of experience in analyzing randomized, controlled trials; it’s a big part of what professional investors do when they analyze pharmaceutical, biotechnology, and medical-device companies. Over the course of my career, I’ve looked at hundreds of these trials, in an effort to see whether or not the market was underappreciating or overhyping new therapies. So let’s break down the Oregon Medicaid study the way a professional would break down the clinical trial for a new cholesterol or diabetes drug.

The Oregon study measured common health outcomes

The first thing to look for in any clinical trial is the trial’s primary endpoint—the key measurement that the trial sets out to test, as an indicator of the prospective value of the treatment over the control arm. In this case, the authors of the Oregon study were comparing enrollment in Medicaid to remaining uninsured. What they wanted to measure, in an objective manner, was: did Medicaid improve the health of its enrollees?

In large, phase III clinical trials for new drugs to treat high cholesterol or high blood pressure, study sponsors are supposed to show that their new drugs reduce the incidence of death, relative to the control group, or the incidence of "major adverse cardiac events" like heart attacks, stroke, coronary arterial occlusion, or death. In diabetes trials, drugs are supposed to demonstrate a reduction in glycated hemoglobin, or HbA1c. (Glycated hemoglobin is considered to be the most reliable indicator of persistently high blood sugar.)

For example, in April, Eli Lilly (NYSE:LLY) announced the results of a trial of a new diabetes drug, dulaglutide, which demonstrated a statistically significant reduction in HbA1c relative to an insulin-based comparator, over the course of 26 weeks of treatment. A "statistically significant" benefit is traditionally defined as one that has less than a 5 percent chance of being statistical noise. (Statisticians describe this as having a "p value" of less than 0.05.)

The Oregon study didn’t have a "primary endpoint" per se, in the way that traditional clinical trials do. But the authors did prospectively seek to examine several objective health outcomes: (1) elevated blood pressure; (2) high cholesterol; (3) elevated HbA1c levels; and (4) long-term cardiovascular risk, as measured by the Framingham risk score.

Other than HbA1c, none of these measurements would cut the mustard in an FDA-approved clinical trial, for which harder endpoints like death and heart attack would be required. But given that the Oregon study is one of the very few randomized, controlled studies in public policy, we’ll let that slide.

The Oregon Medicaid population was partially self-selected

The next thing to look at is whether or not the Oregon study was designed and executed in such a way as to limit the introduction of bias. Unlike the typical clinical trial, the Oregon study wasn’t blinded: That is to say, for obvious reasons, the people who were on Medicaid knew they were on Medicaid, and the people who were uninsured knew they were uninsured. In clinical trials, you try to prevent both the doctor and the patient from knowing whether the patient received the test drug or the placebo, in order to ensure an objective result. In the case of the Oregon study, that bias is still there, and could favor the Medicaid arm of the study over the uninsured control arm.

The next thing to look at is whether nor not there was any bias in the way patients were enrolled in the study. Were the people who enrolled in the Medicaid portion of the study fundamentally similar to those in the uninsured arm, or were their differences? Here the results are mixed. On certain metrics, like age and ethnicity, the uninsured and Medicaid groups look similar.

However, the Oregon study authors appear to have only measured the baseline health status of the uninsured group, not the Medicaid group. "The effect of Medicaid coverage was estimated with the use of two-stage least-squares instrumental-variable regression," the authors write, instead of by using an actual analysis of how a specific Medicaid patient did from the beginning of the study to the end. So we don’t know if the average patient in the Medicaid arm had a higher or lower cholesterol level, say, relative to the average uninsured patient at the beginning of the study. That would never fly in a real clinical trial.

This potential difference between the Medicaid and the uninsured groups is very important. Of the 35,169 Oregonians who "won" the lottery to gain enrollment in Medicaid, only about 30 percent actually enrolled. Indeed, only 60 percent of those who were selected bothered to fill out the forms necessary to sign up for the benefits—which tells you a bit about how uninsured Oregonians perceive the Medicaid program.

The fact that only 60 percent filled out the forms introduces a bias into the studied Medicaid population, because the subpopulation that signs up for benefits is more likely to need treatment, and benefit from treatment, than the subpopulation that doesn’t. The Oregon authors attempt to adjust their analysis to address this, but we don’t have the raw numbers to see if there was a difference in, say, baseline cholesterol or blood sugar levels between the control group and the Medicaid group. In the traditional clinical trial world, this would be considered unacceptable.

Oregon’s Medicaid program is not representative of the U.S. broadly

The last thing we’ll look at, before diving into the study results, is the fact that it was conducted in Oregon. Specifically, the patients whose health outcomes were reviewed came from Oregon’s largest city, Portland, because this limited geographic area made it logistically easier for the study authors to interview the patients. Portland, being a city, isn’t necessarily indicative of how Medicaid performs in more rural areas, though we don’t know which way that would bias the study.

The Oregon Medicaid population is more white (+15 percent) and less black (-15 percent) than the U.S. uninsured population. I’m not aware of any empirical data on the degree to which ethnicity plays a role in the health outcome benefits of insurance, but it’s worth keeping in mind.

One important factor worth noting is that Oregon’s Medicaid program is in substantially better shape than that of the average U.S. state. In Oregon, Medicaid pays primary care physicians approximately 62 percent of what private insurers pay. That compares to the national average of 52 percent; a number of large blue states pay less than 40 percent.

Because Oregon’s Medicaid program pays more, the state’s Medicaid beneficiaries have relatively better access to doctors. While 21 percent of Oregon physicians won’t take new Medicaid patients—an unacceptably high number—the national average is even worse: 31 percent.

This is perhaps the most significant bias in the Oregon Medicaid study. Better access to physicians should lead to better outcomes for Oregon Medicaid beneficiaries than we would likely see in most other states.

The data: No statistical benefit in health outcomes

So, what did the Oregon study authors find? They found no statistically significant difference in elevated blood pressure (1.33 percent less incidence in Medicaid vs. control, p=0.65); high cholesterol (2.43 percent less than control, p=0.37); high HbA1c (0.93 percent less, p=0.61); or Framingham risk score (0.21 percent less than control, p=0.76). According to the p values, the blood pressure result has a 65 percent chance, the cholesterol result a 37 percent chance, the HbA1c result a 61 percent chance, and the Framingham score a 76 percent chance of being statistical noise. Again, statistical significance requires a p value of less than 0.05.

As the authors put it, "Medicaid coverage had no significant effect on the prevalence or diagnosis of [high blood pressure] or high cholesterol levels or on the use of medication for these conditions. It increased the probability of a diagnosis of diabetes, but it had no significant effect on the prevalence of measured glycated hemoglobin levels." The study did a significant increase in the diagnosis and treatment of depression, but the authors did not measure a depression-based clinical outcome, such as improvement in the Hamilton Depression Scale.

And all of that, despite the fact that the study had many biasing factors working in Medicaid’s favor: most notably, the fact that Oregon’s Medicaid program pays doctors better; and also that the Medicaid enrollees were sicker, and therefore more likely to benefit from medical care than the control arm.

Where the study did show significant differences between Medicaid and the uninsured was in spending (Medicaid patients spent an average of $1,172 more than uninsured patients); and utilization of health-care services (which drove the spending). Some of that utilization was a good thing, such as an increase in cholesterol screening. But it didn’t result in better cholesterol health outcomes.

In addition, the study showed a benefit in "reduced financial strain," an unsurprising result, given that the Medicaid law strictly limits the degree to which enrollees can pay for their own care. But, again, that reduced financial strain didn’t result in better health outcomes. And if relieving financial strain is all we are trying to do, we’d be better off giving poor people the cash and letting them spend it how they choose.

In 2011, media hyped flimsy early results from the Oregon study

I’ve been writing about Medicaid’s poor health outcomes for years, pretty much since I first started writing about health policy. I figured that liberals would be just as appalled as I am that Medicaid does so poorly. Instead, to my surprise, I found that otherwise careful liberal health economists would put themselves through all sorts of empirical contortions in order to argue that Medicaid was doing just fine. Because Obamacare relies so heavily on Medicaid to expand health insurance coverage, pointing out Medicaid’s flaws meant pointing out a central flaw with Obamacare.

When the initial results of the Oregon study came out in July 2011, they were trumpeted throughout the media as proving, once and for all, that Medicaid improved health outcomes. "Amazing Fact! Science Proves Health Insurance Works," read one headline. "A new, rigorous study from Oregon confirms that Medicaid does, indeed, save lives," wrote Matthew Yglesias, despite the fact that the study said nothing of the sort. "What we found in a nutshell is that having Medicaid makes a big difference in people’s lives," crowed study co-author Amy Finkelstein, who went on to win a prestigious economic prize, the John Bates Clark Medal, in part because of her work in Oregon.

And just what did the Oregon study show in 2011, at the one-year mark? Actually it showed no difference in health outcomes, such as the ones described above. But the study did find that people on Medicaid felt better about their health and their financial situations, in a subjective survey of enrollees conducted by the authors. These important details were lost on those who hyped the study as proof that Obamacare’s Medicaid expansion was the right thing to do.

The health policy world had been looking forward to seeing the second-year results of the study in July 2012, given how much attention they had gotten the first time around. But 2012 came and went, without any word as to how the study was going. Finally, 22 months after the one-year results were published, the authors published the second-year data. It’s unclear why it took them so long, especially given the fact that state legislatures have been struggling with whether or not to expand their Medicaid programs over the last four months.

Addressing the progressive pro-Oregon counterarguments

In the hours since the Oregon study was published in the New England Journal of Medicine, a number of talking points have emerged from those who insist that Medicaid must be making a significant difference in health outcomes. Let’s go through them.

1. It’s only been two years. Give Medicaid time. Perhaps in 5, 10, or 20 years, Medicaid will show a benefit relative to being uninsured. And that possibility can’t be ruled out. But Obamacare spends $750 billion over the next ten years to expand the Medicaid program. Is it ethical to take all that money out of the pockets of middle-class taxpayers to pay for a program that hasn’t been shown to work? Plus, successful clinical trials of new drugs routinely show a benefit on such metrics over a 1- or 2-year period. And we spend a lot less on new drugs than we spend on Medicaid.
2. The study was too small to show a significant effect. Balderdash. The study examined the health outcomes of 12,229 Oregonians. In trials of drugs for high cholesterol, high blood pressure, and diabetes, such sample sizes are nearly always adequate for demonstrating whether or not the treatment is beneficial. If Medicaid were a new medicine applying for approval from the Food and Drug Administration, it would be summarily rejected.
3. The results weren’t statistically significant, but some of the raw numbers favored Medicaid. Again, if you made that argument in front of the FDA, you’d be laughed out of the room. It’s true that some of the raw numbers favored Medicaid. But some of them didn’t. The probability of statistical noise, as I discussed above, was quite high. Most importantly, the Oregon results on health outcomes are consistent with a mountain of clinical evidence showing that Medicaid makes no meaningful difference, at best, compared to being uninsured.
4. Medicaid did a great job diagnosing and treating depression! That’s great, though again, we don’t know if that made a difference on actual clinical outcome measures of depression. And remember that we spend $450 billion a year on Medicaid. Nearly all of the most highly prescribed drugs for high cholesterol, high blood pressure, diabetes, and depression are generic. If we can’t make people happy for less than $450 billion a year, we’re doing something wrong.
5. Medicaid reduced financial hardship for the poor, by protecting them against catastrophic health risks. Wonderful, but we could have achieved the same outcome for a fraction of the price, by adopting the plan proposed by Florida’s Will Weatherford and Richard Corcoran: Offering low-income Americans a subsidy with which to purchase catastrophic coverage on the open market. That plan was foiled by people—including Republicans—who insisted on expanding Medicaid instead.

I keep mentioning the cost of the Medicaid program. But I want to make clear that I’m not opposed to spending that sum of money on health care for the poor. What I’m opposed to is wasting that sum of money on health care for the poor. There are so many market-based alternatives to Medicaid, alternatives that would offer uninsured, low-income Americans the opportunity to see the doctor of their choice, and gain access to high-quality, private-sector health care.

Let’s build a new health program for low-income Americans, one that pays primary care physicians $150 a month to see each patient, whether they are healthy or sick. That’s what so-called "concierge doctors" charge, and it would give Medicaid patients what they really need: first-class primary care physicians to manage their chronic cardiovascular and metabolic conditions. Then throw on top of that a $2,000-a-year catastrophic plan to protect the poor against financial ruin. The total annual cost of such a program would be $3,800 per person, 37 percent less than what Obamacare’s Medicaid expansion costs. Hell, put the entire country on that kind of plan, along with giving people the opportunity to use health savings accounts to cover the rest.

There are still a number of states that are considering whether or not to expand Medicaid. The pro-expansion argument usually revolves around the piles of cash—from taxpayers in other states—that will redound to the particular state in question. But every state will still have to spend a considerable sum of its own money to sign onto the Medicaid expansion. At a time when middle-income Americans are themselves being squeezed by Obamacare’s hikes to health insurance premiums, it is positively unethical to take money from them in order to fund a broken program that doesn’t improve health outcomes.

Progressives have long enjoyed wielding the straw man. "If you oppose expanding Medicaid," they say, "you oppose health care for the poor. Plain and simple." But the truth is, if you support expanding Medicaid, you’re doubling down on a broken program, one that shuts the door on real reforms that could provide quality health care to those who most need it.

This piece originally appeared in Forbes

This piece originally appeared in Forbes