Pulling It Together: Predicting the Future

A fair amount of attention was given recently to projections made by the Chief Actuary of the Centers for Medicare and Medicaid Services (CMS) about the new health reform law, and how they compare to previous estimates by the Congressional Budget Office (CBO). No doubt the various projections will be grist for claims made in the upcoming political season, so it is important to be clear about the differences between the two estimates and to keep in mind what this kind of statistical modeling does and does not do.

The CMS Actuary’s statement that national health spending will go up under reform has attracted the most attention. It should be no surprise that spending will go up. The law will significantly increase the number of people who have health coverage — by 32 million according to the CBO, and 34 million by the Actuary’s reckoning. These are people who would not have had coverage otherwise, and will use more health services as a result. The Actuary projects that net national health spending will go up by an average of less than one percent per year over the next ten years, a small increase in the context of overall health spending and an amount that could be lost in the shuffle of the next update of national health spending projections.

From the perspective of the federal budget, the law was designed so that revenues and savings will offset new spending. Critics dispute the estimates, and some don’t like the specific revenue and savings measures contained in the legislation. But, the CBO says that the revenue and savings measures in the law will more than offset its costs, leading to a reduction in the federal budget deficit over time. (Technically, the CBO takes its revenue estimates from the Joint Committee on Taxation and incorporates them into its model.)

Unlike the CBO, however, the CMS Actuary does not include estimates of revenues, and thus he reaches no conclusion about the impact on the deficit. Conversely, the CBO does not estimate the effect on national health spending. In other words, the CBO and the Actuary do not estimate the same things. Essentially, when you put the two analyses together, they are saying that the law covers 32-34 million additional people, and it does so with essentially the same amount we would spend anyway as a nation and in the federal budget, after taking into account its revenue and cost saving measures.

These specific projections were the focus of recent attention, but how should we think about the role of modeling in health reform more generally? In the health reform debate “micro-simulation models” were used to project the impact of legislation relative to the status quo. Based on existing studies and economic logic, assumptions are made to predict how people and employers will react to all the various changes in the law — from expanded Medicaid eligibility, to tax credits to help pay for private insurance, to the insurance reform and the individual and employer mandates. These changes in behavior are then aggregated to predict the overall impact of the law.

The most fundamental assumption the CBO and the Actuary must make when they do their projections is that the law will be implemented as it was written. Particularly in the context in which they are doing their estimates, this seems reasonable enough, as they have no way of predicting how the law might morph or change in the future. It is, however, a basic constraint the rest of us should be aware of in interpreting official projections. A complex law like this could change substantially in implementation and initial modeling assumptions could change depending on how governors and states react and handle their responsibilities (no doubt there will be great variation), how the private sector reacts, what health care providers do, and most of all how the public responds once the law is implemented. Of course the law itself could also be modified in the future.

Certainly, with all its moving parts affecting so many aspects of the health care system and the economy, simulating the effects of the health reform law is infinitely more complex than modeling the likely impact of, say, the last big change in health policy, the Medicare prescription drug law. That was complex enough, but involved a single change in one program affecting only Medicare beneficiaries. As it turned out, Medicare spending under the prescription drug law was significantly lower than originally projected by the CBO.  There are many who believe that the CBO is decidedly conservative when it comes to scoring the potential savings that will come from the delivery and payment reforms in the health reform law, as well as its prevention components. Where there is no research or experience to base savings on, a conservative approach is reasonable given the CBO’s vital role in the legislative process. But, it also makes it difficult to project and count on savings for things which have not been tried before or have been tried on a small scale in the private sector but not evaluated, such as some of those delivery and payment reforms.

Critics of the law, of course, are more pessimistic and take the view that cost savings will not materialize as envisioned and that some elements of the law scored as reducing costs, such as the Cadillac plan tax, are unlikely to even be implemented. What seems likely in the real world is that some elements of the law will work better than envisioned and some not as well, and when that happens there will be pressure to make adjustments and improvements. That is the lesson from Massachusetts, which led with coverage and then turned to the need to deal with problems such as cost containment and primary care supply as those issues emerged in implementation. But the kind of process which has played out in Massachusetts, one in which policymakers learn and adapt based on real world experience, is not easily modeled in advance.

Also important are the assumptions that modelers must make to ultimately arrive at their predictions. For example, the CBO predicts that 3.9 million people will ultimately pay penalties under the law rather than buy insurance. That estimate, presumably, is based on an analysis of the costs and benefits of buying a policy versus paying the penalty. But can that number really be known or estimated with precision? How dependent are the projections on how the economy is doing at the time and how rapidly people’s wages are rising and what they can afford? To what extent do people value insurance protection for their families in ways that are difficult to quantify?  If policies end up being unaffordable for many people in the real world, will the penalty be relaxed? Or the subsidies improved? Or more people exempted from the mandate to buy insurance? All kinds of reasonable scenarios are possible. There are many examples in the models where reasonable and informed assumptions are made based on published and unpublished studies (or economic logic). But the evidence is not always ironclad, and other assumptions might have been made instead, especially if you expect, as modelers cannot, that other more real world variables will enter the equation.

There is also a lot the CBO and the Actuary do not attempt to estimate, because their models are designed to address specific questions about the impact of federal legislation on the federal budget or national health spending. For example, they do not estimate the impact of the legislation on individual states, which is information in high demand right now as states gear up to take on their substantial implementation responsibilities. And they do not estimate the impact of the legislation on particular population groups, whether it’s the chronically ill or the middle class or minorities or people in rural areas. These things can be modeled but are outside of the purview of the analyses the CBO and the Actuary take on.

In the end the policymaking process is much better off because of the presence of honest brokers trying to use the best economic tools available to make projections about what policy changes will mean. The benefit is not only that it introduces budget discipline into the policymaking process on Capitol Hill — which would otherwise be entirely about differences in policy and about politics and ideology — but also that it introduces facts, data and objectivity into a process that is often about other things. However, in a partisan and conflict-oriented process that elevates modeling from an analytic tool to the status of a referee, it will be important in the coming political season to watch out for statements that take estimates from the CBO and the Actuary out of context or that imbue them with a precision and clairvoyance modeling does not have.