Measuring Changes in Insurance Coverage Under the Affordable Care Act

The first open enrollment period under the Affordable Care Act (ACA) has come to an end, and many are looking for ways to assess the law and its implementation thus far. Of particular interest is how many people who were previously uninsured took up new coverage options, but questions about whether people with insurance changed their type of coverage also are receiving attention. Changes in employer-based insurance are particularly important because so many people get their coverage in this way.

The ACA provides significant new coverage options for people, particularly for those with lower incomes or problems with their health. The scope of the reforms, and the intense political controversy surrounding their approach and implementation, has fueled an intense demand for data about their effectiveness. Unfortunately, the information needed to adequately understand enrollment changes across private and public coverage sources will not be available for many months.

Most of what we know about who has health insurance and what type of coverage they have comes from large federal surveys, which provide estimates of the number of people enrolled in different types of coverage, including those with no coverage, along with information about their household demographics and incomes. These surveys can be used to track changes in coverage for different types of people over time. The main advantages of these surveys are their large size and their sophisticated sampling and interviewing techniques, which allow detailed analysis of coverage and coverage changes for people in different demographic and income groups. Further, many federal surveys enable analysis at the state level for at least some states, which is important because ACA implementation (e.g., the availability of expanded Medicaid coverage or the existence of a state-operated Marketplace) will vary greatly across states. Their main disadvantage is that they do not provide rapid turnaround. The data needed to evaluate the coverage changes between 2013 and 2014 will not become available until 2015.

In the interim, people will need to look to other sources of information. One is administrative data, such as the number of people who have enrolled through new health insurance marketplaces or the number of people who have enrolled in Medicaid. A problem with this type of information is that we do not know the enrollees’ coverage status prior to enrollment: did they have insurance before and, if so, what type? It also is difficult to distinguish new enrollment from coverage changes that would have occurred in the absence of the law, since people’s job status and income change throughout the year. Also, there is currently no administrative data that covers the individual market outside of the marketplaces or any data that captures the entire employer market, so there is no information to help us understand how things are changing in the market that covers the majority of nonelderly people. And most importantly, because people without insurance are not enrolled in anything, they cannot be counted in administrative data. Administrative data can provide clues about where to look for changes, but we do not have administrative systems that provide information about changes across types of coverage or changes in the number of people without insurance.

The second interim source of information about health coverage is surveys by private entities, which ask about health insurance by type of coverage and track changes over time (or at least between a few points in time). This means that they provide some opportunity to look at changes across type of coverage as well as changes in the number of people who have any coverage. The main advantage of these surveys is their rapid turn around: indeed, several private surveys already have released findings and show that the number of people without health insurance has fallen between late 2013 and early 2014. Differences in approach and sampling mean that these surveys have different strengths and weaknesses and that their results may not be consistent or comparable.

Below we discuss the details and timing of some of the private and federal surveys that will be used to look at how coverage has changed due to the ACA. Different surveys offer different information and insight into coverage under the ACA, and we discuss the contribution and challenges in each type of effort (see Textbox 1).

Private Surveys

Surveys conducted by private organizations have provided the first look at coverage changes under the ACA. Private surveys have been used to analyze a number of aspects of the early implementation, including public knowledge and attitudes, changes in the share of people with insurance, and the prior insurance status of early Marketplace enrollees. Many of these surveys are fielded at regular intervals with the goal of understanding how things change over time. In some cases, such as the Gallup-Healthways Well-Being Index (“Gallup”), these are broad public opinion surveys that contain a few questions related to health insurance. In other cases, such as Urban Institute’s Health Reform Monitoring Survey (“HRMS”), the Rand Health Reform Opinion Survey (“RAND”) and our Kaiser California Uninsured Panel Survey, surveys were developed specifically to track ACA implementation. Others, like our Kaiser Health Tracking Poll, focus mainly on opinions, knowledge and early experiences rather than on measuring changes in the share of people with health insurance or non-group coverage.

The main advantage of these private, population-based surveys is their rapidly available results. As opposed to the large federal surveys that will release initial indications many months following the close of open enrollment, we already have some estimates of coverage changes from private surveys, with updates likely in the near future. Several of these surveys had established baseline coverage estimates prior to the beginning of the 2014 open enrollment, making comparisons before and after implementation possible with mostly consistent questions and approaches.

There are, however, several potential limitations to many of the private surveys that need to be kept in mind when interpreting their results.⁴ One is that the populations that we most want to know about account for a very small share of the overall population and therefore a small share of the sample in any population-based survey. These populations include, for example: the uninsured in states that expand Medicaid compared to states that do not, those with nongroup coverage before ACA implementation who switch to coverage through the Marketplace, and those uninsured who are now newly eligible for Medicaid or for subsidies in exchanges. Most private surveys have relatively small samples for these specific populations, which means that the estimates for these groups are imprecise (that is, estimates have large confidence intervals) and subject to meaningful volatility. Limited precision makes it hard to detect and compare statistically significant changes, particularly for subgroups (e.g., by race, former insurance status, or location).

A second challenge is that asking the questions to determine individuals’ prior and current insurance status in a way the respondents can answer accurately is very complicated and time consuming in a survey. Compared to the large federal surveys, which generally devote a number of questions to identifying type of coverage, private surveys often make due with simpler approaches. People are often confused about the type of coverage they have and may answer wrongly or inconsistently when just asked to pick from a list of coverage sources. Several of the more prominent private surveys also have changed their coverage questions recently, which make interpreting changes over time more difficult.

As a result of these limitations, these private surveys are more likely to shed light on broader questions, such as changes in the overall number of people who have health insurance, than on narrower issues, such as changes by race, income, state, or type of coverage. And even though some of the private surveys manage to obtain a fairly large number of respondents, they still may have a relatively large uncertainty around their estimates, which means that they may be more reliable for pointing to trends in the direction of change rather than providing precise measures of the actual coverage rates overall or by type of coverage.

The recent releases of estimates of coverage changes from several private surveys illustrate some of these issues. Results from Gallup, HRMS, and RAND all find that the number of people without health insurance fell during the initial months of 2014 as new coverage options under the ACA took effect.⁵ (Our own monthly tracking poll has shown a similar reduction in the number of adults uninsured at the beginning of 2014, though the survey was not designed to detect such changes, which are generally within the poll’s margin of error in any given month.)

The three surveys agree that the number of uninsured people has gone down, despite using very different sampling approaches: Gallup accumulates response from their daily tracking poll, which uses telephone interviews of a random sample of adults each day⁶; HRMS is based on interviews from successive samples of an internet panel, which was randomly selected⁷; and RAND is based on repeated interviewing of the same group of adults in an internet panel, a portion of which were randomly selected and a portion of which are from a convenience sample.⁸

The agreement among the surveys on the direction of change reinforces the overall result, although a closer look at their actual coverage estimates shows that some differences make interpreting the specific results somewhat difficult (see Table 1).

To take one example, both HRMS and Gallup showed a comparable (roughly 2 percentage point) reduction in the percentage of nonelderly adults without health insurance over the first several months of 2014, but their estimates of percentage of nonelderly adults who were or are uninsured are quite different.⁹ Focusing on the fourth quarter of 2013, Gallup reported that 17.1 percent of adults, including the elderly, were uninsured in the fourth quarter of 2013, which translates into about 20.5 percent of nonelderly adults.¹⁰ In contrast, HRMS’ fourth quarter 2013 estimate of uninsured adults is 17.5 percent of nonelderly adults.¹¹ So while both show that the share of uninsured adults fell, they are starting from fairly different places in their estimates of the share of nonelderly adults without health insurance. (We should note that the Gallup estimate is for 2014 is an average over a period, but that the point estimate continued to fall throughout the period, so that the estimate of percent of adults without health insurance in the second half of March was 14.5 percent).¹²

Another example is the difference between RAND and Gallup relating to changes in employer-based coverage. Again, both surveys found that more people had health insurance in the early part of 2014 compared to the fall of 2013. RAND, however, found a significant increase in the number of nonelderly adults covered by employer-based coverage while Gallup found little change.¹³ The differences between the two surveys may result from differences in approach, sampling, questions or just random variation. This issue will be worth watching as more survey findings are released.

Table 1: Early Results from Surveys of Coverage Under the ACA Conducted by Private Entities
	September 2013		March 2014		Change In Share of Uninsured, Adults Age 18-64	Gains In Coverage
Survey	Interview Dates	Percent Uninsured	Interview Dates	Percent Uninsured
Gallup-Healthways Well-Being Index	July 1 – Sept 30, 2013	18.0% adults age 18 and older	Jan 2- Mar 31, 2014	15.6% adults age 18 and older	-2.5 percentage points	N/A
Urban Institute’s Health Reform Monitoring Survey (HRMS)	Sept 1- Sept 30, 2013	17.9% adults age 18-64	Mar 1 – Mar 31, 2014	15.2% adults age 18-64	-2.7 percentage points	Gain in coverage for about 5.4M
RAND Health Reform Opinion Survey	Sept 1- Sept 30, 2013	20.5% adults age 18-64	Mar 1 – Mar 31, 2014	15.8% adults age 18-64	-4.7 percentage points	Gain in coverage for about 9.3M

Federal Surveys

Most of what we know about who has health insurance and the type of coverage they have comes from large, federal population surveys, such as the National Health Interview Survey (NHIS), the Annual Social and Economic Supplement (ASEC) of the Current Population Survey, and the Survey of Income and Program Participation (SIPP). These surveys collect demographic, economic, health coverage and other information from large samples of the population, which can be used to provide fairly complete pictures of how people are distributed into different types of coverage and how this distribution changes over time. Their main advantages are their large size, sophisticated sampling, and interviewing techniques (often in person). While health insurance coverage estimates differ somewhat across the different federal surveys, in part because they each have different questions and approaches, their results are generally consistent. When available, the information from these surveys will provide the most complete and reliable descriptions of how health insurance has changed as the ACA has been implemented.

The main disadvantage of the federal surveys is that they will not have results reflecting coverage after ACA implementation for many months. An additional challenge is that several of the main surveys are using new questions or approaches to understanding health insurance coverage in order to accommodate the new coverage options under the ACA. These changes will in general improve the ability to compare insurance coverage before and after full implementation of the ACA, but methodological changes may in some cases make it challenging to discern trends across the period leading up to full ACA implementation, from 2012 to 2013.

Following are descriptions of the major federal surveys that will provide information about how coverage has changed under the ACA.

Table 2: Availability of Post-ACA Health Insurance Coverage Data from Major Federal Surveys
Survey	Supports State-Level Analysis?	Post-ACA Data Availability
		Date Released:	Type of Data Available:	Reflects Coverage for Period:
National Health Interview Survey	Only through restricted data center and limited to 40 largest states	September 2014	Preliminary Q1 Data	At date of interview (Jan-March 2014)*
		December 2014	Preliminary Q2 Data	At date of interview (Jan-June 2014)*
		March 2015	Preliminary Q3 Data	At date of interview (Jan-Sept. 2014)*
		June 2015	Main 2014 Public Use File**	At date of interview (Jan-Dec. 2014)*
Behavioral Risk Factor Surveillance System	Yes	July 2015	Main 2014 Public Use File	At date of survey (Jan-Dec. 2014)
Annual Social and Economic Supplement	Yes, but for some analyses (such as insurance coverage rates) two years of data must be pooled	Spring-Summer 2015	New coverage questions from 2014 CPS ASEC	Monthly for all of 2013 up to date of interview (Feb-April 2014)
		September 2015	2015 CPS ASEC	Monthly for all of 2014 up to date of interview(Feb.-April 2015)
Survey of Income and Program Participation	Limited to 20 largest states	Spring 2016	2014 Panel Wave 2	Monthly for all of 2013 and all of 2014
American Community Survey	Yes	December 2015	Main 2014 Public Use File	At date of survey (Jan-Dec. 2014)
* Includes some information about prior coverage.** Imputed income file may not be available for several months.

National Health Interview Survey

The first-available federal survey that will have health insurance information covering at least some of the ACA’s 2014 open enrollment period is the National Health Interview Survey (NHIS). The NHIS is a national household survey of civilians living outside of institutions conducted by the National Center for Health Statistics, with contractual assistance from the US Census Bureau.¹⁴ The survey collects information throughout the year on a range of health topics (including health insurance status at the time of the interview) and on the income, employment, and other personal characteristics of respondents.

While full NHIS survey results for 2014 are not expected until June 2015 (see Table 2), preliminary data and reports are made available earlier through an early release program. In recent years, early release information for interviews conducted for the first quarter (January through March) of a year have been released in September of that year, with information for the first two quarters (January through June) released in December and information from the first three quarters (January through September) released the following March. The early release data has included estimates of the percent of people in different age groups who were uninsured at the time of the interview, the percent of each group uninsured for at least part of the year prior to the interview, and the percent of each group who have been uninsured for more than a year at the time of the interview. Estimates of the percent of each age group with public or private coverage also are typically made available, along with a limited set of demographic variables that can be used to look at coverage statistics for some subpopulations. Household income information deserves caution: approximately one-quarter of respondents’ family incomes are affected by an income imputation procedure that will not be implemented until a few months after the full file release in 2015. Additionally, poverty classification is limited to six categories: Under 100% of the federal poverty level (FPL), 100 to 138% FPL, greater than 138% up to 250% FPL, greater than 250% up to 400% FPL, above 400% FPL, and unknown.¹⁵

The first quarter early release data may provide some insight into the impact of the ACA open enrollment period, but the information will understate the full effect because most of the interviews were conducted in January, February, and early March, before the surge in enrollment at the end of March (see Textbox 1). The second quarter release will be more valuable because roughly half of the interviews will have occurred after the formal close of the open enrollment periods in the federal and state Marketplaces. Although enrollment opportunities in Medicaid (and even in Marketplaces in some circumstances) continue beyond March, interviews conducted in the second quarter are more likely to reflect the substantial enrollment activity that occurred at the end of March (with some ambiguity for people whose coverage had not yet become effective at the time of their interviews). Estimates from the early release data should be comparable to estimates from prior years, allowing for analysis of the change in the percentage of people uninsured as of the end of open enrollment. This will be the first available federal survey allowing for assessment of overall growth in the non-group market and also changes in employer coverage.  Although the preliminary quarterly files cannot be used to distinguish Medicare and military coverage from other government insurance sources, if restricted to the non-elderly, changes in public coverage should largely reflect changes in Medicaid and CHIP.

If the release schedule follows that of the past, the full data release for the 2014 NHIS should occur mid-summer in 2015, but full income information may not be available for several more months. The survey will provide estimates of coverage for both major public programs and private sources at the time of their interview. New questions ask whether coverage was obtained through healthcare.gov or a state exchange and whether the premium for that coverage is based on family income. People without health insurance are asked how long it has been since they last had coverage and why they lost their previous coverage. The main 2014 NHIS release will provide the first reasonably complete look at the first-year coverage effects of the ACA, although all of the first-quarter interviews will have been completed prior to the end of the open enrollment period. Since “month of interview” is included in the public release, earlier interviews can be examined separately or discarded, depending on the analysis aim. This survey is large enough to support analysis of subgroups, although analyses involving income will need to wait until the full income information is released later in 2015. The survey’s sample size is sufficient to support analysis of some of the larger states, but accessing these variables requires application to a Research Data Center. While insurance coverage is estimated at the point of the interview in the NHIS, a series of insurance transition questions will allow an assessment of whether people who report coverage through healthcare.gov or a state exchange were previously uninsured. Insurance coverage question wording remained consistent enough that annual trends across the 2013-2014 period, and to previous years, should be valid.

Behavioral Risk Factor Surveillance System

The Behavioral Risk Factor Surveillance System (BRFSS) is a nationwide telephone survey conducted by state health departments with assistance from the Centers for Disease Control and Prevention (CDC) to monitor health behavior and identify emerging health problems.¹⁶ Data are collected monthly in all states and some territories; over 400,000 interviews were conducted in 2012. While the survey focuses on health behavior, it also collects limited information on whether or not respondents have health insurance at the time of the interview. The survey does not collect information on the type of insurance that respondents have, so it is not a source of data for changes across types of coverage.¹⁷

While BRFSS has not historically been viewed as an authoritative source of information about coverage, its recent national estimates of people without coverage have tracked reasonably well to Census data. In 2011 and 2012, the BRFSS found that 21.3 and 20.4 of 18-64 year olds lacked insurance coverage compared to rates of 21.2 and 21.0 from the CPS ASEC over the same period.¹⁸ Data from BRFSS for 2014 are expected to be released in mid-2015 and should be a source of information about whether the number of people without coverage declined. BRFSS is designed to support state estimates, and with its large sample size, will be the first available information about changes in the number of adults with coverage at the state level.

Annual Social and Economic Supplement

Another large survey that is often a source of health coverage estimates is Annual Social and Economic Supplement (ASEC) to the Current Population Survey (often referred to as the “CPS”), which provides socioeconomic and demographic information, including health coverage status, for the non-institutionalized U.S. population. The CPS ASEC interviews over 85,000 households each year during the months of February, March or April, with findings and data released in September of the same year. The CPS ASEC is the most widely-used source for counts of the uninsured because it is timely, supports both national and state-level estimates, and provides detailed information on insurance coverage, income, employment, and other personal characteristics. The health coverage questions on the CPS ASEC have historically asked respondents whether they had coverage in the previous year, the source of that coverage (e.g., through an employer, a public program, or purchased directly), and whether individuals are covered in their own name or as a dependent on someone else’s policy. Although widely used, the Census Bureau acknowledges that health coverage is underreported in the survey. Technically, the CPS ASEC asks about coverage in the previous calendar year from the date of the interview, and the number of uninsured reported through the CPS ASEC represents those who were uninsured for all of the previous year. However, researchers believe that many people may instead report their coverage status at the time of the interview rather than for the previous year.¹⁹

Changes to the 2014 CPS ASEC survey (which will reflect data for 2013) should greatly reduce this confusion and provide much better information about health coverage. However, this improvement will both cause a one-time delay of the release of the full survey results and create a break in the trend of health insurance questions from the 2013 CPS ASEC (which reflects data for calendar year 2012). Previously, respondents were asked, for each major type of health insurance, whether or not they were covered at any time during the previous year by that type of coverage. Respondents who said no for each coverage type were subsequently asked to verify that they were not covered by any type of insurance. Starting with the 2014 survey, the survey asks respondents about their coverage at the time of the interview (February, March or April of 2014) and then asks additional questions about coverage in each month from the date of the interview back to January of 2013. This approach should provide clearer information about the type of coverage people have at each point in time and new information about how people’s coverage changes over the course of a year. In addition, the survey asks specifically about enrollment through Marketplaces and whether the premium for the coverage is subsidized based on family income. Other new questions ask respondents who are working but do not report having employment-based coverage whether their employer offers health insurance, whether they are eligible, and why they did not enroll.

Because of these and other changes to the survey²⁰, the 2014 CPS ASEC results will be released in three stages. The Fall 2014 release will report on coverage in 2013 and provide information similar to prior releases, looking at whether or not respondents had a type of coverage in 2013 or had no coverage. Information from the new questions, including coverage at the time of the interview and information about coverage in each month in 2013, will be released sometime in 2015, after the Census Bureau has time to analyze the results but before the complete 2015 CPS ASEC release in September of 2015. When available, the information from these new questions will allow analysis of how coverage during the first few months of 2014 compared to coverage by month in 2013, including providing estimates of those newly insured and those changing the type of coverage that they have. The survey is large enough to support some analyses at the state-level and by other subgroups like income category.

There will be some reasons for caution, however. Because many of the interviews will have been conducted before the formal open enrollment period ended on March 31, the survey will not capture the full effect of open enrollment. Further, it is unclear how people who were in the process of signing up for coverage at the time of the interview will answer the coverage question. For example, will people who have picked a plan but where coverage is effective the following month say that they were covered by the plan or say that they were uninsured at the time of the interview? Also, given the new question format, reported enrollment patterns may differ from those that we have previously seen from this survey. It is also unclear how consistent respondents will be in reporting coverage at the time of the interview versus recalling how they were covered in each month of the previous year.

The 2015 CPS ASEC should be released in the Fall of 2015 and will have coverage information at the point of the interview (February, March and April) in 2015 and for each month of 2013 and 2014. This release will provide a current insurance estimate following the close of the 2015 open enrollment period (November 15, 2014 through February 15, 2015) as well as monthly coverage estimates for the entire 2014 calendar year. If the Census Bureau does not make additional changes to the questionnaire, the health insurance categories should trend cleanly across the 2014 and 2015 CPS ASEC data sets, providing information on how people were covered before and after implementation of the ACA.

One aspect that will be missing from the CPS ASEC (and which will be supplied by SIPP, discussed below), will be the ability to fully tie coverage in 2014 to coverage status in the previous year for a given individual. While the 2015 CPS ASEC will have information about coverage changes throughout 2014, information on respondents’ coverage status in 2013 will not be available. Thus, for many who gained coverage in 2014, we will be unable to ascertain whether they are newly-insured or whether they switched their coverage type. While the 2014 CPS ASEC will allow us to connect coverage in 2013 to the first few months of 2014, many people in the survey will have been interviewed prior to the close of open enrollment (March 31, 2014); thus, it will be difficult to measure the immediate effect of the availability of coverage in January 2014.

In general, the change in the health insurance questions in the CPS ASEC will allow for fuller and more precise estimates of the effect of the ACA. At the same time, because of the timing of the changes to the CPS ASEC, there will be no way to compare coverage at the time of interviews in 2014 to any previous year and no way to look at trends leading up to the first year of new coverage options under the ACA.

Survey of Income and Program Participation

The Survey of Income and Program Participation (SIPP) is another federal survey often used for coverage estimates. SIPP is a panel survey that follows a sample of households over a period of years. For previous panels, respondents were interviewed three times each year for several years (the panels vary in duration). SIPP collects detailed health coverage, income, employment and demographic information, including health coverage, on a monthly basis, which can be used to analyze changes in circumstances for people and families with different types of coverage. The SIPP 2008 Panel started with more than 40,000 eligible households and ended in 2013.

The approach for the SIPP 2014 Panel has been revised so that households are contacted just once annually to collect information about the previous year and the current year up to the time of the interview. Specifically, respondents are being contacted between February and May of 2014 to collect monthly information, including health coverage, for the interview month and for the previous months all the way back to January of 2013. The initial release (the first wave) of the 2014 SIPP Panel will have monthly information for entire 2013 calendar year and will be released in the Spring of 2015. This release is not expected to include any information about coverage in 2014. Interviews for the SIPP 2014 Panel’s second wave should begin early in 2015 and collect monthly information from the same panel of households for each month during the 2014 calendar year. Data for this second wave should be released by the Spring of 2016, and with this release, it will be possible to analyze monthly coverage information for panel households from January of 2013 through December of 2014, which is the year prior to the first open enrollment period and the first year under the new coverage provisions. These data should provide a comprehensive picture of how coverage changed during the first full year of implementation. SIPP is large enough to support analysis of about half of all states and many other subgroups. The Census Bureau expects to release the third wave of this panel by early 2017 – about one year after the second wave – allowing for detailed exploration of coverage dynamics over the period of January of 2013 through December 2015.

Although there is high demand to know the immediate coverage impacts of the ACA, it is likely that the ultimate coverage changes will play out over a longer period of time as people become more familiar with new options for coverage and as employers revise their plans to accommodate new responsibilities and coverage alternatives. Over time, SIPP will likely be the best source of information to analyze those dynamics nationally, though its limited ability to permit analysis at the state level may restrict its usefulness given how much of the ACA is implemented as the state level.

American Community Survey

Another federal survey that has information on health insurance is the American Community Survey (ACS). This survey originated as a replacement for the long-form of the US Decennial Census and is distributed to one percent of the entire United States population every year. It’s very large sample size allows for coverage estimates for very small geographic areas. However, because it has fewer health insurance questions and collects less information than other survey on family income and structure, it has limited usefulness for national estimates of health insurance coverage. Comparing the 2012 ACS to the 2012 CPS, the two surveys find the nationwide uninsured rate to be about half a percentage point apart (14.8 and 15.4, respectively).

Though it has been in existence since as far back as 1996, the modern ACS only began asking a series of health insurance questions in 2008. While the Census Bureau expects to test new questions collecting ACA Marketplace-related information in the future, there is not currently any timeframe for making changes to the instrument; the health insurance questions currently in the field for the 2014 survey have the exact same structure as they’ve had since 2008. The Census Bureau generally releases the single-year file about two years after the completion of data collection, so the single-year 2013 ACS should be released in December of 2014 and the single-year 2014 ACS should be released in December of 2015.

Conclusion

The ACA includes provisions to address cost, quality, and access of health insurance coverage, and the expansion of health coverage to more Americans is a core goal of the law. Thus, a key measure of success of the ACA is whether the number of uninsured Americans drops. While that outcome seems like a relatively straightforward metric, it will in fact be surprisingly difficult to evaluate.

Early results from polls and surveys by private organizations – Gallup, the Urban Institute, and RAND – show clearly that the number of people uninsured nationally is falling as the ACA goes fully into effect. However, these polls are limited in their ability to precisely estimate the magnitude of the change and discern shifts among different types of coverage. They generally lack the sample size of large, federal surveys and therefore have substantial margins of error and generally do not support state-level analysis. And they are not able to collect as much detailed information on health insurance coverage of demographic groups as the significantly more resource-intensive surveys that often use in-person interviews.

Federal surveys also have their limitations, and in many cases these data sources will not be available for quite some time. For example, the CPS ASEC survey – the most widely-cited source for tabulations of the uninsured – is in the field February, March, and April and has historically asked respondents if they were uninsured for all of the entire previous year. However, researchers have long believed that many people respond to the CPS ASEC based on their insurance status at a point in time instead. The 2014 survey was changed to make the questions more precise, asking about insurance in the previous year as well as at the time of the interview. This will allow for a much better assessment of the effects of the ACA – permitting a comparison of the number of uninsured in 2013 vs. 2014 – but the initial release of this data will not be available until Spring of 2015. And because many of the interviews were completed while open enrollment was still in process, the 2014 survey will not reflect the surge of enrollment in late March.

Other federal survey data that can be used to evaluate the effects of the ACA will be available earlier. First quarter early release results from the NHIS should be available by September of 2014, though it too will not fully reflect the open enrollment period. NHIS results from the first half of 2014 (expected in December) will allow for a fuller assessment of coverage obtained during open enrollment, but even that will not account for Medicaid signups that can occur throughout the year.

A more complete picture of coverage under the ACA will start to emerge in June 2015, when NHIS insurance coverage data for all of 2013 and 2014 will be available. In September 2015 CPS ASEC coverage data for 2013, 2014, and early 2015 will be released (including information by state). By the end of 2015, ACS data will be released, allowing for coverage comparisons with larger sample sizes at the state level. And, by Spring of 2016, SIPP data for 2013 and 2014 will become available, which will permit tracking of coverage changes for the same individuals over time.

A complete understanding of the first year of full ACA implementation will require triangulating across many data sources. Private polls will provide the earliest look at overall coverage changes, but data from larger and more comprehensive federal surveys – which in many cases will not be available until well into 2015 — will be needed to precisely estimate the change in the number of uninsured, shifts across different types of coverage, the demographics of those who have signed up and those who remain uninsured, and trends by state. Even then, it will be difficult to sort out which changes in insurance coverage are due to the ACA and which would have occurred regardless amidst an improving economy. Just as the coverage changes under the ACA will take several years to fully roll out, it will also take time to capture the full effect of the law. In the meantime, efforts to quantify the impact of coverage expansions on individuals will be key to gauging the law’s success.