Click here to return to the previous page
Weighting and Statistical Significance
Because Kaiser/HRET selects firms randomly, it is possible through the use of statistical weights to extrapolate the results to national (as well as firm size, regional, and industry) averages. These weights allow Kaiser/HRET to present findings based on the number of workers covered by health plans, the number of total workers, and the number of firms. In general, findings in dollar amounts (such as premiums, worker contributions, and cost sharing) are weighted by covered workers. Other estimates, such as the offer rate, are weighted by firms. Specific weights were created to analyze the HDHP/SO plans that are offered with an HRA or that are HSA qualified. These weights represent the proportion of employees enrolled in each of these arrangements.
Calculation of the weights follows a common approach. First, the basic weight is determined, followed by a nonresponse adjustment. As part of this nonresponse adjustment, Kaiser/HRET conducted a small follow-up survey of those firms with 3 to 49 workers that refused to participate in the full survey. We applied an additional nonresponse adjustment to the weight to reflect the findings of this survey.
Next, we trimmed the weights in order to reduce the influence of weight outliers. First, we identified common groups of observations. Within each group, we identified the median and the interquartile range of the weights and calculated the trimming cut point as the median plus six times the interquartile range (M + [6 * IQR]). Weight values larger than this cut point are trimmed to the cut point. In all instances, less than one percent of the weight values were trimmed.
Finally, we applied a post-stratification adjustment. We used the U.S. Census Bureau’s 2004 Statistics of U.S. Businesses as the basis for the stratification and the post-stratification adjustment for firms in the private sector, and we used the 2002 Census of Governments as the basis for post-stratification for public sector firms.
The survey contains a few questions on employee cost sharing that are asked only of firms that indicate in a previous question that they have a certain cost-sharing provision. For example, the copayment amount for prescription drugs is asked only of those that report they have copayments for prescription drugs. Because the composite variables are reflective of only those plans with the provision, separate weights for the relevant variables were created in order to account for the fact that not all covered workers have such provisions.
The data are analyzed with SUDAAN,7 which computes appropriate standard error estimates by controlling for the complex design of the survey. All statistical tests are performed at the .05 level unless otherwise noted. For figures with multiple years, statistical tests are conducted for each year against the previous year shown, unless otherwise noted. No statistical tests are conducted for years prior to 1999.
Statistical tests for a given subgroup (firms with 25-49 workers, for instance) are tested against all other firm sizes not included in that subgroup (all firm sizes NOT including firms with 25-49 workers in this example). Tests are done similarly for region and industry; for example, Northeast is compared to all firms NOT in the Northeast (an aggregate of firms in the Midwest, South, and West). However, statistical tests for estimates compared across plan types (for example, average premiums in PPOs) are tested against the “All Plans” estimate. In some cases, we also test plan specific estimates against similar estimates for other plan types (for example, single and family premiums for HDHP/SOs against single and family premiums in HMO, PPO, and POS plans); these are noted specifically in the text. The two types of statistical tests performed are the t-test and the Pearson Chi-square test.
The small number of observations for some variables, particularly variables specific to plans with Health Savings Accounts or Health Reimbursement Arrangements, resulted in large variability around the point estimates. These observations sometimes carry large weights, primarily for small firms. The reader should be cautioned that these influential weights may result in large movements in point estimates; however, often these movements are not statistically significant.
Historical Data
Data in this report focus primarily on findings from surveys jointly authored by the Kaiser Family Foundation and the Health Research and Educational Trust, which have been conducted since 1999. Prior to 1999, the survey was conducted by the Health Insurance Association of America (HIAA) and KPMG using a similar survey instrument, but data is not available for all the intervening years. Following the survey’s introduction in 1987, the HIAA conducted the survey through 1990, but some data are not available for analysis. KPMG conducted the survey from 1991-1998. However, in 1991, 1992, 1994, and 1997, only larger firms were sampled. In 1993, 1995, 1996, and 1998, KPMG interviewed both large and small firms.
This report uses data from the 1993, 1996, and 1998 KPMG Surveys of Employer-Sponsored Health Benefits and the 1999-2007 Kaiser/HRET Survey of Employer-Sponsored Health Benefits. For a longer-term perspective, we also use the 1988 survey of the nation’s employers conducted by the HIAA, on which the KPMG and Kaiser/HRET surveys are based. Many questions in the HIAA, the KPMG, and Kaiser/HRET surveys are identical. The survey designs among the three surveys are also similar.
Click here to continue on to Exhibit M.1
