Language Barriers in Health Care: Findings from the KFF Survey on Racism, Discrimination, and Health

The Survey on Racism, Discrimination, and Health was designed and analyzed by researchers at KFF. The survey was conducted June 6 – August 14, 2023, online and by telephone among a nationally representative sample of 6,292 U.S. adults in English (5,706), Spanish (520), Chinese (37), Korean (16), and Vietnamese (13).

The sample includes 5,073 adults who were reached through an address-based sample (ABS) and completed the survey online (4,529) or over the phone (544). An additional 1,219 adults were reached through a random digit dial telephone (RDD) sample of prepaid (pay-as-you-go) cell phone numbers. Marketing Systems Groups (MSG) provided both the ABS and RDD sample. All fieldwork was managed by SSRS of Glen Mills, PA; sampling design and weighting was done in collaboration with KFF.

Sampling strategy:

The project was designed to reach a large sample of Black adults, Hispanic adults, and Asian adults. To accomplish this, the sampling strategy included increased efforts to reach geographic areas with larger shares of the population having less than a college education and larger shares of households with a Hispanic, Black, and/or Asian resident within the ABS sample, and geographic areas with larger shares of Hispanic and non-Hispanic Black adults within the RDD sample.

The ABS was divided into areas (strata) based on the share of households with a Hispanic, Black, and/or Asian resident, as well as the share of the population with a college degree within each Census block group. To increase the likelihood of reaching the populations of interest, strata with higher incidence of Hispanic, Black, and Asian households, and with lower educational attainment, were oversampled in the ABS design. The RDD sample of prepaid (pay-as-you-go) cell phone numbers was disproportionately stratified to reach Hispanic and non-Hispanic Black respondents based on incidence of these populations at the county level.

Incentives:

Respondents received a $10 incentive for their participation, with interviews completed by phone receiving a mailed check and web respondents receiving a $10 electronic gift card incentive to their choice of six companies, a Visa gift card, or a CharityChoice donation.

Community and expert input:

Input from organizations and individuals that directly serve or have expertise in issues facing historically underserved or marginalized populations helped shape the questionnaire and reporting. These community representatives were offered a modest honorarium for their time and effort to provide input, attend meetings, and offer their expertise on dissemination of findings.

Translation:

After the content of the questionnaire was largely finalized, SSRS conducted a telephone pretest in English and adjustments were made to the questionnaire. Following the English pretest, Cetra Language Solutions translated the survey instrument from English into the four languages outlined above and checked the CATI and web programming to ensure translations were properly overlayed. Additionally, phone interviewing supervisors fluent in each language reviewed the final programmed survey to ensure all translations were accurate and reflected the same meaning as the English version of the survey.

Data quality check:

A series of data quality checks were run on the final data. The online questionnaire included two questions designed to establish that respondents were paying attention and cases were monitored for data quality including item non-response, mean length, and straight lining. Cases were removed from the data if they failed two or more of these quality checks. Based on this criterion, 4 cases were removed.

Weighting:

The combined cell phone and ABS samples were weighted to match the sample’s demographics to the national U.S. adult population using data from the Census Bureau’s 2021 Current Population Survey (CPS). The combined sample was divided into five groups based on race or ethnicity (White alone, non-Hispanic; Hispanic; Black alone, non-Hispanic; Asian alone, non-Hispanic; and other race or multi-racial, non-Hispanic) and each group was weighted separately. Within each group, the weighting parameters included sex, age, education, nativity, citizenship, census region, urbanicity, and household tenure. For the Hispanic and Asian groups, English language proficiency and country of origin were also included in the weighting adjustment. The general population weight combines the five groups and weights them proportionally to their population size.

A separate weight was created for the American Indian and Alaska Native (AIAN) sample using data from the Census Bureau’s 2022 American Community Survey (ACS). The weighting parameters for this group included sex, education, race and ethnicity, region, nativity, and citizenship. For more information on the AIAN sample including some limitations, adjustments made to make the sample more representative, and considerations for data interpretation, see Appendix 2.

All weights also take into account differences in the probability of selection for each sample type (ABS and prepaid cell phone). This includes adjustment for the sample design and geographic stratification of the samples, and within household probability of selection.

The margin of sampling error including the design effect for the full sample is plus or minus 2 percentage points. Numbers of respondents and margins of sampling error for key subgroups are shown in the table below. Appendix 1 provides more detail on how race and ethnicity was measured in this survey and the coding of the analysis groups. For results based on other subgroups, the margin of sampling error may be higher. All tests of statistical significance account for the design effect due to weighting. Dependent t-tests were used to test for statistical significance across the overlapping groups.

Sample sizes and margins of sampling error for other subgroups are available by request. Sampling error is only one of many potential sources of error and there may be other unmeasured error in this or any other public opinion poll. KFF public opinion and survey research is a charter member of the Transparency Initiative of the American Association for Public Opinion Research.