Understanding the Diversity in the Asian Immigrant Experience in the U.S.: The 2023 KFF/LA Times Survey of Immigrants

The KFF/LA Times Survey of Immigrants is a partnership survey conducted by KFF and the LA Times examining the U.S. immigrant experience.

The survey was conducted April 10-June 12, 2023, online, by telephone, and by mail among a nationally representative sample of 3,358 immigrants, defined as adults living in the U.S. who were born outside the U.S. and its territories. Respondents had the option to complete the survey in one of ten languages: English (n=2,435), Spanish (n=627), Chinese (n=171), Korean (n=52), Vietnamese (n=22), Portuguese (n=16), Haitian-Creole (n=13), Arabic (n=9), French (n=9), and Tagalog (n=4). These languages were chosen as they are most commonly spoken by immigrant adults from countries of focus for the survey with limited English proficiency (LEP), based on the 2021 American Community Survey (2021).

Teams from KFF and The Los Angeles Times worked together to develop the questionnaire and both organizations contributed financing for the survey. KFF researchers analyzed the data, and each organization bears the sole responsibility for the work that appears under its name. Sampling, data collection, weighting, and tabulation were managed by SSRS of Glenn Mills, Pennsylvania in collaboration with public opinion researchers at KFF.

Respondents were reached through one of three sampling modes: an address-based sample (ABS) (n=2,661); a random digit dial telephone (RDD) sample of prepaid (pay-as-you go) cell phone numbers (n=565); and callbacks to telephone numbers that that were previously randomly sampled for RDD surveys and were identified as speaking a language other than English or Spanish (n=132). Respondents from all three samples were asked to specify their country of birth and qualified for the survey if they were born outside of the U.S.

Project design was informed by a pilot study conducted from January 31-March 14, 2022 among a sample of 1,089 immigrants in collaboration with SSRS. Prior to fielding the pilot study, KFF and SSRS conducted interviews with experts who had previous experience surveying immigrants. These conversations informed decisions on sampling, modes of data collection, recruitment strategies, and languages of interviews. The pilot test measured incidence of immigrant households across four different sample types and offered a short survey in 8 different languages both online and on the telephone. Based on the results of the pilot test, the following recruitment and data collection protocol was implemented:

Sampling strategy and interview modes:

The ABS was divided into areas (strata), defined by Census tract, based on the incidence of immigrants among the population overall and by countries of origin. Within each stratum, the sample was further divided into addresses that were flagged by Marketing Systems Group (MSG) as possibly occupied by foreign-born adults and unflagged addresses. To increase the likelihood of reaching immigrant adults, strata with higher incidence of immigrant households overall, and of immigrants from certain countries of origin were oversampled.

Households in the ABS were invited to participate through multiple mail invitations: 1) an initial letter in English with a short paragraph of instructions in each of the 10 survey languages on the back; 2) a reminder postcard in English plus up to two additional languages; 3) a follow-up letter accompanied by hardcopy questionnaires in English and one additional language; and 4) a final reminder including short messages in all 10 languages. For mailings 2 and 3, additional languages were chosen by using flags to identify the language other than English likely spoken at home. Invitation letters requested the household member ages 18 or older who was born outside of the U.S. with the most recent birthday to complete the survey in one of three ways: by going online, dialing into a toll-free number, or returning the completed paper questionnaire. In addition, interviewers attempted outbound calls to telephone numbers that were matched to sampled addresses. ABS respondents completed the survey online (n=2,087), over the phone (n=105), or by mail on paper (n=469). The random sample of addresses was provided by MSG.

The RDD sample of prepaid (pay-as-you-go) cell phone numbers was obtained through MSG. The prepaid cell phone component was disproportionately stratified to effectively reach immigrants from different countries based on county-level information. To increase the likelihood of reaching immigrant adults, counties with higher incidence of immigrants overall, and of certain countries of origin were oversampled.

The callback sample included 132 respondents who were reached by calling back telephone numbers that were previously randomly sampled for SSRS RDD surveys within two years and coded by interviewers as non-English or non-Spanish speaking.  as having respondents speaking languages other than English or Spanish.

Incentives:

Initial mailings to the ABS sample included $2 as part of the invitation package, and respondents received a $10 incentive if they completed the survey in the first two weeks after the initial mailing. In order to increase participation among under-represented groups, the incentive increased to $20 for those who did not respond within the first two weeks. ABS phone respondents received this incentive via a check received by mail, paper respondents received a Visa gift card by mail, and web respondents received an electronic gift card incentive. Respondents in both phone samples received a $25 incentive via a check received by mail.

Questionnaire design and translation:

In addition to collaboration between KFF and the LA Times, input from organizations and individuals that directly serve or have expertise in issues facing immigrant populations helped shaped the questionnaire. These community representatives were offered a modest honorarium for their time and effort to review questionnaire drafts, provide input, attend meetings, and offer their expertise on dissemination of findings.

After the content of the questionnaire was largely finalized, SSRS conducted a telephone pretest in English and adjustments were made to the questionnaire. Following the English pretest, Research Support Services Inc. (RSS) translated the survey instrument from English into the nine languages outlined above and performed cognitive testing through qualitative interviews in all languages including English. The results of the cognitive testing were used to adjust questionnaire wording in all languages including English to ensure comprehension and cohesiveness across languages and modes of interview. As a final check on translation and its overlay into the web and CATI program, translators from Cetra Language Solution reviewed each question, as it appears in the program, and provided feedback. The questionnaire was revised and finalized based on this feedback.

Data quality checks:

A series of data quality checks were run on the final data. The online questionnaire included two questions designed to establish that respondents were paying attention and cases were monitored for data quality. Fifteen cases were removed from the data because they failed two or more quality checks, failed both attention check questions, or skipped over 50% of survey questions. An additional 67 interviews were removed after deemed ineligible by SSRS researchers (they were not U.S. immigrants).

Weighting:

The combined sample was weighted to adjust for the sampling design and to match the characteristics of the U.S. adult immigrant population, based on data from the Census Bureau’s 2021 American Community Survey (ACS). Weighting was done separately for each of 11 groups defined by country or region of origin (Mexico, China, Other East/Southeast Asia, South Asia, Europe, Central America, South America, Caribbean, Middle East/North Africa, Sub-Sahara Africa, all others). The samples were weighted by sex, age, education, race/ethnicity, census region, number of adults in the household, presence of children in the household, home ownership, time living in the U.S., English proficiency, and U.S. citizenship. The overall sample was also weighted to match the share of U.S. adult immigrants from each country/region of origin group. The weights take into account differences in the probability of selection for each of the three sample types. This includes adjustment for the sample design and geographic stratification, and within household probability of selection. Subgroup analysis includes data checks to ensure that the weighted demographics of subgroups are within reasonable range from benchmarks whenever possible.

The margin of sampling error including the design effect for the full sample is plus or minus 2 percentage points. Numbers of respondents and margins of sampling error for key subgroups are shown in the table below. For results based on other subgroups, the margin of sampling error may be higher. Sample sizes and margins of sampling error for other subgroups are available by request. Sampling error is only one of many potential sources of error and there may be other unmeasured error in this or any other public opinion poll. KFF public opinion and survey research is a charter member of the Transparency Initiative of the American Association for Public Opinion Research.

Focus group methodology:

As part of this project, KFF conducted 13 focus groups with immigrant adults across the country to help inform survey questionnaire development, provide deeper insights into the experiences of immigrant groups that had a smaller sample size in the survey, and to provide a richer understanding of some of the survey findings.

Two rounds of focus groups were completed. The first round of 6 groups was conducted between September-October 2022 virtually among participants living across the country who are Hispanic immigrants (conducted in Spanish), Asian (excluding Chinese) immigrants (conducted in English), or Chinese immigrants (conducted in Mandarin Chinese). The groups were separated by gender, lasted 90 minutes, and included 5-7 participants each.

The second round of groups were conducted in-person between May-June 2023 in Los Angeles, CA and Fresno, CA with Hispanic immigrants conducted in Spanish; and in Houston, TX and Irvine, CA with Vietnamese immigrants conducted in Vietnamese. In addition, virtual groups were conducted among participants living in the Texas border region (Hispanic immigrants), the Miami, FL region (Haitian immigrants), and nationally (Black immigrants from sub-Saharan Africa). Groups were mixed gender, lasted between 90 minutes and two hours, and were conducted in English, Spanish, Vietnamese, and Haitian-Creole with 5-8 participants each.

For each group, participants were chosen based on the following criteria: Must be at least 18 years of age and have been born outside of the U.S. and its territories; for groups conducted in languages other than English, must speak English “less than very well” and be able to speak conversationally in the group’s language (i.e., Spanish). In addition, groups were chosen to represent a mix of household composition, including at least some participants who are parents; a mix of household income levels, with a preference for recruiting lower income participants; a mix of health insurance types; and a mix of immigration statuses. Goodwin Simon Strategic Research (GSSR) recruited and hosted the first round of focus groups. PerryUndem recruited and hosted the second round of focus groups. The screener questionnaire and discussion guides were developed by researchers at KFF in consultation with the firms who recruited and hosted the groups. Groups were audio and video recorded with participants’ permission. Each participant was given $150-$175 after participating.