2

Overview: Survey of Income and Program Participation (SIPP) and Its Uses

SIPP has been conducted by the Census Bureau since 1983. SIPP was designed to provide data for researchers and policymakers to estimate the costs, impacts, and effectiveness of government social programs designed to help the poor, including how well programs reached their target populations, what the impact of program participation was on participants and their households, what overlap occurred among populations served by different programs, and why and how often people transitioned into and out of programs. SIPP was also designed to answer questions about short-term financial and family dynamics. Since its original design, it has evolved to provide data on a broader range of social issues and has become a general-purpose dataset for information about U.S. society. SIPP data have unique features concerning these topics that are not provided by any other household survey, either federal or nonfederal (see National Academies of Sciences, Engineering, and Medicine, 2018, as well as the other National Academies reports summarized in Chapter 1).

This chapter provides background information about SIPP: a brief history of the survey, its uniqueness, the content collected, data products produced, historical approaches to protecting data from disclosure, and how the data are used.

FORMATIVE AND EARLY YEARS OF SIPP

Until 1983, when SIPP was introduced, the country’s major source of information on income and program participation was the March Income Supplement of the Current Population Survey (CPS). However, the main

statutory purpose of the CPS was to collect sufficient data to estimate unemployment rates, and CPS is not designed to collect the kinds of data that affect eligibility for program participation. Nor was it designed to collect general socioeconomic characteristics on the U.S. population. Furthermore, aside from a minor feature of the CPS allowing the analysis of individual and household characteristics as they change over time, the CPS was not longitudinal and hence the evolution of household income, poverty status, and program participation could not be tracked.¹

The Department of Health, Education and Welfare initiated the predecessor of SIPP in 1975 in collaboration with the Census Bureau. The department spent the next five years testing and designing the Income Survey Development Program, a longitudinal survey to collect general socioeconomic and social program participation information for a representative sample of the U.S. population. The first nationwide test of this new survey, which sampled approximately 2,000 households, occurred in 1978, and a representative national sample of 8,200 households was surveyed in 1979. In 1981, the Department of Health, Education and Welfare lost its funding for the Income Survey Development Program, and work halted. Work resumed in 1983, when the Census Bureau received funding to continue the renamed Survey of Income and Program Participation, which officially launched in October of that year.

The original design of SIPP focused on a nationally representative sample of households in the civilian non-institutionalized population, with every person age 15 or older in the household being interviewed. Four months after the initial interview, respondents were surveyed again; this process repeated over a 32-month period. Each February, a new group of respondents was interviewed, creating an overlapping series of survey panels.

During the late 1980s, SIPP suffered from erratic and sometimes limited funding. Although each panel was supposed to go through eight interviews, this often was not the case. The 1989 panel, part of which was folded into the 1990 panel, was only interviewed three times. Funding problems also limited panel sizes, and the difficulty in field operations caused by having multiple panels in the field at the same time were significant obstacles to data collection and quality. Each panel was supposed to comprise about 20,000 households, but that number was rarely achieved.

The Census Bureau redesigned SIPP in 1996, with input from National Research Council (1993). Instead of conducting 32-month panels, the Census Bureau switched to 48-month panels to improve the quality of longitudinal estimates and to provide better information for policymakers. Field representatives interviewed respondents 12 or 13 times over the course of the panel—about three times per year—thereby retaining the rough

___________________

¹ See https://www.census.gov/programs-surveys/sipp/about/sipp-introduction-history.html

every-four-month interview schedule in the prior panels. In 1996 the Census Bureau also introduced computer assisted interviewing, which allowed for faster data processing and for real-time answer consistency checks.²

The difficulty in field operations from four-month interviewing and a variety of other factors led to increasing costs and reductions in interviewing and sample sizes. After much analysis and planning, SIPP was again redesigned in 2014, with input from the National Academies (see National Research Council, 2009). The catalyst for this redesign was SIPP’s imminent cancellation. Though SIPP was saved by support from stakeholders, major changes were deemed necessary. The goals of this redesign were to reduce costs and respondent burden and to improve data quality and timeliness. Three major changes were that (1) sampled households would now be contacted only once a year rather than three times, (2) topical modules were eliminated, and (3) the survey instrument was completely redeveloped around an Event History Calendar as a tool for collecting data on intra-year dynamics. The National Academies provided an assessment of that redesign and considered content changes for future improvement of SIPP (National Academies, 2018).

The basic design of SIPP has not changed appreciably since the 2014 revision, except that SIPP started using yearly overlapping panels in 2019.

WHAT MAKES SIPP UNIQUE AND VALUABLE

Compared to other countries, the United States is rich in datasets from household surveys that provide important information on the population and society. Despite this richness, SIPP occupies a unique position in the landscape of social datasets that no other dataset can replicate.

Among datasets that are primarily cross-sectional in nature, the most heavily used alternative household survey is the CPS. As already noted, the CPS was not designed to be a general-purpose dataset collecting information on all economic, demographic, labor market, health, and social program participation domains of importance to understanding U.S. society. The CPS collects only a small subset of the near-3,000 variables collected on individuals and households in each wave of SIPP. Another important household survey is the American Community Survey (ACS), which focuses on local areas with large enough sample sizes to obtain reliable statistics and hence serves a vital role for state and local policymakers. SIPP does not have the capacity to generate statistics at as detailed a level of geography as the ACS. But the ACS is cross-sectional and does not follow households over time. It also has a much smaller set of variables and could not be used to

___________________

² Taken from https://www.census.gov/history/www/programs/demographic/survey_of_income_and_program_participation.html

provide the national-level statistics on all the variables that SIPP can. Finally, the National Health Interview Survey is the country’s leading population-representative survey of the health status of U.S. individuals and arguably has better information on health and medical care than SIPP, but it does not collect anywhere close to the number of other domains as SIPP collects.

There are other social surveys that follow individuals or households over time, as SIPP does, and are hence longitudinal in nature. The National Longitudinal Surveys collected by the U.S. Bureau of Labor Statistics, for example, follow birth cohorts of individuals over time, charting their life courses with a particular focus on labor market outcomes. But these surveys only collect data on selected birth cohorts and hence are unsuitable for cross-sectional statistics on the U.S. population as a whole. The Health and Retirement Study, sponsored by the National Institutes of Health, also collects valuable health and economic information over time but only on the older population. The Panel Study of Income Dynamics is the nation’s longest general-purpose social information survey, having started in 1968 and continuing to survey individuals who were alive at that time as well as all their children as they start their own families. This survey does not have the same sample size as SIPP and hence cannot produce statistics for the same detailed subgroups, nor does it collect the detailed social program information or a number of other dimensions that SIPP does. But the uniqueness of the Panel Study of Income Dynamics is that it follows individuals over their entire life course, not just over four years, and it can be used to analyze intergenerational mobility. SIPP is superior in its cross-sectional dimensions and in its short-term dynamics over the typically four years of SIPP panels.

In summary, SIPP is unique because it combines three features: (1) an extremely large number of variables collected on all major domains of importance to U.S. individuals and households; (2) its high level of detail on social program participation across a large number of programs; and (3) its ability, afforded by its longitudinal dimension, to follow individuals and households over time to chart the evolution of their status in the large number of domains on which it collects information. Furthermore, by creating an almost-complete record in these dimensions over the past 50 years, SIPP is a unique source for studying historical trends in the country. In the panel’s view, preserving these unique characteristics of SIPP deserves high priority.

Two other characteristics also contribute greatly to the value of SIPP, though they might not necessarily be unique. SIPP collects multiple waves of data within a single panel year, thus making a greater number of cases available for analysis in any given year. Moreover, if needed and the questionnaire items have not changed, SIPP data can be combined across multiple years, providing a greater capacity to study small groups. Larger surveys, such as the ACS, may not have the same need to expand the number of

cases for analysis, but SIPP’s design makes it very efficient in providing data even for rare groups. SIPP also is well designed for making linkages with administrative data and even provides SIPP synthetic data in which the links have already been made (and synthesized). Other surveys also allow linkages, but SIPP’s collection of data on program participation makes the potential for linkages especially useful.

SAMPLING DESIGN AND METHODOLOGY FOR SIPP

SIPP is a longitudinal survey, designed to be nationally representative of the civilian non-institutionalized population. In general, since 2014, four years of monthly data have been collected through annual surveys. While the data can be (and often are) analyzed cross-sectionally, focusing on a single point of time, the repeated observations of month-by-month data over four years support the analysis of change over time.

Similar to the 1990 through 1993 panels, beginning in 2019 the SIPP panels have been designed to be overlapping in time (with some exceptions). For example, the 2023 data collection includes the first wave of the 2023 panel, the second wave of the 2022 panel, the third wave of the 2021 panel, and the fourth (and final) wave of the 2020 panel. All four of these collectively are considered as the 2023 SIPP; that is, each SIPP is named based on the year of the data collection, not the date of the first wave of a panel. When an analyst wishes to focus on a particular year, all four applicable panels for that year are provided together, greatly increasing the amount of data available for analysis. Alternatively, an analyst can examine change over time by combining multiple waves within a single panel. (Different weights are available depending on how the research is to be conducted.)

The primary sampling unit is the household, primarily based on the decennial census. Low-income households are oversampled. Within each household, all adults (age 15 or older) are interviewed, either directly or through proxies if a household member is unavailable. Information about household members under 15 years of age is collected via a proxy interview. In follow-up waves, the original set of adults is interviewed (including those who have moved to new addresses, with a few exceptions such as those living abroad or in institutions), along with any new adults living with the original sample members. New people who join a household are considered to be part of the SIPP panel for as long as they continue to live with an original sample member. People are considered members of the household if they are usual residents (e.g., sleeping there a majority of the time).

Since 1996, all first waves of a panel have contained at least 37,000 households. The 2020 SIPP sampled 57,834 designated Living Quarters, which yielded 50,352 eligible Living Quarters, 21,989 households that were interviewed, and 53,332 person interviews (U.S. Census Bureau, 2021).

Separate weights are provided or may be constructed for different types of analyses: monthly weights for a cross-sectional sample at a given month, calendar-year weights for samples that provide data in December of the reference year, and longitudinal weights for samples that provide monthly data for more than one consecutive year (with the potential for three different sets of longitudinal weights for reference periods of two, three, or four calendar years). The nonzero final person weights for SIPP 2020 Wave 1 ranged from 387.1 to 39,950.8.³ The data files also include replicate weights and longitudinal replicate weights which are needed for correctly calculating standard errors, given the complex sampling design.

SIPP has traditionally relied on in-person household interviews (except for those who move out of an area, who are interviewed by telephone), but during the COVID-19 pandemic the mode was switched to telephone interviewing on March 19, 2020.

To reduce respondent burden, SIPP groups some related people into clumps, limiting the number of people who must be asked certain questions. Identification of the clump is not provided in the public-use data.

SIPP contains different kinds of nonresponse. One form occurs when an eligible household is not interviewed at all, another when at least one person is interviewed but not everyone in the household is, and a third occurs when a respondent refuses to give a response or gives an invalid response to a particular question or set of questions (item nonresponse). For the third of these, while nonresponse rates are low for many variables, they are quite high for some important variables, like household income (49% nonresponse rate; Tersine, 2022). For the first two types of nonresponse, the Census Bureau provides weights to correct for the missing responses. For the third type, the Census Bureau uses imputation to fill in a value for the missing item. For imputation, the Census Bureau uses a combination of model-based imputation, sequential hot-deck imputation, logical imputation, and cold-deck imputation.

THE CONTENT COLLECTED THROUGH SIPP

Since the 2014 SIPP, the reference year for which respondents are asked to report is generally the preceding year. For example, for the 2020 SIPP, questions were generally asked about 2019. Exceptions to this include questions on disability status and parent-child relationships.

The core of SIPP is the collection of data on income and program participation, including four years of employment data, assets, debts, retirement accounts, and types and amounts of other income including social

___________________

³ Weights of zero are assigned when data are not available for all 12 months or when the person was not part of the sample at Wave 1, with some exceptions.

insurance, social welfare, and other support paid and received such as alimony and child support.⁴ SIPP also collects detailed information about health and disability, such as health insurance, functional limitations, health care utilization, and medical expenditures.

Other data collected by SIPP include adult well-being, education, language, education enrollment, demographic data, relationships among household members, citizenship and immigration status, marital history, dependent care, fertility history, biological parents’ nativity and mortality, and child well-being. Some of these characteristics (such as demographics and citizenship) are collected once per wave, while other characteristics (such as residence, family, and household composition; marital status; and commuting and work schedules) are collected for each month of the study period.

Topics Covered

Table 2-1 displays the basic types of data collected and provided by SIPP.

The broad categories listed above sometimes hide considerable detail. For example, the adult well-being section in 2020 consisted of 11 questions asking about conditions such as cracks in the ceilings or walls, holes in the floor, problems with pests, plumbing problems, street noise or traffic problems, trash in the streets, staying home for safety reasons, neighborhood safety from crime, difficulty in paying the rent or mortgage, and difficulty paying utility bills.

Over the period 1984–2008, SIPP also contained topical questions that were not repeated in each wave, typically in separate sections but sometimes intermixed with related core questions when that improved the question flow.⁵ Since the focus of the consensus panel is on disclosure avoidance for the current survey, the prior topical modules are not discussed here in detail.

One of the unusual features of SIPP is that it collects data on each member of the household and their relationships with all other household members. This differs from other household surveys, such as the ACS, which collect data on relationships to the householder only. A total of 20 different relationship types are covered for each household member (see Box 2-1). Thus, though the core of SIPP data consists of data on income and program participation, other SIPP data can be used to address a wide variety of sociological issues, such as the relationship between health conditions and divorce behavior, transitions in parental presence among children,

___________________

⁴ https://www.census.gov/programs-surveys/sipp/about/sipp-content-information.html

⁵ https://www.census.gov/programs-surveys/sipp/tech-documentation/topical-modules.html

TABLE 2-1 Data Collected in SIPP 2020, by Broad Category

SOURCES: U.S. Census Bureau, 2021; 2020 Survey of Income and Program Participation (SIPP): Public Use Instrument Specifications.

and (based on a different section of the questionnaire) the prevalence of multiple-partner fertility.

The public-use file is not designed to replicate the survey data on a variable-by-variable basis. Some public-use variables have value ranges that differ from those in the original data, some variables are combinations of other variables, and prior to the 2014 SIPP panel some variables were in topical modules rather than in the core wave files.

Selected Studies Show the Range and Depth of SIPP

While the core of the SIPP questionnaire is devoted to financial measures, the broad range and specificity of the data collected has supported a wide range of research. A Google Scholar search (in January 2023) for “Survey of Income and Program Participation” produced over 20,000 results, though those results include citations in addition to independent analyses of SIPP data. Following are illustrative uses of the data that, while far from comprehensive, show the range and depth of research conducted using SIPP.

Economic Data

As might be expected, SIPP has often been used to examine various aspects of the U.S. economy.

• Occupations and earnings. The Census Bureau, as part of its Current Population Reports, regularly publishes data on occupation, earnings, and job characteristics, using SIPP as a primary source. The data include what occupational categories workers are employed in, their demographic characteristics, their employment characteristics (full- or part-time employment, holding of multiple jobs simultaneously, sector of employment), work schedule, work-from-home status, median earnings, and health insurance coverage (Gumber & Sullivan, 2022).
• Poverty dynamics. The Census Bureau also uses SIPP to publish data on poverty, including the monthly poverty rate overall and by demographic characteristics, as well as rates of episodic and chronic poverty (Warren & Tettenhorst, 2022).
• Income sources of older households. Another Census Bureau report describes the sources of income for older households, including social programs, pensions, and private retirement savings. Reporting on each household income decile, it describes the income ranges, receipt rates of six types of income, the mean income from each type, and the distribution of income types, along with supplementary statistics on certain categories of households (Thompson & King, 2022).

Monitoring of Federal Programs

The federal government uses SIPP to examine the usage of its programs.

• SNAP. Over 90 percent of children receiving SNAP benefits also received at least one other form of assistance. The programs include Medicaid, State Children’s Health Insurance Program (SCHIP), SNAP, TANF, SSI, and Social Security (King & Giefer, 2021).
• Student debt. Students with student loan debt are more likely than other students to have other types of household debt, with the debt rates described in terms of the rates and amounts of debt, the probability of debt, and the predicted amount of debt by type, student loan status, and student demographic group (Bennett & King, 2022).
• Joint federal and state programs. Some federal programs, such as Medicaid, SCHIP, and TANF, are administered in part by the states,

• with different states setting different eligibility requirements. These differences in part can provide a testing ground for evaluating the impact of changes in federal policies, and they are also important when monitoring the impact of federal programs. SIPP is a valuable tool for evaluating such differences (Irvin & Czajka, 2010). For such evaluations and monitoring, state-level identifiers are required.

Uses by Policy Researchers

Policy researchers use SIPP to examine the need for assistance and the impact of policies and programs.

• Housing and health quality. Boch et al. (2020) examined the association between poor housing quality and adult health outcomes, adjusting for socioeconomic factors and contextual housing characteristics.
• Health care. Carman et al. (2020) calculated the total costs to households in paying for health care, finding that higher-income households paid the most for health care, but payment as a share of income was greater among lower-income households.
• Education and earnings. Carr and Wiemers (2020) found an increase in inequality in long-run earnings for both men and women and a resulting decline in intergenerational mobility.
• Immigration policies and health. Examining differences in state policies concerning immigration, Dondero and Altman (2020, p. 1) found that “restrictive immigrant policy climates exacerbate nativity gaps in health provider visits among working-age adults,” and that “even immigrant policies not directly related to health have consequences for immigrants’ health care utilization.”
• Extreme poverty. Shaefer and Edin (2013) found an increase in extreme poverty between 1996 and 2011 among U.S. households with children, and that adding SNAP benefits reduced the number of such households by 48 percent. Also, accounting for refundable tax credits and housing subsidies reduced the number by 63 percent (Shaefer & Edin, 2013).
• Occupational licensing. Gittleman et al. (2018, p. 57) found that “those with a license earn higher pay, are more likely to be employed, and have a higher probability of employer-sponsored health insurance offers.”
• The safety net and material hardship. McKernan et al. (2021) estimated the impact of participation in TANF, SNAP, or Medicaid/SCHIP on the number of material hardships and the incidence of food insufficiency.

• Minimum wage. Comparing states that were bound to increases in the federal minimum wage to states that were not, Clemens and Wither (2019) estimated that the minimum wage increases reduced employment rates by at least half a percentage point in the states that were bound.
• Federal program evaluations. In addition to academic research, SIPP is a key input for models developed by federal agencies to evaluate programs. The Modeling Income in the Near Term (MINT) model used by the Social Security Administration (SSA) is a dynamic micro-simulation model that projects into the future a database consisting of SIPP data matched to Social Security administrative records to evaluate various SSA programs (Smith et al., 2021). In a joint statistical project, the Economic Research Service and Census Bureau have examined SIPP linked to the administrative SNAP data (Scherpf et al., 2015), and with the Food and Nutrition Service, Mathematica developed the MATH SIPP+ simulation model to evaluate SNAP eligibility (Leftin et al., 2014). The Congressional Budget Office has also created simulation models using SIPP data (see National Academies, 2018, pp. 33–39).

Sociological Research

The breadth of data offered by SIPP also supported sociological research, not always with financial data.

• Parental presence. Scherer and Mayol-Garcia (2020) examined the demographic and socioeconomic characteristics of children who experienced at least one transition in parents (including changes in the number of resident parents, changes in which parent was coresident, and changes in a parent’s cohabiting partner).
• Multiple-partner fertility research. Defining multiple-partner fertility as someone who has had biological children with more than one partner, Monte (2017) found that 10 percent of all adults have had children with more than one partner, as have 21 percent of all parents with two or more children.
• Leave usage after first birth. Scherer (2022) found that leave usage among men after the birth of their first child increased over time, and it was more common among men with higher levels of educational attainment.
• Children in shared households. Pilkauskas and Cross (2018) found a more than 4 percentage point increase from 1996 to 2016 in the percentage of children who lived in three-generation households.

DATA PRODUCTS FROM SIPP

Census Bureau Reports

The Census Bureau’s primary method for publishing estimates is through its P-70 series.⁶ These reports provide access to key SIPP data at a summary level and may be based on the internal data files, without top-coding and other confidentiality protection techniques. They are topical in nature; consist of text, tables, and graphics; and are highly variable in length (recent reports range from six pages to roughly 100 pages).

The Census Bureau also publishes table packages (found at Survey of Income and Program Participation Table Packages [census.gov]), which are sets of tables about a particular topic. For example, recent releases include

• A Child’s Day (with tables about children’s extracurricular activities and school outcomes);
• State-Level Wealth, Asset Ownership, & Debt of Households Detailed Tables: 2019;
• Wealth, Asset Ownership, & Debt of Households Detailed Tables: 2019; and
• Survey of Income and Program Participation (SIPP) Detailed Program Receipt Tables: 2019.

These most recent packages consist of between two and 17 tables, and unlike the P-70 reports they consist entirely of tables. They can be used as reference tools for providing key statistics without independent data analysis.

Finally, the Census Bureau publishes working papers based on SIPP.⁷ These tend to be more analytical than the P-70 reports and may include modeling, experimental designs, and other advanced statistical analysis. In length, recent (2022) papers range from a single page (e.g., as for a poster session) to 35 pages. These papers are designed to examine an individual topic in depth, and they are more likely to be cited in the same way that journal articles are than to be used as reference tools for looking up key statistics.

Microdata for Data Analysis

For those users who wish to conduct customized data analyses, the Census Bureau provides three options: through public-use files, using SIPP Synthetic Beta, and using the Federal Statistical Research Data Centers (FSRDCs).

___________________

⁶ The issue briefs can be found at https://www.census.gov/programs-surveys/sipp/library/publications.html

⁷ The working papers can be found at https://www.census.gov/programs-surveys/sipp/library/working-papers.html

Public-Use Files

Anyone may download SIPP public-use data from the SIPP data website.⁸ No registration is required to download the data, so it is not possible to directly track users. The number of downloads is potentially countable, but the same person may download the data multiple times or may download the data without using them. Public-use files have some data suppressed and some data altered (i.e., by top-coding or bottom-coding). The public-use metadata report for the 2020 SIPP lists 2,873 variables (not counting status flags,⁹ which exist for almost every variable), though data for 414 of these variables were suppressed. The downloadable file contains 5,026 variables and 622,339 observations. Note that SIPP has a separate record for each of the 12 months and for almost every household member, so the number of observations is much greater than the number of households. After downloading the full file, a data user might extract only those records and variables of interest, perhaps also creating summary measures based on combining multiple records.

SIPP Synthetic Beta (SSB)¹⁰

The SSB combines data from nine SIPP panels (1984, 1990, 1991, 1992, 1993, 1996, 2001, 2004, and 2008) with linked administrative tax and benefit data from the SSA/Internal Revenue Service (IRS) Form W-2 records and SSA records of retirement and disability benefit receipt. Earlier SSB files were partially synthetic, but the most recent version is fully synthetic. The Census Bureau describes the process in the following way:

Some advantages of using the synthetic data are that confidential linked administrative data are available that cannot be released publicly, and editing has been performed to make SIPP data more consistent with the administrative data. The data therefore require less data preparation than would otherwise be the case. On the other hand, the Synthetic Beta contains

___________________

⁸ https://www.census.gov/programs-surveys/sipp/data/datasets.html

⁹ Status flags are used to indicate whether and how a particular value was imputed.

¹⁰ SIPP SSB is described at https://www.census.gov/programs-surveys/sipp/guidance/sipp-synthetic-beta-data-product.html

¹¹ https://www.census.gov/programs-surveys/sipp/guidance/sipp-synthetic-beta-data-product.html

only 141 variables, so it has only a small fraction of the data collected through SIPP.

Data users accessed the SSB through an online server, making the data potentially available to all—that is, without requiring travel to a special location. The server is currently unavailable as it transfers from its past host institution (Cornell). The access process requires that users first must submit an application to use the server, providing contact information, a brief description of the proposed project, and a list of the variables to be used. File access is approved or denied solely based on the feasibility of the proposal—that is, whether the needed data are available on the SSB. There are no requirements for a background review of the applicants. Once approved, access to the server is free. Findings based on the synthetic data do not require disclosure review, because the data are considered public-use.

Because the data have been synthesized, statistical results may differ from the original data. Data users are encouraged to have their findings verified by requesting the Census Bureau to rerun their final programs against the Gold Standard File.¹² According to the Census Bureau, it has received 139 validation requests since 2012 from a total of 275 SSB users since 2007.¹³ It is not known how many of the 139 requests were from unique users. Thus, the number who made validation requests is at most roughly half the number of users. While no data are available on the number of requests between 2007 and 2012, about two-thirds of the recorded validation requests were received in 2017 or later, suggesting a pattern of increased use over time, in which case the number of requests prior to 2012 (when the Census Bureau started keeping records) is likely to have been small. Moreover, given that some users may make multiple validation requests, it appears that many SSB users never make validation requests. A separate data collection conducted by this panel found that 11 of 18 SSB users made validation requests, though the respondents to the data collection request might not be representative of all SSB users.

The process of validation can be burdensome both for data users, who must prepare a package for the Census Bureau, and for the Census Bureau, which must perform the validation run and then review the results for potential disclosure risks. As part of the disclosure review process, the Census Bureau also examines the number of requests submitted by a user, looking at whether the collective set of runs may present a disclosure risk that would

___________________

¹² The Gold Standard File is a file created by the Census Bureau containing the original (nonsynthesized) data as a step toward producing the synthetic data. It is also used to verify whether statistics based on the synthetic data are consistent with those using the original data. It is not a master file of all SIPP original data but rather was created specifically to produce SIPP synthetic data.

¹³ Email communication with the authors from Holly Fee, Census Bureau, Aug. 16, 2022.

not be present in a single run. The total Census Bureau review process typically ranges from 2 to 15 hours; a recent instance that involved a lot of back-and-forth with a user involved about 60 hours of analyst time.¹⁴ The Census Bureau has declared a desire to move to a system in which the validation is fully automated.¹⁵ The validation process also has been subject to disruption. In August 2022, the Census Bureau announced that the Synthetic Beta server operating at Cornell University would be shutting down on September 30, 2022, and that new validation requests would not be accepted after September 2, 2022. The Census Bureau has declared an intention to find another server location, with the location and timing undetermined at the time of this writing.

FSRDCs

A third way to access SIPP data is at FSRDCs. These research data centers are based on partnerships between universities, nonprofit research institutions, and government agencies. As of 2023, SIPP data were available for the 1984–2021 panels.¹⁶ The Census Bureau lists 33 open FSRDCs located throughout the country.¹⁷

The advantages of accessing the data through FSRDCs are that they are more detailed than the public-use files, with no or minimal top-coding, uncollapsed variables, and finer geographic detail, and that they allow access to some linked data, such as SNAP data. The data remain stripped of personally identifiable information, instead having a Protected Identification Key.

Access to FSRDCs requires multiple steps. The researcher must attain Researcher Special Sworn Status, which requires a background check, a lifetime oath of confidentiality, and completion of yearly training to maintain the status. Until the COVID-19 pandemic, to access the data users also were required to be physically present at an FSRDC site. Each of these sites has a secure connection to the Census Bureau’s data facility in Bowie, MD, where special security conditions are in place; for example, no internet access is allowed and printing is strictly controlled. However, a pilot extension allowing approved unpaid Special Sworn Status supporters to work from their personal residences with virtual access to Census Bureau data has now been indefinitely extended,¹⁸ although linked data

___________________

¹⁴ Email communication from Holly Fee, Census Bureau, Sept. 12, 2022.

¹⁵ Briefing by Rachel Shattuck, June 30, 2023.

¹⁶ https://www2.census.gov/programs-surveys/sipp/2015/2015%20PAA%20-Using%20SIPP%20in%20the%20FSRDCs.pdf

¹⁷ https://www.census.gov/about/adrm/fsrdc/locations.html

¹⁸ Jan. 12, 2023, email from Michael Snow, Data Stewardship Executive Policy Committee coordinator, to David Johnson.

from other agencies, such as the SSA and IRS, would be subject to the requirements set by those agencies.

Additionally, there are limits on what analyses may be performed; tabulations are discouraged, with model-based analyses preferred instead. The analyst must submit a proposal that defines the research agenda, reviews Center for Economic Studies Working Papers, states the benefits to the Census Bureau, justifies the data requirement, and provides a project timeline. The Census Bureau reviews the proposal based on scientific merit, the need for nonpublic data, the benefit to Census Bureau programs, feasibility, whether a sensitive population is involved, and the absence of a disclosure risk to an individual or business. The Census Bureau states that approval requires a maximum of 12 weeks unless approval by additional data providers is needed, after which the data requester must obtain a moderate security clearance.¹⁹

The Census Bureau does not charge a fee for use of its data. However, the research organizations that provide access may charge various types of fees.²⁰ For example, the Berkeley FSRDC charges hosting fees of between $15,000 and $20,000 per year for new users.²¹ If an FSRDC is not nearby, researchers may also face transportation costs, though the recent allowing of virtual access for projects using stand-alone SIPP data (i.e., with no administrative record linkages) removes this potential cost.

The process for applying for access was revised and simplified in December 2022 with the creation of a standard application process through an online portal.²²

As with the FSRDC access to SIPP linked to administrative data, the federal agency projects (e.g., SSA, Food and Nutrition Service, and Economic Research Service) use similar agreements or joint statistical projects and operate under similar restricted access and disclosure procedures. The SSA has had special agreements to collect supplemental data for use in its MINT model. It is expected that these will continue under the current agreements.

DISCLOSURE AVOIDANCE IN SIPP FROM THE 1990s TO THE PRESENT

SIPP disclosure avoidance procedures are described in Box 2-2. Separate procedures are considered for data published in tables and for data released in microdata files.

___________________

¹⁹ https://www.census.gov/topics/research/guidance/restricted-use-microdata/standard-application-process.html

²⁰ https://www.census.gov/about/adrm/fsrdc/about/fsrdc-network-fees.html

²¹ https://dlab.berkeley.edu/data/fsrdc-data-and-guidelines

²² See https://www.census.gov/topics/research/guidance/restricted-use-microdata/standard-application-process.html

Administrative Data Added to SIPP

Administrative data are merged with SIPP data for three purposes: to support imputation of missing data, to verify and sometimes correct survey responses, and to provide supplemental data not contained in SIPP data (e.g., the entire earnings history, not limited to the four years covered by SIPP).

No administrative data are directly included in the public-use files; when imputation is performed, the imputed values are based on statistical models rather than on directly inserting the administrative data. SIPP Synthetic Beta (described above) differs by including administrative data in synthetic form, including sometimes by replacing survey values with the values in the administrative data. SIPP data housed at the FSRDCs offer the

most opportunities for merging with administrative data or external data sources. Additional information on these three tiers of access is presented later in this report.

Following are some examples of how data linkages have been used analytically:

• The Census Bureau uses administrative records to impute missing data for selected variables in SIPP.²³ For example, SSA has data on work history, W-2 earnings, self-employed earnings, deferred earnings, job count, benefits received, payment history, and receipt of SSI; the IRS has data on earnings, number of W-2 jobs, deferred earnings, and firm characteristics; and Numident provides birth dates.
• The Census Bureau uses administrative data from SSI to evaluate and correct for respondents’ confusion between SSI and Social Security income, making a combined modeled and logical edit.²⁴
• Capps et al. (2018) combined data from SIPP, which identifies unauthorized immigrants, with data from the ACS, which does not, to examine how many unauthorized immigrants reside in the country, where they reside, and the conditions in which they live.
• Carman et al. (2020) created a synthetic population, combining data from SIPP, the Medical Expenditure Panel Survey, Kaiser Family Foundation/Health Research Educational Trust Employer Health Benefits Survey, ACS, and National Health Expenditure Accounts. They then estimated payments at the individual level for financing health care services and the dollar value of the services received.
• Eggleston and Reeder (2018) combined data from SIPP and IRS 1040 tax data to perform a methodological study measuring the accuracy of respondents’ self-reporting depending on whether or not they consulted financial records, and also looking at the impact in terms of total duration of the SIPP interview.
• Hryshko et al. (2017) combined SIPP and Social Security earnings to examine inequality in the earnings of married couples and the instability of their combined earnings.
• Maples (2017) examined combining SIPP and ACS data to improve state estimates of disability.
• Kim et al. (2018) matched SIPP data with longitudinal earnings records from administrative tax information to assess the predictive power of variables such as occupation, education, and short-term earnings on cumulative earnings over 20 years.

___________________

²³ B. Gurrentz, Model-based imputation & administrative records in SIPP processing, presentation to panel on June 30, 2022.

²⁴ Ibid.

HOW SIPP DATA ARE USED

An earlier National Academies panel commissioned a literature search to determine the topics examined using SIPP data with the intention of assessing the 2014 redesign of SIPP. This panel has a somewhat different focus, being interested in how the selection of topics might affect disclosure avoidance and how the methods used in the statistical analyses affect how the data need to be accessed.

Table 2-2 provides the results from the earlier National Academies study associated with SIPP redesign. Though the bulk of the references concern economic applications, the results show a wide variety of research topics.

It is evident that SIPP data are highly valued for analyzing the U.S. population. The wide variety of topics covered by SIPP and the wide range of studies that have been conducted demonstrate its immense contributions. The three unique features of SIPP described here—the large number of individual and household characteristics, the detail on social program participation, and its longitudinal nature, allowing studies of the evolution of individuals and households—are not available in this combination in any other dataset and are responsible for these contributions.

It is now recognized that these contributions come at an increasing risk of disclosure of private information to outside parties. In the rest of the report, the panel describes the nature of this problem and what steps might be taken to reduce disclosure risk, if necessary, while preserving the main benefits of SIPP and preserving the utility and the usefulness of the data for understanding U.S. society. The panel first discusses how to assess whether the current public-use files released by the Census Bureau run the risk of unacceptably high levels of disclosure, and draws conclusions and recommendations on this issue. The panel then discusses methods for limiting disclosure risk, if and when it is found or believed to be at an unacceptably high level. The panel also discusses methods for providing access to users in alternative ways if they cannot be provided in public-use form. After discussing these topics, the panel ends with a discussion of how to balance the utility and usability of SIPP with methods for reducing disclosure risk.

TABLE 2-2 SIPP Bibliographic References, 2000–2014, by Topic

SOURCE: National Academies of Sciences, Engineering, and Medicine, 2018, Table 3-1, p. 29.

CONCLUSIONS

Conclusion 2-1: The Survey of Income and Program Participation has made major contributions to the understanding of U.S. society and to the evaluation of the effects of government policies across an extremely wide variety of domains.

Conclusion 2-2: There is great value in a public-use data file of the Survey of Income and Program Participation (SIPP), which appears to be the most used mode of SIPP data access, and in including at least the minimum number of variables needed across a broad set of research topics. The lack of a public-use file would result in tremendous loss in research and a reduced number of users, and ultimately it would lower the support for SIPP.

Conclusion 2-3: Accessing restricted data through physical or virtual presence at a Federal Statistical Research Data Center (FSRDC) is extremely time- and process-intensive and costly, making access prohibitive to researchers who are not working in locations with FSRDCs and those who do not have the financial resources to pay fees when required, which results in inequitable access to the data.