8

Disclosure Limitation Approaches: Geography Variables

As discussed in earlier chapters, risks of disclosure increase when respondents can be broken down into very small subgroups. This can happen when one combines many different measures together but especially is a risk when a single measure is sufficient to create very small subgroups. Measures of geography are a notable source of risk because they are highly identifiable and they narrow down the sample and the overall population to much smaller subgroups. For example, through a re-identification study the Census Bureau identified disclosure risks from the geography measures used in the Current Population Survey, requiring changes to the measures (Benedetto et al., 2022).

Recognizing these risks, the Census Bureau limits the amount of geographic data available through the Survey of Income and Program Participation (SIPP; more strongly than it limits these data in the Current Population Survey). As is standard for confidential surveys, home addresses are automatically suppressed, as are the names of counties and specific metropolitan areas. In fact, while SIPP includes a measure indicating whether the respondent is in a metropolitan area, that measure is used only when a state has multiple metropolitan areas. SIPP does include state identifiers for all the states. However, as was discussed in Chapter 3, the inclusion of state identifiers can be highly disclosive.

The challenge from a data usability perspective is that geography variables can be highly valuable for research. For example, variations in state or local policies or programs create a type of natural experiment that researchers can use to estimate the impacts of those policies or programs. There also are naturally occurring differences that appear across states or regions that

can be meaningful to researchers, such as the common differentiation in political dialogue between “red” states and “blue” states.

Different types of solutions can be developed depending on the specific needs of a research project. For example, some data users may only need to distinguish one state from another without actually identifying the specific states. Other researchers may be interested in identifying geographic areas but again without the need to identify specific states. Still other data users will need the ability to identify specific states. Thus, the panel considered three use case scenarios in developing potential strategies for providing geographical variables and limiting disclosure.

USE 1: IDENTIFYING SPECIFIC GEOGRAPHIES IS SOMETIMES UNNECESSARY

Although geography is a natural variable used for producing small area estimates, it is not strictly needed to produce the national-level regression analyses that are often sought by data users, in which the inferential task does not involve the geography itself. Instead, for these analyses it is sufficient to release public-use microdata with a “mask-id” (i.e., a “group-level” identifier) rather than a state-level identifier. Releasing a mask-id would allow researchers to incorporate this variable for analysis as either a fixed or random effect in a regression model. In other words, the inclusion of a mask-id variable as either a fixed or random effect would provide the same benefit as including a state-level (or other geographic-level) identifier but with the benefit of not necessarily being disclosive. It is envisioned that proceeding in this way will allow data users to produce national-level estimates for a subset of analyses associated with specific subpopulations.

Still, some states may be identifiable even with a mask-id. For example, the 10 largest states can each be identified by assuming that the number of respondents (or the sum of the sampling weights) in SIPP have the same rank ordering as the state population sizes. If the state-level id is determined to be a disclosure risk, then it may not be viable to simply associate a mask-id with each respondent. In this case, another potential path forward would be to consider partially synthetic data, where the state-level id (or mask-id) is synthesized for each record in the public-use file. Although there has been some research on generating synthetic geographies (Quick et al., 2015; Wang & Reiter, 2012), this is typically conducted to produce a synthetic location for each respondent. Instead, generating a synthetic state-level id (or mask-id) for each respondent would proceed from a latent class model and require additional research. In principle, this type of partial synthesis could also proceed using formal privacy methods. See Chapter 6 and the references therein for additional discussion on synthetic data methodology.

USE 2: MAKING SUBNATIONAL ESTIMATES

Sample surveys are designed to produce reliable estimates over large geographies and for populations and subpopulations with sufficient sample sizes. Nevertheless, there is an increasing demand for reliable estimates at more granular levels of geography and for smaller subpopulations. To meet this demand, modeling is often utilized as a means of producing estimates of sufficient accuracy that would not be possible based solely on a sample survey.

In many cases, small geographies or subpopulations have insufficient sample sizes (or no sample) to produce reliable estimates. In these cases, model-based approaches allow for the “borrowing of strength” by leveraging different sources of dependence and by incorporating auxiliary information. Therefore, estimates can be achieved for additional domains of interest, with the estimated quantities typically having increased precision. The Census Bureau has a rich history of running various small area estimation (SAE) programs. Arguably the two most notable of these programs are the Small Area Income and Poverty Estimates and the Small Area Health Insurance Estimates. Both programs use Bayesian model-based methodology to provide estimates of increased precision at under-sampled geographies.

To produce a broad range of state-level estimates for various subpopulations, a robust model-based SAE program is desired. To achieve this goal the Census Bureau can proceed using either an area- or unit-level approach (Parker et al., 2023b; Rao & Molina, 2015). In either case, publishing estimates at a high enough level of aggregation could allow for dissemination without the need for extra noise infusion prior to release (Reiter, 2012) to reduce disclosure risk.

Area-level model-based approaches often proceed using a Bayesian hierarchical framework. A now standard formulation for the Bayesian hierarchical model conditionally expresses the model as having three components: the data model (or likelihood), the process model, and the parameter model (e.g., see Cressie & Wikle, 2011, and the references therein). The advantage to using a Bayesian hierarchical model in this context resides in its ability to accommodate a broad range of modeling tasks and to provide meaningful measures of uncertainty. The well-known Fay-Herriot model (Fay & Herriot, 1979) is a special case of the multivariate spatial mixed effects model, where the process model’s random effects are independent and identically distributed (see Appendix D for the model formulation). The Fay-Herriot model is widely used and forms the basis for the model used in Small Area Income and Poverty Estimates¹ (Bell et al., 2016).

___________________

¹ https://www.census.gov/programs-surveys/saipe.html

Importantly, in both the Gaussian and non-Gaussian cases, the Bayesian hierarchical model framework is general and provides the needed flexibility for customization to SIPP. The main considerations in developing area-level models include whether to use multivariate or univariate modeling, to incorporate geographical (spatial) or temporal (longitudinal) dependence, to incorporate auxiliary information (e.g., administrative records and/or data from other surveys), and to deal with potentially non-Gaussian responses. Importantly, in the development of area-level models for specific data products, such as SIPP, many of these issues are not mutually exclusive.

Note that geographical (spatial) dependence may be defined in multiple ways (Nandy et al., 2023). A primary focus in this chapter is to look at states because states are often of interest in policy research (e.g., based on variations in policies across states) and because state is the most detailed clearly identifiable measure of geography incorporated in SIPP. However, other types of geographic dependence are of interest, including at a more detailed level (such as metropolitan areas) and at a more aggregated level (such as geographic regions). The Census Bureau has a definition of geographic region that it often applies,² and there often are commonalities within regions that help to distinguish them from one another (e.g., level of education, racial/ethnic composition, and poverty levels). If such measures are used to define neighbor relationships, then one has removed some identifying (state-level) information from the models and provided additional protection from disclosure.

Area-level models usually begin with the response variable consisting of the direct estimate (design-based estimate). Along with this estimate, the statistical agency (e.g., the Census Bureau) typically publishes an estimate of the sampling error variance. That is, in the Gaussian case, the response is given by the direct estimate, with the observation-level (data model) variance usually assumed to be known and taken to be the sampling error variance. Consequently, in this setting the proposed model incorporates the survey design by construction.

As an example, consider state-level estimates of wealth. These could be produced using a straightforward Fay-Herriot model, where the response variable is a direct estimate (design-based estimate) of state-level median household wealth regressed on the latent “true” state-level median wealth plus an independent error with known sampling error variance. In other words, a noisy version of true state-level median household wealth constitutes the response. At the next level in the model hierarchy, the “true” median household wealth is modeled conditional on state-level demographic variables and other wealth-related covariates. At this stage in the model hierarchy, a state-level random effect could be included to improve the model predictions.

___________________

² https://www2.census.gov/geo/pdfs/maps-data/maps/reference/us_regdiv.pdf

The main goal for SAE within a statistical agency is to improve the precision of the estimates being disseminated and to provide estimates that may not otherwise be possible. As such, it is advantageous to leverage various sources of dependence and to incorporate auxiliary information. One significant source of dependence arises from multivariate relationships in the data. There are several ways to incorporate multivariate dependence depending on the specific problem being considered. For the Gaussian case, one can consider a model where the responses are assumed to be conditionally independent given the covariates (Wang et al., 2012). That is, the covariance matrix of the observations does not contain any sampling error covariance. Instead, the multivariate dependence between responses is incorporated in the process stage of the model hierarchy. When the errors in the process stage only contain multivariate dependence, this model can be viewed as a multivariate Fay-Herriot model (Benavent & Morales, 2016; Porter et al., 2015).

Spatio-temporal Models

Herein, the panel describes multivariate spatio-temporal models and notes that Fay-Herriot models, univariate models, and/or spatial-only models can be considered as special cases. In the Gaussian case, one can proceed using a multivariate spatio-temporal mixed-effects model (Bradley et al., 2015). The basic formulation is given in Appendix D. See Bradley et al. (2015) for additional details. Models for non-Gaussian data, such as data from the natural exponential family, can be efficiently accommodated using conjugate distribution theory (Bradley et al., 2020), data augmentation (Polson et al., 2013), and/or specialized software such as RStan, INLA, or NIMBLE, depending on the specific problem (de Valpine et al., 2017; Rue et al., 2009; Stan Development Team, 2023).³

Revisiting the example of state-level median wealth income, a spatial or multivariate spatial model could be considered. In the case of a spatial model, the state-level random effect can be replaced by a conditional autoregressive model or intrinsic conditional autoregressive (ICAR) model (e.g., see Banerjee et al., 2014; Cressie & Wikle, 2011; and the references therein). Similarly, given an outcome correlated with state-level median wealth (also from SIPP), a multivariate or multivariate spatial model could be specified. The difference would reside in the specification of the fixed and random effects at the process level of the model hierarchy (e.g., see Bradley et al., 2015; Cressie & Wikle, 2011, for additional details).

Again, in the case of a Gaussian response, typically the observation variance is considered fixed and known and taken to be the sampling error

___________________

³ See also RStan: the R interface to Stan. R package version 2.21. 8, https://mc-stan.org/

variance. However, this need not be the case. Instead, one can simultaneously model the mean and the variance (e.g., see Parker et al., 2023a, and the references therein). In particular, Parker et al. (2023a) provide a computationally efficient example of leveraging spatial correlation and covariate information in the model for the variance. Although there has been some research focused on shrinking both the mean and the variance, this research is relatively sparse by comparison.

When modeling non-Gaussian data using an area-level model, the question arises as to how to effectively account for the sampling design. One reason this problem arises is because the data model (likelihood) does not present a convenient form for incorporating the sampling error variance, in contrast to the Gaussian data model case. For the binomial likelihood, one can utilize the effective sample size and effective probability of success based on the design effect, the ratio of the variance under the survey design and under simple random sampling (Malec et al., 1999). Mohadjer et al. (2012) use a nonlinear linking model for modeling proportions that lead to unmatched sampling and linking models. For the Poisson likelihood, one can utilize the equidispersion property and treat both the sampling error variances and direct estimates as separate sources of data; see Bradley, Wikle et al. (2016) for further detail.

Modeling at the Unit Level

Another framework for SAE considers modeling at the unit level (e.g., see Parker et al., 2023b, and the references therein). These models rely directly on answers provided by individual survey respondents as the dependent variable and can resolve some of the disadvantages associated with area-level models. Specifically, by starting with data at the record level of the survey as the response variable (i.e., person-level or household-level in the case of SIPP), the method can be considered a “bottom-up” approach. In other words, in principle, predictions can be made for any level of geography (or subpopulation) and aggregated up to any desired tabulation level, such as county, state, or national. In contrast to area-level modeling, there is no need for so-called benchmarking, where lower-level estimates are forced to aggregate to higher-level estimates as a means of achieving internal consistency among estimates at various tabulation levels (Bell et al., 2013). Additionally, because unit-level models directly leverage the entire dataset, rather than summary-level statistics (e.g., direct estimates), they are potentially more precise than estimates coming from an area-level model (Hidiroglou & You, 2016).

Even though there are several advantages to unit-level modeling, they also present several challenges. First, unit-level models are often

non-Gaussian (e.g., binary or categorical) and, compared with area-level models, can be extremely high-dimensional. Taken together, this leads to a computationally intensive prediction problem. Perhaps more crucially, complex sample survey designs such as those in SIPP typically lead to informative samples (i.e., the probability of sample selection is correlated with the response). Importantly, if left unaddressed, the issue of informative sampling can lead to biased estimates.

In cases where the design is noninformative (e.g., a simple random sample), the distribution of the sampled responses is the same as the nonsampled responses after conditioning on covariates. As such, the basic unit-level model introduced by Battese et al. (1988) can be used (see Appendix D). This approach constitutes an unweighted model-based analysis.

Supposing that the survey design is informative, Gelman and Little (1997) describe a general approach to Bayesian unit-level modeling that conditions on the design variables. The idea is that if all the design variables are accounted for in the model, then the conditional distribution of the response, given the covariates, is independent of the probability of selection. Also, by using a Bayesian unit-level approach, the unsampled population can then be generated through the posterior predictive distribution. Consequently, a distribution can be obtained for any finite population quantity and, thus, estimates and appropriate measures of uncertainty can be produced.

In practice, conditioning only on the design variables may be impractical, because not all the design variables may end up being accounted for and the functional relationship is typically unknown. Therefore, other methods to account for informative sampling are desired, leading to models with survey weight adjustments.

One popular approach for handling informative sampling is the Bayesian pseudo-likelihood (Savitsky & Toth, 2016). This model extends the pseudo-likelihood of Binder (1983) and Skinner (1989) to the Bayesian setting. This extension essentially replaces the likelihood by an exponentially weighted likelihood (see Appendix D).

Cast within a Bayesian hierarchical framework, the pseudo-likelihood-based model results in a pseudo-posterior from which estimated model parameters can be obtained. Then, given the estimated model parameters, it is possible to generate estimates and measures of uncertainty for finite population quantities using post-stratification (Gelman & Little, 1997; Park et al., 2006).

Another popular approach to unit-level modeling proceeds by regressing on the survey weights. For example, Si et al. (2015) introduce a unit-level approach that addresses several issues encountered with unit-level modeling, namely lack of knowledge regarding the covariates, sampling weights, and population sizes associated with the nonsampled units.

Using a multinomial distribution, the proposed methodology models the observed post-stratification cells conditional on the population size.

Incorporating Spatial Dependence and Correlated Random Effects

The unit-level models previously discussed offer several avenues for incorporating spatial dependence. The most straightforward approach for incorporating spatial dependence is to incorporate an incidence vector (i.e., a vector that equals 1 if the i_th unit is in area i and 0 otherwise) in the random-effect component of the underlying process model (Sun et al., 2022). This vector maps observations to specific geographies, providing an extra source of variation for units within the same geography. The specification above can also account for more complex spatial dependence using spatial basis functions in the random-effect or by using a conditional autoregressive or ICAR model. See Parker et al. (2023b) and the references therein for additional detail. Area-level spatial dependence relies on the notion that units in neighboring geographic areas are correlated, whereas units in distant geographic areas are uncorrelated. For example, Vandendijck et al. (2016) propose an approach similar to Si et al. (2015) that incorporates spatial dependence using an ICAR model.

In the case of a Gaussian response, adding spatially correlated random effects is relatively straightforward. In addition, the resulting formulation is computationally feasible, even in moderate to high-dimensional settings. In contrast, for the non-Gaussian case, estimation can be computationally intensive due to non-conjugacy in the model specification. For the Poisson, binomial, and multinomial cases, methods have been proposed to circumvent this issue (e.g., see Parker et al., 2020, 2023a).

Similar to the area-level modeling case, another issue to be considered when building small area statistical models is whether to specify multiple univariate models or, alternatively, to leverage multivariate dependence. In general, characterizing multivariate dependence can be challenging, especially when moving beyond bivariate relationships. Although there has been some research for multivariate modeling at the unit level (e.g., see Parker et al., 2022), model development will directly depend on the survey design associated with SIPP and the specific needs of the data user (i.e., the most frequent use cases). In particular, the survey design associated with SIPP produces several sources of temporal dependence. At the unit level, one way to incorporate temporal dependence is to have time-varying model parameters. Deciding which model parameters are allowed to vary over time would necessitate an exploratory analysis and (or) subject matter expertise.

Leveraging Auxiliary Data and Applying Disclosure Avoidance

In both area- and unit-level modeling, leveraging auxiliary data using data integration methodology can often lead to better estimates (e.g., see Bradley, Holan et al., 2016; Lohr & Raghunathan, 2017). Sources of auxiliary data can include other surveys (e.g., the American Community Survey) or administrative records (Maples, 2017). In the case of integrating multiple surveys, several issues apply. At the area level, it is important to determine whether the tabulations are definitionally consistent and what the coverage of each tabulation is. At the unit level, in addition to definitional consistency, there may be issues associated with record linkage. Also at the unit level, administrative records may be used as a source of auxiliary data, though issues associated with coverage and record linkage may arise (Reiter, 2021).

Adding disclosure avoidance methodology to SAE is an important area of research. One viable path forward, as described previously, is to produce model-based predictions using the direct estimates on a large enough domain. Since this would constitute an indirect estimate that uses the entire dataset, in principle, the risk of disclosure would be significantly reduced relative to the direct estimator. This approach would lack the formal privacy guarantees associated with differential privacy (Janicki et al., 2022), similar to many of the legacy disclosure avoidance methods applied to direct estimates such as top-coding, swapping, and suppression, among others (see Chapter 4).

Implementing formal privacy methods in the SAE setting raises several research questions. The first question is at what stage the differential privacy noise is to be added. Unlike the case with the 2020 decennial census (Janicki et al., 2023), adding differential privacy noise to the direct estimates and then modeling the protected data results in a response variable with two sources of error (i.e., from differential privacy and sampling), thus making it more difficult to specify the statistical model. Similar issues arise from adding the differential privacy noise to model-based predictions. In addition, implementing differential privacy in either case requires specifying the privacy-loss budget in the presence of longitudinal dependence and constitutes an open area of research.

For data coming from a complex survey, a promising direction is to generate differentially private synthetic data or use administrative records to generate a synthetic SIPP/administrative records dataset (e.g., see Chapter 6). Given the synthetic data, small area estimates can be produced. Conveying to data users the methods for accounting for the synthesis uncertainty of the synthetic data would be helpful when constructing small area estimates.

USE 3: IDENTIFYING SPECIFIC STATES OR LOCALITIES

A third situation occurs when there is a need to identify specific states or localities, such as when the data user wishes to measure the impact of a policy or program by making use of differences across states or localities (e.g., Dondero & Altman, 2020; Ribar, 2005). In such situations, data users will wish to access the raw data on geography. Either secure online data access (SODA) or use of a Federal Statistical Research Data Center (FSRDC) will provide another path forward for producing small area estimates. In conjunction with the public-use files, SODA and FSRDCs can be thought of as a tiered system of access, where the specific “tier” depends on the desired analysis.

At a high level, SIPP access could be viewed as follows. Public-use files would contain “protected” data with access to limited variables. SODA would allow researchers access to a less restrictive version of SIPP that does not require any auxiliary data sources. Finally, the FSRDC would allow full access to the data and the possibility of linking additional data sources (comingled data) as needed (e.g., linking SIPP with administrative records). Dissemination of small area estimates obtained through SODA or an FSRDC would require some form of disclosure review prior to release. For a comprehensive discussion regarding SODA and FSRDCs, see Chapter 5.

RECOMMENDATION

Recommendation 8-1: The Census Bureau should continue to pursue the development of a small area estimation program to meet the needs of Survey of Income and Program Participation users for geography-based analysis that preserves confidentiality and limits disclosure risk.