Correcting for Nonignorable Nonresponse Bias in Ordinal Observational Survey Data

Many political surveys rely on post-stratification, raking, or related weighting adjustments to align respondents with the target population. But when respondents differ from nonrespondents on the outcome itself (nonignorable nonresponse), these adjustments can fail, introducing bias even into basic descriptives.We provide a practical method that corrects for nonignorable nonresponse by leveraging response-propensity proxies (e.g., interviewer-coded cooperativeness) observed among respondents to extrapolate toward nonrespondents, while directly integrating observable covariates and retaining the benefits of post-stratification with known population shares. The method generalizes the variable-response-propensity (VRP) framework of Peress (2010) from binary to ordinal outcomes, which are widely used to measure trust, satisfaction, and policy attitudes. The resulting estimator is computed by maximum likelihood and implemented in a compact R routine that handles both ordinal and binary outcomes. Using the 2024 American National Election Study (ANES), we show that accounting for nonignorable nonresponse produces substantively meaningful shifts for life satisfaction (estimated latent correlation $ρ\approx 0.49$), while yielding negligible changes for retrospective economic evaluations ($ρ\approx 0$), highlighting when nonignorable nonresponse substantively affects survey estimates.

💡 Research Summary

This paper tackles the pervasive problem of nonignorable nonresponse in political and social surveys, where the probability of responding is correlated with the outcome of interest. Traditional weighting adjustments such as post‑stratification or raking align respondents with known population margins on demographic variables, but they cannot eliminate bias when respondents differ systematically from nonrespondents on the substantive variable itself. To address this, the authors extend the Variable‑Response‑Propensity (VRP) framework originally proposed by Peress (2010) from binary outcomes to ordinal outcomes, which are common in survey research (e.g., life‑satisfaction scales, policy‑evaluation Likert items).

The core idea is to exploit a response‑propensity proxy that is observed for respondents but missing for nonrespondents. In the ANES context, the proxy is the interview‑coded cooperativeness rating, a paradata measure that reflects the respondent’s willingness to engage. The authors embed this proxy in a joint ordered‑probit model for the outcome and the proxy. Let yₙ be an ordinal outcome taking values 1,…,Y and rₙ be the ordinal proxy taking values 1,…,R for respondents; nonresponse is coded as a special category R+1. The latent continuous variables yₙ = αᵀxₙ + εₙ and rₙ = βᵀzₙ + ηₙ are linked to the observed categories through threshold parameters γ and θ. Crucially, the error terms (εₙ, ηₙ) are assumed bivariate normal with unit variances and correlation ρ. The parameter ρ captures the degree of nonignorable selection: a positive ρ indicates that higher (or lower) outcome values are associated with lower propensity to respond, leading to systematic under‑ or over‑representation of certain outcome levels among nonrespondents.

Identification relies on two ingredients. First, the proxy must exhibit sufficient dispersion across respondents, allowing the model to learn how outcome distributions vary with response propensity. Second, the set of covariates zₙ used in the response equation should contain at least one variable that influences response propensity but does not directly affect the outcome (an exclusion restriction). In the empirical illustration, interview cooperativeness serves as the proxy, while demographic variables (marital status, spouse gender, race, education) define post‑stratification cells with known population shares p_zₖ.

Maximum‑likelihood estimation proceeds by constructing a log‑likelihood that combines (i) the contribution of observed respondents, which involves a double integral over the bivariate normal density, and (ii) the contribution of nonrespondents, which reduces to a single integral over η because yₙ is missing. The total number of nonrespondents N_miss is either taken from the known sample‑frame response rate or treated as a sensitivity parameter. Standard errors are obtained via the delta method using a numerically evaluated Jacobian.

The method is implemented in a compact R package that extends Peress’s original C++ code to handle ordinal outcomes. Computational demands are modest; the full ANES application runs in about five minutes on a standard laptop.

Empirically, the authors apply the estimator to the 2024 American National Election Study (ANES) with roughly 3,000 respondents and an overall response rate of about 50 %. Two survey questions are examined: (1) “How satisfied are you with life?” (5‑point ordinal scale) and (2) “Has the national economy gotten better or worse?” (4‑point ordinal scale). For the life‑satisfaction item, the distribution of the cooperativeness proxy varies systematically with satisfaction: respondents who are less satisfied tend to give lower cooperativeness ratings. The estimated correlation ρ is about 0.49, indicating substantial nonignorable nonresponse. Sensitivity analyses assuming 20 %, 50 % and 70 % nonresponse rates show that the adjusted population proportions shift markedly, especially inflating the share of “not satisfied at all” when the nonresponse rate is high. By contrast, the economic‑evaluation item shows virtually no monotonic relationship between the proxy and the outcome; the estimated ρ is near zero (≈ −0.001), and the adjusted estimates are indistinguishable from the raw survey‑weighted figures. This contrast demonstrates that the value of nonresponse correction is outcome‑specific and can be empirically assessed.

Methodologically, the contribution has three main strengths. First, it preserves the information contained in ordinal scales, avoiding the loss of nuance that occurs when collapsing to binary variables. Second, it integrates seamlessly with existing post‑stratification, so that known population margins are still honored while correcting for selection on the outcome. Third, the R implementation makes the approach accessible to applied researchers. Limitations include reliance on the bivariate normal latent‑error assumption, the need for a well‑behaved proxy with sufficient variation, and the dependence on an exclusion restriction for identification. The authors suggest extensions such as non‑Gaussian latent distributions, Bayesian priors for ρ, and the use of multiple proxies to strengthen identification.

In sum, the paper provides a practical, theoretically grounded, and computationally tractable solution for correcting nonignorable nonresponse bias in ordinal survey data. By making the extrapolation from respondents to nonrespondents explicit, transparent, and easy to implement, the ordinal VRP estimator equips researchers with a valuable tool for sensitivity‑aware descriptive inference in the many surveys that rely on ordinal measures.