A Simulation Study to Compare Inferential Properties when Modelling Ordinal Outcomes: The Case for the (Plain but Robust) Proportional Odds Model

Ordinal measurements are common outcomes in studies within psychology, as well as in the social and behavioral sciences. Choosing an appropriate regression model for analysing such data poses a difficult task. This paper aims to facilitate modeling decisions for quantitative researchers by presenting the results of an extensive simulation study on the inferential properties of common ordinal regression models: the proportional odds model, the category-specific odds model, the location-shift model, the location-scale model, and the linear model, which incorrectly treats ordinal outcomes as metric. The simulations were conducted under different data generating processes based on each of the ordinal models and varying parameter configurations within each model class. We examined the bias of parameter estimates as well as type I error rates ($α$-errors) and the power of statistical parameter testing procedures corresponding to the respective models. Our findings reveal several highlights. For parameter estimates, we observed that cumulative ordinal regression models exhibited large biases in cases of large parameter values and high skewness of the outcome distribution in the true data generation process. Regarding statistical hypothesis testing, the proportional odds model and the linear model showed the most reliable results. Due to its better fit and interpretability for ordinal outcomes, we recommend the use of the proportional odds model unless there are relevant contraindications.

💡 Research Summary

The paper conducts a comprehensive simulation study to evaluate five regression approaches for ordinal outcomes: the proportional odds model (PO), the category‑specific odds model (CSO), the location‑shift model (LSH), the location‑scale model (LSC), and the ordinary linear model (LM) that treats the ordinal variable as continuous. Drawing on a literature review of 458 recent articles, the authors note that PO is by far the most frequently employed method, while many studies either omit model specifications or rely on linear models despite the ordinal nature of the data.

Simulation conditions vary sample size (n = 250, 500, 1 000), number of outcome categories (k = 3, 5, 7), number of covariates (p = 5 or 35), and the proportion of truly informative predictors (0, 1, 4). Parameter values for location effects (β) and dispersion effects (γ) span from 0 to 2, with both positive and negative signs for CSO. Three threshold configurations (uniform, skewed, unstructured) generate different underlying outcome distributions, allowing the authors to probe model behavior under symmetric and highly asymmetric conditions. Each of the 4 032 design points is replicated 2 000 times, and model fit failures (e.g., categories with fewer than five observations or violations of monotonic cumulative probabilities) trigger data regeneration.

Results show that cumulative models (PO, LSH, LSC) suffer increasing bias when β is large or when the outcome distribution is strongly skewed. CSO exhibits the greatest bias and the highest non‑convergence rates (over 20 % of fits fail in many settings) because it estimates a separate coefficient vector for each category. LSH and LSC, which incorporate dispersion parameters, display modest bias and slightly lower power than PO but remain more stable than CSO. In terms of inferential performance, PO and LM achieve the most reliable Type I error control (≈ 0.05) and high power (0.8–0.95 for non‑zero effects). LM’s performance improves with larger samples, narrowing the gap with PO despite its misspecified continuous‑outcome assumption. LSH and LSC provide modest gains when true dispersion effects exist, yet their added complexity can exacerbate convergence problems in small‑sample, high‑category scenarios.

Overall, the study concludes that the proportional odds model offers the best trade‑off between simplicity, interpretability, bias, and inferential reliability, justifying its recommendation as the default choice for ordinal outcomes unless substantive theory dictates the need for more elaborate structures. Researchers are advised to conduct exploratory checks of outcome distribution and to be cautious when employing highly parameter‑rich models such as CSO or LSC, especially in settings with limited sample size or many outcome categories.