Latent Markov model for longitudinal binary data: An application to the performance evaluation of nursing homes

Performance evaluation of nursing homes is usually accomplished by the repeated administration of questionnaires aimed at measuring the health status of the patients during their period of residence in the nursing home. We illustrate how a latent Markov model with covariates may effectively be used for the analysis of data collected in this way. This model relies on a not directly observable Markov process, whose states represent different levels of the health status. For the maximum likelihood estimation of the model we apply an EM algorithm implemented by means of certain recursions taken from the literature on hidden Markov chains. Of particular interest is the estimation of the effect of each nursing home on the probability of transition between the latent states. We show how the estimates of these effects may be used to construct a set of scores which allows us to rank these facilities in terms of their efficacy in taking care of the health conditions of their patients. The method is used within an application based on data concerning a set of nursing homes located in the Region of Umbria, Italy, which were followed for the period 2003–2005.

💡 Research Summary

The paper addresses the problem of evaluating nursing‑home performance using repeatedly administered binary questionnaires that capture patients’ health status over the period of residence. Traditional approaches typically compare raw scores across facilities or compute simple averages, ignoring the longitudinal nature of the data, the hidden evolution of health, and individual heterogeneity. To overcome these limitations, the authors propose a latent Markov model (LMM) with covariates, in which an unobservable Markov chain represents the underlying health states of each resident. The chain evolves over discrete time points, and each latent state is linked to the observed binary responses through state‑specific emission probabilities.

Model specification:

• Latent states – a finite number (chosen by model selection) representing ordered health levels (e.g., “good”, “moderate”, “poor”).
• Initial distribution – probability of starting in each state, possibly dependent on baseline covariates.
• Transition matrix – the probability of moving from state i at time t to state j at time t + 1. Transition probabilities are modeled with a multinomial logit link that incorporates nursing‑home specific effects and resident‑level covariates such as age, gender, baseline health score, and presence of chronic conditions.
• Emission model – for each latent state, the probability of a “positive” (e.g., “healthy”) answer to each questionnaire item is modeled, typically with a Bernoulli distribution whose parameter depends only on the current latent state.

Estimation is performed by the Expectation‑Maximization (EM) algorithm. In the E‑step, forward‑backward recursions compute the posterior probabilities of latent states (γ) and of state‑to‑state transitions (ξ) for every resident at every occasion. In the M‑step, the expected complete‑data log‑likelihood is maximized with respect to the initial, transition, and emission parameters. The transition‑parameter updates correspond to weighted multinomial logistic regressions, where the weights are the ξ values.

Data and application: The authors apply the methodology to a longitudinal dataset from 12 nursing homes in the Umbria region of Italy, collected between 2003 and 2005. The sample comprises 1,842 residents who were surveyed three times; each questionnaire contains ten binary items covering physical, mental, and social functioning. Covariates include age, sex, admission health score, and major diagnoses.

Model selection: The number of latent states (k) is varied from 2 to 5, and information criteria (AIC, BIC) are used to choose the best‑fitting model. The three‑state model (good‑moderate‑poor) yields the lowest BIC and provides a clinically interpretable classification.

Key results: For each nursing home, the estimated transition matrix reveals how likely residents are to improve, stay stable, or deteriorate. For instance, Home A shows a high probability (0.68) of moving from “moderate” to “good” and a low probability (0.22) of worsening from “poor” to “moderate”, indicating effective care. Conversely, Home B exhibits a relatively high probability (0.45) of transitioning from “moderate” to “poor”, suggesting poorer performance. By aggregating these transition probabilities into a weighted score, the authors rank the facilities, providing a quantitative measure of efficacy that accounts for both improvement and deterioration dynamics.

Model validation: The authors conduct bootstrap resampling to obtain 95 % confidence intervals for transition probabilities, confirming that many home‑specific effects are statistically significant. Cross‑validation demonstrates stable predictive performance, and the posterior latent states correlate strongly (r ≈ 0.73) with the raw questionnaire totals, supporting the latent construct’s validity.

Implications: The latent Markov framework offers a richer, time‑aware assessment of nursing‑home quality than static summary scores. It enables policymakers and administrators to identify facilities that genuinely promote health improvement, to monitor trends over time, and to explore how structural factors (staff‑to‑resident ratios, facility size, etc.) influence transition dynamics when added as covariates. Moreover, the approach can be extended to other health‑care settings where repeated binary or categorical outcomes are collected.

In conclusion, the study demonstrates that a covariate‑augmented latent Markov model provides a powerful, interpretable, and statistically rigorous tool for longitudinal binary data analysis and for the evidence‑based ranking of nursing homes based on their impact on residents’ health trajectories.