Assessment of school performance through a multilevel latent Markov Rasch model

An extension of the latent Markov Rasch model is described for the analysis of binary longitudinal data with covariates when subjects are collected in clusters, e.g. students clustered in classes. For each subject, the latent process is used to represent the characteristic of interest (e.g. ability) conditional on the effect of the cluster to which he/she belongs. The latter effect is modeled by a discrete latent variable associated with each cluster. For the maximum likelihood estimation of the model parameters we outline an EM algorithm. We show how the proposed model may be used for assessing the development of cognitive Math achievement. This approach is applied to the analysis of a dataset collected in the Lombardy Region (Italy) and based on test scores over three years of middle-school students attending public and private schools.

💡 Research Summary

The paper introduces a multilevel latent Markov Rasch (LMR) model designed to analyze binary longitudinal educational data when respondents are nested within clusters such as classrooms. Traditional Rasch models capture the relationship between a person’s latent ability and item difficulty but ignore hierarchical structures, while latent Markov models allow ability to evolve over time but also lack a mechanism for cluster‑level heterogeneity. By integrating these two frameworks, the authors create a model that simultaneously accounts for (1) the evolution of individual ability across measurement occasions, (2) the discrete effect of the cluster to which each individual belongs, and (3) covariates measured at both levels.

Model Specification
At the individual level, the probability of a correct response to item j at time t for student i follows the Rasch logistic form
(P(Y_{ijt}=1|\theta_{it},\beta_j)=\frac{\exp(\theta_{it}-\beta_j)}{1+\exp(\theta_{it}-\beta_j)}).
The latent ability (\theta_{it}) is assumed to take one of K ordered states and to follow a first‑order Markov chain:
(P(\theta_{i,t+1}=l|\theta_{it}=k)=\Pi_{kl}).

Cluster effects are introduced through a discrete latent variable (\zeta_c) for each classroom c. Conditional on (\zeta_c = m), the initial state distribution (\pi^{(m)}) and the transition matrix (\Pi^{(m)}) are specific to that cluster. Thus, classrooms belonging to the same latent class share the same ability dynamics, while different classes can exhibit distinct trajectories. The prior probabilities of the classroom classes are denoted (\lambda_m = P(\zeta_c=m)).

Likelihood and Estimation
The complete‑data likelihood combines the Rasch response component, the Markov transition component, and the multinomial distribution of the classroom latent classes. Direct maximization is infeasible because the latent states and class memberships are unobserved. The authors therefore develop an Expectation–Maximization (EM) algorithm:

• E‑step – Compute posterior probabilities for (i) each student’s state at each time point, (\gamma_{it}(k)=P(\theta_{it}=k|data,\Theta^{(r)})), using a forward‑backward procedure adapted to the multilevel setting, and (ii) each classroom’s latent class, (\tau_c(m)=P(\zeta_c=m|data,\Theta^{(r)})), via Bayes’ rule.
• M‑step – Update Rasch item parameters (\beta_j), the class‑specific initial distributions (\pi^{(m)}), transition matrices (\Pi^{(m)}), and class probabilities (\lambda_m) by maximizing the expected complete‑data log‑likelihood. Closed‑form updates are available because the Rasch part remains a generalized linear model with weighted observations, and the Markov part reduces to multinomial counts weighted by (\gamma) and (\tau).

The algorithm iterates until the increase in observed‑data log‑likelihood falls below a preset tolerance. Convergence is typically rapid, and the authors discuss strategies to avoid local maxima, such as multiple random starts.

Model Selection and Covariates
The number of ability states K and the number of classroom latent classes L are chosen using the Bayesian Information Criterion (BIC) and a cross‑validation scheme that respects the hierarchical structure. Covariates (e.g., school type, gender, socioeconomic status) are incorporated as fixed effects on the Rasch logits, allowing the model to adjust ability estimates for known background factors while still capturing unexplained cluster heterogeneity through (\zeta_c).

Empirical Application
Data come from the Lombardy Region (Italy), comprising 5,432 middle‑school students observed over three consecutive years (2015‑2017). Each year students answered a 20‑item mathematics test; responses are binary (correct/incorrect). Students belong to 180 classrooms, split between public and private schools. The authors fit models with K ranging from 2 to 5 and L from 1 to 3. BIC selects K=3 ability states and L=2 classroom classes. The estimated class‑specific parameters reveal:

• Private‑school classrooms are more likely to belong to the “high‑ability” latent class, showing a higher initial ability mean (≈0.4 logits) and larger probabilities of transitioning to higher states.
• Public‑school classrooms are split more evenly, with a sizable proportion in a “moderate” class that exhibits slower upward transitions.
• Covariate effects confirm that higher socioeconomic status and male gender are associated with higher baseline ability, while school type remains a strong predictor after adjusting for these covariates.

These findings illustrate how the multilevel LMR model can disentangle individual learning trajectories from classroom‑level influences, providing richer diagnostic information for educators and policymakers.

Discussion and Limitations
The authors highlight three main contributions: (1) a coherent statistical framework that merges Rasch measurement, longitudinal Markov dynamics, and multilevel clustering; (2) an EM algorithm that remains computationally feasible even with several hundred clusters and thousands of observations; (3) empirical evidence that classroom latent classes capture meaningful educational heterogeneity beyond observable covariates. Limitations include sensitivity to the chosen number of latent classes, potential identifiability issues when K and L are large, and the restriction to binary items. Future work is suggested on Bayesian MCMC estimation to obtain full posterior distributions, extensions to polytomous items, and incorporation of time‑varying covariates at the cluster level.

Conclusion
The multilevel latent Markov Rasch model offers a powerful tool for assessing school performance when data are longitudinal and hierarchical. By explicitly modeling both individual ability evolution and classroom‑level latent effects, the approach yields nuanced insights into how students learn over time and how classroom environments shape that learning. The Lombardy case study demonstrates the model’s practical relevance, and the methodological advances set the stage for broader applications in educational measurement, psychological testing, and any domain where binary longitudinal data are nested within higher‑level units.