On the history and use of some standard statistical models

This paper tries to tell the story of the general linear model, which saw the light of day 200 years ago, and the assumptions underlying it. We distinguish three principal stages (ignoring earlier more isolated instances). The model was first proposed in the context of astronomical and geodesic observations, where the main source of variation was observational error. This was the main use of the model during the 19th century. In the 1920’s it was developed in a new direction by R.A. Fisher whose principal applications were in agriculture and biology. Finally, beginning in the 1930’s and 40’s it became an important tool for the social sciences. As new areas of applications were added, the assumptions underlying the model tended to become more questionable, and the resulting statistical techniques more prone to misuse.

💡 Research Summary

The paper offers a historical and methodological review of the general linear model (GLM), tracing its evolution from its inception in the early nineteenth century to its widespread use across diverse scientific disciplines. The author divides this development into three principal stages, each characterized by a distinct application domain and a corresponding shift in the plausibility of the model’s underlying assumptions—most notably the normality, independence, and identical distribution (i.i.d.) of error terms.

Stage 1 – Astronomical and Geodetic Observations (c. 1805‑1900).
The story begins with Legendre’s 1805 publication of the method of least squares, which formalized the idea that an observation can be expressed as a deterministic linear combination of explanatory variables plus a random error. In the context of astronomy and geodesy, the dominant source of variation was measurement error, and empirical evidence strongly supported the “Law of Errors,” i.e., that these errors were approximately normally distributed. This empirical regularity was justified theoretically by the law of elementary errors (the sum of many small independent disturbances) and the Central Limit Theorem. Consequently, the normal i.i.d. error assumption was regarded as both a theoretical necessity and a practical reality.

Stage 2 – Agricultural and Biological Experiments (1920s‑1930s).
R. A. Fisher extended the GLM to the analysis of agricultural yields, plant breeding trials, and other biological experiments. Here the variability of the experimental units (e.g., plants, plots, animals) entered the model alongside measurement error. Fisher assumed that the underlying population distribution of the trait of interest was normal, a premise that was far weaker than the earlier measurement‑error case. The paper details the well‑known controversy between Fisher and E. S. Pearson (and later G. Gosset) concerning the explicitness of the normality assumption. Pearson criticized Fisher for presenting tests (especially the F‑test for variances) as “exact” while providing insufficient warning about their sensitivity to departures from normality. Fisher’s own correspondence reveals that he recognized the limitation but defended the approach on the basis of empirical robustness for tests of means, while acknowledging that variance tests are highly vulnerable to non‑normality.

Stage 3 – Social‑Science Applications (1930s‑1940s onward).
The GLM was subsequently adopted by economists, sociologists, and psychologists. Social data frequently exhibit skewness, heavy tails, and heteroscedasticity, violating the normal i.i.d. error assumption. Although Karl Pearson’s family of distributions (gamma, beta, chi‑square, etc.) offered alternatives, the standard linear model continued to be applied with its original normality premise. The author argues that this led to a systematic “misuse” of statistical techniques: researchers often failed to test model assumptions, interpreted p‑values and confidence intervals as if the assumptions held, and consequently drew unreliable conclusions.

Critical Examination of the Assumptions.
The paper emphasizes that statistical models are abstractions; when the abstraction’s assumptions are not met, the inferential statements derived from the model lose validity. Specifically:

• Normality: Empirical support for normal errors is strong in physical measurement contexts but weak in biological and social contexts. Violations affect the distribution of test statistics, especially those involving variances (e.g., ANOVA F‑tests).
• Independence: In many modern data settings (time series, clustered designs, spatial data) observations are correlated, rendering the i.i.d. assumption untenable.
• Identical Distribution: Heteroscedasticity (non‑constant variance) is common in social data, leading to inefficient estimators and biased standard errors.

The author notes that later methodological advances—generalized linear models, mixed‑effects models, robust estimators, bootstrap methods, and Bayesian hierarchical models—were motivated precisely by the need to relax these restrictive assumptions. Nevertheless, the historical narrative demonstrates that the failure to critically reassess assumptions as the GLM migrated across disciplines has been a recurring source of statistical error.

Concluding Remarks.
The paper concludes that the 200‑year trajectory of the general linear model illustrates both the power and the peril of a widely adopted statistical framework. While the GLM has enabled profound scientific insights, its continued utility depends on rigorous diagnostic checking, appropriate model extensions, and a clear awareness of the domain‑specific plausibility of its assumptions. The author urges contemporary researchers to treat the GLM as a starting point rather than a universal solution, to perform thorough residual analyses, and to consider alternative modeling strategies whenever normality, independence, or homoscedasticity are questionable. This historical perspective serves as a cautionary reminder that statistical methodology must evolve hand‑in‑hand with the substantive contexts to which it is applied.