Introduction to Randomness and Statistics

This text provides a practical introduction to randomness and data analysis, in particular in the context of computer simulations. At the beginning, the most basics concepts of probability are given, in particular discrete and continuous random variables. Next, generation of pseudo random numbers is covered, such as uniform generators, discrete random numbers, the inversion method, the rejection method and the Box-Mueller Method. In the third section, estimators, confidence intervals, histograms and resampling using Bootstrap are explained. Furthermore, data plotting using the freely available tools gnuplot and xmgrace is treated. In the fifth section, some foundations of hypothesis testing are given, in particular the chi-squared test, the Kolmogorov-Smirnov test and testing for statistical (in-)dependence. Finally, the maximum-likelihood principle and data fitting are explained. The text is basically self-contained, comes with several example C programs and contains eight practical (mainly programming) exercises.

💡 Research Summary

The manuscript serves as a hands‑on introduction to probability and statistical data analysis with a strong emphasis on computer simulation. It is organized into six coherent sections, each blending theoretical exposition with concrete C‑language examples and practical exercises.

The first section lays the mathematical groundwork. It defines random variables, distinguishes discrete from continuous cases, and introduces probability mass functions, probability density functions, and cumulative distribution functions. Fundamental moments such as expectation, variance, covariance, and correlation are derived, and basic properties like linearity and unbiasedness are highlighted. A brief mention of Bayes’ theorem sets the stage for later inference tasks.

The second section focuses on pseudo‑random number generation (PRNG). Starting with the linear congruential generator, the authors discuss period length, seed selection, and implementation details, providing a complete C program. Building on uniform random numbers, the text explains how to sample discrete distributions (e.g., Bernoulli, Poisson) via the inversion of the cumulative distribution table. For continuous distributions, three techniques are covered: the inverse‑transform method, the rejection (accept‑reject) method, and the Box‑Muller transformation for generating Gaussian deviates. Efficiency considerations, visual quality checks (histograms), and simple statistical tests (χ², Kolmogorov‑Smirnov) for assessing generator quality are also presented.

The third section moves to statistical estimation and uncertainty quantification. Sample means and variances are examined as estimators, with discussion of bias and variance. Confidence intervals are constructed using the normal and t‑distributions, illustrated through step‑by‑step calculations. The bootstrap resampling technique is introduced as a non‑parametric way to approximate the sampling distribution of any statistic; the authors provide code that repeatedly draws with replacement, computes the statistic, and derives standard errors and confidence limits directly from the empirical distribution. Guidance on histogram binning and weighting is included to help readers visualize estimator variability.

The fourth section teaches data visualization using two free tools: gnuplot and xmgrace. The authors demonstrate how to produce basic 2D plots, overlay multiple data series, label axes, add legends, adjust ranges, and customize line styles and colors. They also cover batch scripting and data import from common formats (CSV, plain text), enabling automated generation of publication‑ready figures from large simulation outputs.

The fifth section introduces hypothesis testing. The χ² goodness‑of‑fit test is explained in detail, showing how to compare observed frequencies with expected counts, compute the test statistic, determine degrees of freedom, and interpret p‑values. The Kolmogorov‑Smirnov test is presented as a distribution‑free alternative for continuous data, with a clear description of the empirical CDF, the maximum absolute deviation, and critical values. Tests for statistical independence are covered through the χ² test of independence and correlation‑based approaches (Pearson and Spearman coefficients), with emphasis on assumptions, sample‑size requirements, and interpretation of results.

The final section deals with the maximum‑likelihood principle and model fitting. The log‑likelihood function is defined, and the authors walk through numerical optimization techniques such as Newton‑Raphson, gradient descent, and limited‑memory BFGS, providing C implementations for each. The Fisher information matrix is used to obtain standard errors and confidence intervals for the estimated parameters. Several fitting examples are given: linear regression, polynomial regression, and non‑linear models (exponential, logarithmic, Gaussian). Residual analysis—including residual plots, normality checks, and the coefficient of determination (R²)—is advocated to assess model adequacy.

Throughout the manuscript, each theoretical concept is reinforced by a concrete programming exercise. The eight end‑of‑chapter problems require students to write, compile, and run code that implements the discussed methods, thereby consolidating learning. By integrating probability theory, random‑number generation, estimation, visualization, hypothesis testing, and likelihood‑based fitting, the text offers a comprehensive, self‑contained resource for students and researchers in computer science, physics, engineering, and the life sciences who need to perform simulation‑driven data analysis.