Estimation of experimental data redundancy and related statistics

Redundancy of experimental data is the basic statistic from which the complexity of a natural phenomenon and the proper number of experiments needed for its exploration can be estimated. The redundancy is expressed by the entropy of information pertaining to the probability density function of experimental variables. Since the calculation of entropy is inconvenient due to integration over a range of variables, an approximate expression for redundancy is derived that includes only a sum over the set of experimental data about these variables. The approximation makes feasible an efficient estimation of the redundancy of data along with the related experimental information and information cost function. From the experimental information the complexity of the phenomenon can be simply estimated, while the proper number of experiments needed for its exploration can be determined from the minimum of the cost function. The performance of the approximate estimation of these statistics is demonstrated on two-dimensional normally distributed random data.

💡 Research Summary

The paper introduces a novel statistical framework for quantifying the redundancy of experimental data and, from this measure, estimating both the intrinsic complexity of a natural phenomenon and the optimal number of experiments required for its thorough exploration. Redundancy is defined as the information entropy associated with the probability density function (PDF) of the experimental variables. Traditional entropy calculation demands multidimensional integration over the variable space, which becomes computationally prohibitive as dimensionality grows. To overcome this obstacle, the authors adopt a kernel‑density‑estimation (KDE) approach: each measured data point (x_i) is represented by a Gaussian kernel with a common bandwidth (h) (or covariance matrix (\Sigma)), and the overall PDF is approximated by the sum of these kernels. Substituting this KDE into the entropy definition transforms the integral into a double sum over all data pairs, yielding an expression of the form

\