How to Combine Independent Data Sets for the Same Quantity
📝 Abstract
This paper describes a recent mathematical method called conflation for consolidating data from independent experiments that are designed to measure the same quantity, such as Planck’s constant or the mass of the top quark. Conflation is easy to calculate and visualize, and minimizes the maximum loss in Shannon information in consolidating several independent distributions into a single distribution. In order to benefit the experimentalist with a much more transparent presentation than the previous mathematical treatise, the main basic properties of conflation are derived in the special case of normal (Gaussian) data. Included are examples of applications to real data from measurements of the fundamental physical constants and from measurements in high energy physics, and the conflation operation is generalized to weighted conflation for situations when the underlying experiments are not uniformly reliable.
💡 Analysis
This paper describes a recent mathematical method called conflation for consolidating data from independent experiments that are designed to measure the same quantity, such as Planck’s constant or the mass of the top quark. Conflation is easy to calculate and visualize, and minimizes the maximum loss in Shannon information in consolidating several independent distributions into a single distribution. In order to benefit the experimentalist with a much more transparent presentation than the previous mathematical treatise, the main basic properties of conflation are derived in the special case of normal (Gaussian) data. Included are examples of applications to real data from measurements of the fundamental physical constants and from measurements in high energy physics, and the conflation operation is generalized to weighted conflation for situations when the underlying experiments are not uniformly reliable.
📄 Content
When different experiments are designed to measure the same unknown quantity, such as Planck’s constant, how can their results be consolidated in an unbiased and optimal way? Given data from experiments that may differ in time, geographical location, methodology and even in underlying theory, is there a good method for combining the results from all the experiments into a single distribution? Note that this is not the standard statistical problem of producing point estimates and confidence intervals, but rather simply to summarize all the experimental data with a single distribution. The consolidation of data from different sources can be particularly vexing in the determination of the values of the fundamental physical constants. For example, the U.S. National Institute of Standards and Technology (NIST) recently reported “two major inconsistencies” in some measured values of the molar volume of silicon V m (Si) and the silicon lattice spacing d 220 , leading to an ad hoc factor of 1.5 increase in the uncertainty in the value of Planck’s constant h ([9, p. 54], [10]). (One of those two inconsistencies has subsequently been resolved [8].)
But input data distributions that happen to have different means and standard deviations are not necessarily “inconsistent” or “incoherent” [2, p 2249]. If the various input data are all normal (Gaussian) or exponential, for example, then every interval centered at the unknown positive true value has a positive probability of occurring in every one of the independent experiments. Ideally, of course, all experimental data, past as well as present, should be incorporated into the scientific record. But in the case of the fundamental physical constants, for instance, this could entail listing scores of past and present experimental datasets, each of which includes results from hundreds of experiments with thousands of data points, for each one of the fundamental constants. Most experimentalists and theoreticians who use Planck’s constant, however, need a concise summary of its current value rather than the complete record. Having the mean and estimated standard deviation (e.g. via weighted least squares) does give some information, but without any knowledge of the distribution, knowing the mean within two standard deviations is only valid at the 75% level of significance, and knowing the mean within four standard deviations is not even significant at the standard 95% confidence level. Is there an objective, natural and optimal method for consolidating several input-data distributions into a single posterior distribution P ? This article describes a new such method called conflation.
First, it is useful to review some of the shortcomings of standard methods for consolidating data from several different input distributions. For simplicity, consider the case of only two different experiments in which independent laboratories Lab I and Lab II measure the value of the same quantity. Lab I reports its results as a probability distribution (e.g. via an empirical histogram or probability density function), and Lab II reports its findings as . One common method of consolidating two probability distributions is to simply average themfor every set of values A, set If the distributions both have densities, for example, averaging the probabilities results in a probability distribution with density the average of the two input densities (Figure 1). This method has several significant disadvantages. First, the mean of the resulting distribution is always exactly the average of the means of , independent of the relative accuracies or variances of each. (Recall that the variance is the square of the standard deviation.) But if Lab I performed twice as many of the same type of trials as Lab II, the variance of would be half that of , and it would be unreasonable to weight the two respective empirical means equally.
A second disadvantage of the method of averaging probabilities is that the variance of is always at least as large as the minimum of the variances of (see Figure 1), since
V P V P mean P mean P . If are nearly identical, however, then their average is nearly identical to both inputs, whereas the standard deviation of a reasonable consolidation should probably be strictly less than that of both . The method of averaging probabilities completely ignores the fact that two laboratories independently found nearly the same results. Figure 1 also shows another shortcoming of this method -with normally-distributed input data, it generally produces a multimodal distribution, whereas one might desire the consolidated output distribution to be of the same general form as that of the input data -normal, or at least unimodal. Another common method of consolidating data -one that does preserve normality -is to average the underlying input data itself. That is, if the result of the experiment from Lab I is a random variable (i.e. has distribution ) and the result of Lab II is (independent of ,
This content is AI-processed based on ArXiv data.