The myth of equidistribution for high-dimensional simulation

📝 Abstract

A pseudo-random number generator (RNG) might be used to generate w-bit random samples in d dimensions if the number of state bits is at least dw. Some RNGs perform better than others and the concept of equidistribution has been introduced in the literature in order to rank different RNGs. We define what it means for a RNG to be (d,w)-equidistributed, and then argue that (d,w)-equidistribution is not necessarily a desirable property.

💡 Analysis

A pseudo-random number generator (RNG) might be used to generate w-bit random samples in d dimensions if the number of state bits is at least dw. Some RNGs perform better than others and the concept of equidistribution has been introduced in the literature in order to rank different RNGs. We define what it means for a RNG to be (d,w)-equidistributed, and then argue that (d,w)-equidistribution is not necessarily a desirable property.

📄 Content

There is no such thing as a random number -there are only methods to produce random numbers, and a strict arithmetic procedure of course is not such a method.

John von Neumann [11,p. 768] Suppose we are performing a simulation in d dimensions. For simplicity let the region of interest be the unit hypercube H = [0, 1) d .

For the simulation we may need a sequence y 0 , y 1 , . . . of points uniformly and independently distributed in H. A pseudo-random number generator gives us a sequence x 0 , x 1 , . . . of points in [0, 1). Thus, it is natural to group these points in blocks of d, that is y j = (x jd , x jd+1 , . . . , x jd+d-1 ) .

If our pseudo-random number generator is good and d is not too large, we expect the y j to behave like uniformly and independently distributed points in H.

We are considering applications where the (pseudo-)random number generator should, as far as possible, be indistinguishable from a perfectly random source. In some applications, e.g. Monte Carlo quadrature, it is better to use quasi-random numbers which are intended for that application and give an estimate with smaller variance than we could expect with a perfectly random source.

For example, when estimating a contour integral of an analytic function, we might transform the contour to a circle and use equally spaced points on the circle.

However, when simulating Canberra’s future climate and water supply, it would not be a good idea to assume that exceptionally dry years were equally spaced!

If we use the χ 2 test to test the hypothesis that a set of data is a random sample from some distribution, then we typically reject the hypothesis if the χ 2 statistic is too large. However, we should equally reject the hypothesis if χ 2 is too small (because in this case the fit is too good) [9].

In the “old days” people often followed Lehmer’s suggestion and used linear congruential random number generators of the form

This gives an integer in [0, m) so needs to be scaled:

Typically m is a power of two such as 2 32 or 2 64 , or a prime close to such a power of two.

Unfortunately, all such linear congruential generators perform badly in high dimensions, as shown in Marsaglia’s famous paper Random numbers fall mainly in the planes [7].

Some linear congruential generators perform disastrously. For example, consider the infamous RANDU:

(with z 0 odd). These points satisfy z n+2 -6z n+1 + 9z n = 0 mod 2 31 so in dimension d = 3 the resulting points y j all lie on a small number of planes, in fact 15 planes separated by distance 1/ √

In general, such behaviour is detected by the spectral test [6].

Even the best linear congruential generators perform badly because they have period at most m, so the average distance between points y j is of order

(so the set of points closest to any one y j has volume of order 1/m).

Nowadays, linear congruential generators are rarely used in high-dimensional simulations. Instead, generators with much longer periods are used. A popular class is those given by a linear recurrence over F 2 . These take the form

where u i is an n-bit state vector, v i is a w-bit output vector which may be regarded as a fixed-point number x i , and the linear algebra is performed over the field F 2 = GF(2) of two elements {0, 1}. Here A is an n × n matrix and B is a w×n matrix (both over F 2 ). Usually A is sparse (so the matrix-vector multiplication can be performed quickly) and often B is a projection.

Provided the characteristic polynomial of A is primitive over F 2 , and B = 0, the period of such a generator is 2 n -1. This can be very large, e.g. n = 4096 for xorgens [3] and n = 19937 for the Mersenne Twister [8]. For details we refer to L’Ecuyer’s papers [5,12].

Various definitions of (d, w)-equidistribution can be found in the literature. We follow Panneton and L’Ecuyer [12] without attempting to be too general.

Consider w-bit fixed-point numbers. There are 2 w such numbers in [0, 1). Each such number can be regarded as representing a small interval of length 2 -w .

Similarly, in d dimensions, we can consider small hypercubes whose sides have length 2 -w . Each small hypercube has volume 2 -dw and there are 2 dw of them in the unit hypercube [0, 1) d . A small hypercube can be specified by a d-dimensional vector of w-bit numbers (a total of dw bits).

Consider a random number generator with period 2 n . (A slight change in the definition can be made to accomodate generators with period 2 n -1.) If the generator is run for a complete period to generate 2 n pseudorandom points in [0, 1) d , we say that the generator is (d, w)-equidistributed if the same number of points fall in each small hypercube.

The condition n ≥ dw is necessary. The number of points in each small hypercube is 2 n-dw . RANDU (with n = 29) is not (d, w)-equidistributed for any d ≥ 3, w ≥ 4. However, most good long-period generators are (d, w)-equidistributed for dw ≪ n.

The maximum w for which a generator can be (d, w)-equidistribut

This content is AI-processed based on ArXiv data.