A note on the ranking of earthquake forecasts

The task of earthquake forecasting is to predict a map )} ( { k   of target events rate in cells } { k  of a region G for a time period T  . Reliable estimates of )} ( { k   are usually problematical. For this reason there are many approaches to the estimation of such maps that are not based on instrumental data alone. This circumstance is incorporated in the Regional Earthquake Likelihood Models (RELM) project (Schorlemmer et al., 2007), and this stimulated comparison and ranking of the normalized models

is the number of target events in a cell during a time T  , and N is the sum of these numbers. Independently, the comparison problem of the models (1) is highly important in seismic risk analysis.

The ranking problem is solved in the project by pairwise comparison of the competing models (Schorlemmer et al., 2007;Eberhard et al., 2012;Zechar et al., 2013). This approach is based on the classical theory of hypothesis testing, where it is always assumed that one of two competing hypotheses is true. Therefore the matrix of all pairwise comparisons among

cannot provide a logically consistent ranking of the models (Rhoads et al., 2011;Molchan, 2011). New approaches arose as a result. Some of these are based on comparative and integrated analysis of maps (Clements et al., 2011;Taroni et al., 2014), while others try to rank models using a measure of forecasting quality, R ( Zechar &Zhuang, 2014).

The R-ranking method proposed in the last paper is purely intuitive. Our goal is to formulate desirable properties of R at least on the stage of large N and to analyze some R-ranking methods on this basis.

To be able to discuss forecasting quality measures, it is sufficient to consider the simplest seismicity model. We assume that )} ( { k n  come from a Poisson sequence of events with the rate )

The problem of agreement of the model

be considered for large N . Under the conditions of stationarity,

3. The quality measure of the model, R .

The quality measure of the model, R , can be treated as a continuous functional of three

, and a reference model

it is therefore desirable that its choice should not substantially affect the ranking of the competing models (the 0-requirement on R ).

Suppose that one of the competing models is explicit,

; then for large N this model must almost surely get the highest rating. In the limit,   N , this entails the 1requirement on R :

(3)

In the above, we have replaced the first argument . The R test cannot reject the wrong P model, if these limits are identical (Molchan, 2011). For this reason it would be desirable to have the following

In mathematical statistics this requirement is considered as the consistency property of the statistical test R (Borovkov, 1984).

Obviously, any metric } , { 2 1 P P  on the space of distributions satisfies requirements 1 and 2 and can be used to rank models by their distances from the empirical distribution

. But any choice of the metric is dictated by additional requirements. To give an example, divide the seismic region G into zones ) (r J of different degrees of importance for some user, choose a suitable metric ) .

(

. Due to (2), one has

where

for pairwise comparison among the models.

It is easy to see that the information score R satisfies requirement 1. Indeed, by ( 5), one has 7) is a hyperplane that passes through a point P of the simplex

. The intersection of these geometrical objects defines the set of solutions } {  P of equation ( 7). The intersection is obviously a simplex S S   of dimension 2 К  , provided P is an inner point on S. For this reason requirement 2 is violated.

Let us approximate the information score (5) using the relation

. We get a new version of the R score, viz.,

Using formal analogies with gaming, Zechar&Zhuang (2014) came to a symmetric version of (8), namely } , , { 0

where

, and K is the number of cells . In addition, the authors choose 0 P as follows:

where n w / 1   and n is the number of competing models. The R score (9,10) is the basis of the pari-mutuel gambling method in the merit ranking of forecasts.

Practically in the same form, the R score was used by Zechar&Zhuang (2010) for testing of binary forecasts. However, for this purpose the R score ( 9) was potentially unstable ( Molchan & Romashkova, 2011) . Now we are going to check whether the R function (9,10) can be applied to probabilistic forecasts. Inequality (3) combined with ( 9) looks as follows:

is a simplex again. In the generic situation, the dimension of the intersection is K-2. That means that the right-hand side of ( 11) is an alternating function on S . Therefore, in general, the requirement 1 for R is not satisfied.

Let us consider now the original case in which 0 P is given by (10). We fix the distribution

and consider R  as a function of

Here the unknown constant  is a result of the condition:

. By summing ( 14) over k , we conclude that with the reference distribution ( 10) is in agreement w

This content is AI-processed based on open access ArXiv data.