The best known methods for estimating hazard rate functions in survival analysis models are either purely parametric or purely nonparametric. The parametric ones are sometimes too biased while the nonparametric ones are sometimes too variable. In the present paper a certain semiparametric approach to hazard rate estimation, proposed in Hjort (1991), is developed further, aiming to combine parametric and nonparametric features. It uses a dynamic local likelihood approach to fit the locally most suitable member in a given parametric class of hazard rates, and amounts to a version of nonparametric parameter smoothing within the parametric class. Thus the parametric hazard rate estimate at time $s$ inserts a parameter estimate that also depends on $s$. We study bias and variance properties of the resulting estimator and methods for choosing the local smoothing parameter. It is shown that dynamic likelihood estimation often leads to better performance than the purely nonparametric methods, while also having capacity for not losing much to the parametric methods in cases where the model being smoothed is adequate.
1. Introduction and summary. This paper concerns a class of semiparametric type methods of estimating hazard rate functions in models for life history data. The best known methods for estimating such hazard rates are those that are either purely parametric or purely nonparametric. The parametric methods are usually biased since parametric models are usually imperfect, and the nonparametric methods often have high estimation variance. There should accordingly be room for methods that somehow lie between the parametric and the nonparametric ones. One might hope that such methods are better than the nonparametric ones if the true hazard is in the vicinity of the parametric model, while not being much worse than the parametric ones if the parametric model is true.
Although results can be obtained in a more general framework of counting process models we shall mainly be content to illustrate and investigate ideas for the ‘random censorship’ model, which is the simplest and perhaps most important special case of such models for censored lifetime data. It postulates that life-times X 0 1 , . . . , X 0 n from a population are i.i.d. with density f (.), cumulative distribution F (.), and hazard rate function α(.) given by α(s) = f (s)/F [s, ∞); α(s) ds is the probability of failing in [s, s + ds) given that an individual is still at risk at time s. The life-time X 0 i may not be directly observed, however, because of a possibly interfering censoring variable C i ; only X i = min(X 0 i , C i ) and the indicator variable δ i = I{X 0 i ≤ C i } are observed. For simplicity and concreteness we stipulate that the C i ’s are independent of the life-times and i.i.d. according to a distribution with cumulative function G. In particular the n pairs (X i , δ i ) are i.i.d. Finally we shall assume that data are obtained on a finite time horizon basis, say on [0, T ] for a known and finite T . This is convenient for some of the martingale convergence theory and is not a practical limitation.
The parametric approach is to postulate that α(s) = α(s, θ) for a suitable family, indexed by some one-or multi-dimensional θ. Typical examples include the exponential, the Weibull, the simple frailty model with α(s) = θ 1 /(1 + θ 2 s), the piecewise constant hazard rate model, the Gompertz-Makeham distribution, the gamma, and the log-normal. Properties of the maximum likelihood method for estimating θ with censored data have been studied by Borgan (1984) and others under the condition that the model is correct, i.e. that there really is some θ 0 with α(s) = α(s, θ 0 ) on [0, T ]. In practice the model is never perfect, however, and it is useful to study estimation methods outside model conditions, where the best parameter is to be thought of as being ’least false’ or ‘most suitable’, as opposed to ’true’. The large-sample behaviour of several estimation methods in this wider setting has been explored in Hjort (1992). Some results about this are reviewed in Section 2 and are used in later sections.
In Section 3 a dynamic likelihood approach to parametric hazard rate estimation is presented. It takes as its basis any given parametric hazard function and consists of inserting a local parameter estimate θ(s) in α(s, θ) at time s, producing
where the parameter estimate is obtained using only information on those individuals that have survived up to s -1 2 h and what happens to them on [s -1 2 h, s + 1 2 h]. This amounts to a kind of nonparametric parameter smoothing within a given parametric class. A more general estimator involving smoothing with a kernel function is also discussed. Bias and variance properties are studied in Section 3 for one-dimensional and in Section 4 for multi-dimensional families. It turns out that E α(s) . = α(s) + 1 2 β K h 2 b(s) and Var α(s)
where β K and γ K are characteristics of the kernel function used and y(s) is the limiting proportion of individuals still at risk at time s. The b(s) is a certain bias factor, the size of which depends on both α (s) and characteristics of the underlying parametric model used. These results match closely those of the most usual nonparametric method, that of smoothing the empirical cumulative hazard function, for which E α(s) . = α(s) + 1 2 β K h 2 α (s) and Var α(s) . = γ K nh α(s) y(s) .
In Section 5 situations are characterised where the new method performs better than the traditional nonparametric method. Methods for choosing the local smoothing parameter h are discussed in Section 6, including the arduous one that for each s expands the s ± 1 2 h interval until a goodness of fit criterion rejects the model. Overall it transpires that a suitable dynamic likelihood estimator often can perform better than the purely nonparametric ones, while at the same time not losing much to parametric ones when the true hazard is close to the parametric hazard. Finally some supplementing results and remarks are offered in Section 7.
This paper expands in several ways on the basic result
This content is AI-processed based on open access ArXiv data.