Posterior Inference in Curved Exponential Families under Increasing Dimensions

The main motivation for this paper is to obtain large sample results for posterior inference in the curved exponential family under increasing dimension. In the exponential family, the log of a density is linear in the parameters θ ∈ Θ; in the curved exponential family, the parameters θ are restricted to lie on a curve η → θ(η) parameterized by a lower dimensional parameter η ∈ Ψ. There are many classical examples of densities that fall in the curved exponential family; see for example Efron (1978), Lehmann and Casella (1998), and Barndorff-Nielsen (1978). Curved exponential densities have also been extensively used in applications Efron (1978); Heckman (1974); Hunter and Handcock (2006); Hunter (2007). An example of the condition that creates a curved structure in an exponential family is a moment restriction of the type: m(x, ν)f (x, θ)dx = 0, that restricts θ to lie on a curve that can be parameterized as {θ(η), η ∈ Ψ}, where component η = (ν, β) contains ν and other parameters β that are sufficient to parameterize all parameters θ ∈ Θ that solve the above equation for some ν. In econometric applications, often moment restrictions represent Euler equations that result from the data being an outcome of an optimization by rational decision-makers; see e.g. Hansen and Singleton (1982), Chamberlain (1987), Imbens (1997), Chernozhukov and Hong (2003), and Donald et al. (2003). In the last section of the paper we discuss in more details other econometric models that fit this framework, such as multivariate linear models, seemingly unrelated regressions, single equation structural models, as in Zellner (1962) and Zellner (1971). We also discuss multinomial model with moment restrictions. Thus, the curved exponential framework is a fundamental complement to the exponential framework.

Under high-dimensionality, despite of its applicability, theoretical properties of the curved exponential family are not as well understood as the corresponding properties of the exponential family. We contribute to the theoretical analysis of the posterior inference in curved exponential families under high dimensionality. We provide sufficient conditions under which consistency and asymptotic normality of the posterior is achieved when both the dimension of the parameter space and the sample size are large. Our framework only requires weak conditions on the prior distribution, which allows for improper priors. In particular, the uninformative prior always satisfies our assumptions. We also study the convergence of moments and the rates with which we can estimate them. We then apply these results to a variety of models where both the parameter dimension and the number of moments are increasing with the sample size.

The present analysis of the posterior inference in the curved exponential family builds upon the work of Ghosal (2000) who studied posterior inference in the exponential family under increasing dimension. Under sufficient growth restrictions on the dimension of the model, it was shown that the posterior distributions concentrate in neighborhoods of the true parameter and can be approximated by an appropriate normal distribution. Such analysis extended in a fundamental way the classical results of Portnoy (1988) for maximum likelihood methods for the exponential family with increasing dimensions.

In addition to a detailed treatment of the curved exponential family, we also revisit the exponential family setting under increasing dimension. We present several new results that complement the results in Ghosal (2000). First, we amend the conditions on priors to allow for a larger set of priors, for example, improper priors; second, we use concentration inequalities for logconcave densities to sharpen the conditions under which the normal approximations apply; and third, we show that the approximation of α-th order moments of the posterior by the corresponding moments of the normal density becomes exponentially difficult in the moment order α.

We also note that by establishing the asymptotic normality of the posterior distribution we can invoke results in Belloni and Chernozhukov (2009) that guarantees good computational properties for MCMC methods. Moreover, new results on sampling from manifolds (see Diaconis et al. (2012)) permits the implementation of different random walk schemes that are useful for implementing inference in curved exponential families.

This work allows for increasing dimension, so it can be thought as a sieve technique. However, this paper does not formally account for the approximation errors resulting from using approximate functional forms as opposed to exact functional forms. Approximation errors can be introduced into the model and our results can also be shown to hold under more stringent conditions (approximations errors need to vanish at rates faster than the sampling errors), a sharp analysis of the impact of the approximation error can be delicate and is outside of the scope of the present

…(Full text truncated)…

This content is AI-processed based on ArXiv data.