At the heart of many scientific applications is the solution of algebraic systems, such as linear systems of equations, eigenvalue problems, and optimization problems, to name a few. TOPS, which stands for Towards Optimal Petascale Simulations, is a SciDAC applied math center focused on the development of solvers for tackling these algebraic systems, as well as the deployment of such technologies in large-scale scientific applications of interest to the U.S. Department of Energy. In this paper, we highlight some of the solver technologies we have developed in optimization and matrix computations. We also describe some accomplishments achieved using these technologies in UNEDF, a SciDAC application project on nuclear physics.
Over the last couple of decades, simulation science has become as important as theoretical and experimental science. The success of simulation science hinges on the ability to perform the calculations efficiently. The inner most kernel in these calculations is often the solution of algebraic systems, including, but not limited to, systems of linear and nonlinear equations, eigenvalue problems, optimization problems, and sensitivity analysis. TOPS, which stands for Towards Optimal Petascale Simulations, is a multi-institutional SciDAC applied math center that focuses on the development of solvers for tackling these algebraic systems, as well as the deployment of such technologies in large-scale scientific applications, particularly those of interest to the U.S. Department of Energy.
In this paper, we highlight two specific areas of TOPS: eigenvalue calculations and optimization.
In particular, we highlight some accomplishments we have achieved in collaboration with computational physicists in UNEDF. The goal of the UNEDF SciDAC application project [1] is to obtain a comprehensive understanding of nuclei and their reactions based on the most accurate knowledge of the strong nuclear interaction. Eigenvalue calculations come up in the solution of the nuclear Schrödinger equation [2,3]. The eigenvalues and the eigenvectors correspond to the energy states and wave functions. Numerical optimization techniques are needed in building the next generation of nuclear energy functionals, which will provide nuclear physicists better tools for predicting the properties and behavior of atomic nuclei over the entire nuclear table.
In nuclear configuration interaction calculation, it is sometimes necessary to investigate, among others, nuclear level densities as a function of the total angular momentum J and excitation energy, and to evaluate scattering amplitudes as a function of J [4]. We will refer to this as a total-J calculation in this paper. In this type of calculation, we are interested in computing a relatively large number of states with a prescribed J value.
One brute-force approach to a total-J calculation is to simply compute a large number of eigenvalues and wave functions of a nuclear many-body Hamiltonian, for example in an Mscheme basis (good angular momentum projection along the z-axis), and select, among these wave functions, the ones that have a prescribed J value. This approach is appropriate when the number of desired energy states and wave functions is small (e.g., ten to twenty states). When that is not true, or when certain properties of a nucleus pertaining to a fixed J are to be calculated, the brute-force approach may require computing a very large number of wave functions, and the computational cost for performing this type of calculation may be prohibitively high. Furthermore, even if we can afford to perform this type of calculation, this may not be an efficient use of resources because we compute a large number of wave functions only to throw away most of them because they do not have the desired J value.
We have developed an alternative approach where we construct an invariant subspace Z that contains all wave functions associated with a fixed J value in advance and project the nuclear many-body Hamiltonian into this subspace to produce a projected Hamiltonian with the minimum dimension consistent with that chosen J. A sparse matrix diagonalization procedure [5,6,7] is then applied to this projected Hamiltonian to obtain the desired energy states and their corresponding wave functions.
To construct Z, we need to work with the total angular momentum square operator Ĵ2 and compute the null space of Ĵ2 -λI, where λ = J(J + 1) is a known eigenvalue of Ĵ2 .
When the many-body basis states associated with the configuration space are properly ordered and grouped, Ĵ2 becomes block diagonal: Ĵ2 = diag( Ĵ2 1 , Ĵ2 2 , …, Ĵ2 ng ). Therefore, the task of computing the desired null space of Ĵ2 -λI reduces to that of computing the desired null spaces of Ĵ2 i -λI, for i = 1, 2, …, n g . However, because the dimensions of the Ĵ2 i ’s vary over a wide range (e.g., from 1 to more than 36,000 for 12 C, N max = 6), it is difficult to maintain a good load balance in the null space calculation. Here, N max is a parameter limiting the total number of oscillator quanta allowed in the many-body states.
We developed a multi-level task and data distribution scheme to achieve optimal parallel performance in the null space calculation by (i) Limiting the granularity of the parallelism; that is, we try to divide the overall task into many small tasks of limited sizes so that good load balance arises from distributing these small tasks evenly among different processors.
(ii) Limiting the communication overhead incurred in the null space calculation so that the overall time of the computation can be minimized.
To achieve these inherently conflicting goals, we classified Ĵ2 i blocks into small, medium and large
This content is AI-processed based on open access ArXiv data.
