Benchmarking Python Tools for Automatic Differentiation
📝 Abstract
In this paper we compare several Python tools for automatic differentiation. In order to assess the difference in performance and precision, the problem of finding the optimal geometrical structure of the cluster with identical atoms is used as follows. First, we compare performance of calculating gradients for the objective function. We showed that the PyADOL-C and PyCppAD tools have much better performance for big clusters than the other ones. Second, we assess precision of these two tools by calculating the difference between the obtained at the optimal configuration gradient norms. We conclude that PyCppAD has the best performance among others, while having almost the same precision as the second- best performing tool - PyADOL-C.
💡 Analysis
In this paper we compare several Python tools for automatic differentiation. In order to assess the difference in performance and precision, the problem of finding the optimal geometrical structure of the cluster with identical atoms is used as follows. First, we compare performance of calculating gradients for the objective function. We showed that the PyADOL-C and PyCppAD tools have much better performance for big clusters than the other ones. Second, we assess precision of these two tools by calculating the difference between the obtained at the optimal configuration gradient norms. We conclude that PyCppAD has the best performance among others, while having almost the same precision as the second- best performing tool - PyADOL-C.
📄 Content
Abstract—In this paper we compare several Python tools for automatic differentiation. In order to assess the difference in performance and precision, the problem of finding the optimal geometrical structure of the cluster with identical atoms is used as follows. First, we compare performance of calculating gradients for the objective function. We showed that the PyADOL-C and PyCppAD tools have much better performance for big clusters than the other ones. Second, we assess precision of these two tools by calculating the difference between the obtained at the optimal configuration gradient norms. We conclude that PyCppAD has the best performance among others, while having almost the same precision as the second- best performing tool – PyADOL-C.
Keywords—Automatic differentiation, Python, PyADOL-C, PyCppAD, CasADi, Theano, CGT, AD.
I. INTRODUCTION In many different applications one who solves numerical computing problems has to deal with the exact derivative calculation task. This includes Jacobians and Hessians calculation which are used for solving ordinary and partial differential equations as well as for finding solutions to different optimization problems. Nowadays one of the possibilities to solve the task is to apply the algorithmic or automatic differentiation technique (see, for instance, [1,2,3]). It should be noted that automatic differentiation is neither numerical nor symbolic differentiation, though the main principle behind the procedure of computing derivatives is partly symbolic and partly numerical [4]. There are two ways to implement automatic differentiation for computer programs: operator overloading and source transformation. Operator overloading is a technique that implies redefinition of such elementary operation as summation, multiplication, division to update the associated gradient object by means of differentiation rules. Source transformation is another way to implement differentiation which implies rewriting the code so that it contains the implementation of a gradient for the piece of code. While the implementation of a source transformation technique is much more complex than the operator overloaded one, it usually leads to faster run-time speeds [5].
Manuscript received June 19, 2016.
A. Turkin is with the National Research University of Electronic
Technology; Dorodnicyn Computing Center, Federal Research Center
“Computer Science and Control” of Russian Academy of Sciences, Russia
(e-mail: andrei_turkin@hotmail.com).
Aung Thu is with the National Research University of Electronic
Technology (e-mail: aungaungthu61050@gmail.com)
For
further
speculations
on
some
advantages
and
disadvantages of these approaches we refer to [6].
Nowadays many researcher use Python as a scientific
environment, while applying many third-party open source
libraries to computation tasks. Some of them can be used to
implement automatic differentiation for Python code. The
simplest way to apply automatic differentiation to Python
programs is to use one of the following tools: PyADOL-C
[7], PyCppAD [8], CasADi[9], Computation Graph Toolkit
(CGT) [10], Theano [11,12], or AD [13]. All of these tools
can be used for Jacobian evaluation by applying the operator
overloading
technique
to
implement
automatic
differentiation except CasADi, which is based on source
code transformation (see issue 884 on the GitHub page for
CasADi project for further details). In the next section we
briefly describe these tools and provide information about
their features while solving a cluster of identical atoms
optimization problem which we describe in Section 3.
The purpose of this benchmarking is to call attention to
automatic differentiation in Python, to provide information
on main features of programming tools, and to highlight the
advantages of using each of them. The next section is a brief
description of several tools for automatic differentiation that
can be used in Python. In section 3 we provide information
on the problem we solve with the tools, while speculating on
their features. We conclude with the experimental results for
the problem.
II. AUTOMATIC DIFFERENTIATION IN PYTHON
In this section we provide information on the automatic
differentiation tools in Python and point out some
difficulties in using it. First, it is important for any tool to be
easy-to-install and to-use i.e. one should use it with no or
little modifications of the code to implement automatic
differentiation. Second, it should be mentioned, that the
main challenge of using automatic differentiation tools in
python is the difference in syntax and data initialization for
the procedure. In this section we provide information about
the tools listed before as well as point out some aspects of
their usage. Before discussing some details, it should be
mentioned that not all of the tools can be easily installed
with the pip environment. PyCppAd
This content is AI-processed based on ArXiv data.