A memristor crossbar, which is constructed with memristor devices, has the unique ability to change and memorize the state of each of its memristor elements. It also has other highly desirable features such as high density, low power operation and excellent scalability. Hence the memristor crossbar technology can potentially be utilized for developing low-complexity and high-scalability solution frameworks for solving a large class of convex optimization problems, which involve extensive matrix operations and have critical applications in multiple disciplines. This paper, as the first attempt towards this direction, proposes a novel memristor crossbar-based framework for solving two important convex optimization problems, i.e., second-order cone programming (SOCP) and homogeneous quadratically constrained quadratic programming (QCQP) problems. In this paper, the alternating direction method of multipliers (ADMM) is adopted. It splits the SOCP and homogeneous QCQP problems into sub-problems that involve the solution of linear systems, which could be effectively solved using the memristor crossbar in O(1) time complexity. The proposed algorithm is an iterative procedure that iterates a constant number of times. Therefore, algorithms to solve SOCP and homogeneous QCQP problems have pseudo-O(N) complexity, which is a significant reduction compared to the state-of-the-art software solvers (O(N^3.5) - O(N^4)).
Convex optimization is a research field that aims to find the optimal solution for the problem of minimizing a convex objective function subject to some convex constraints. The utility of convex optimization has been shown extensively in various applications such as signal processing, communications, smart grid, machine learning, circuit design, and other applications [1] [2]. It is especially required in state-of-the-art large-scale applications in machine learning (e.g., the support vector machine [3]) and compressed sensing techniques [4].
There does not exist a common solution for general convex optimization problems [5]. But for a number of important types of convex optimization problems, such as semidefinite programming (SDP), quadratically constrained quadratic programming (QCQP) and second-order cone programming (SOCP), optimal software-based solutions exist that use effective algorithms such as extensions of the primal-dual interior point (PDIP) method [8]. However, in the era of data deluge, software-based optimization solvers suffer from limited scalability in high-dimensional data regimes. For example, solving a SDP problem has an O(N 6 ) complexity using state-of-the-art softwarebased solvers [20]. This complexity is prohibitive for problems with large volumes of data. Therefore, it is imperative to develop new techniques and new solvers that overcome these limitations.
The recently invented memristor crossbar can potentially resolve the limitations efficiently. Because the memristor device, invented by HP Lab [13], has the unique property that its state (memristance) can be changed when the voltage drop at its two terminals is higher than a threshold voltage. Thus, a single memristor device can be readily utilized to represent a matrix element. Moreover, its promising features of non-volatility, excellent scalability, high density and low power operation make it a candidate to be arranged in a crossbar structure to represent matrices and perform matrix computations efficiently (often in O(1) time complexity). As many convex optimization problems, such as SOCP problems, need to perform a large number of matrix operations (matrix-vector multiplications and solving linear systems, etc.), they can potentially be solved by using memristor crossbar technology that provides low computational complexity, high speed and energy efficiency.
Despite the fact that memristor devices have the potential to be utilized to solve certain important convex optimization problems, there are multiple challenges and limitations from both algorithm side and hardware side. From the algorithm side, the algorithms proven to be successful in solving SOCP problems with software-based solvers may not be appropriate for hardware implementations. With respect to the hardware side, the memristor crossbar can only deal with square matrix computations and the matrix elements can only be non-negative numbers because memristance cannot be negative. Consequently, an algorithm-hardware co-design and co-optimization framework is required to overcome these limitations with high efficiency and low computational complexity.
For ease of hardware implementation, we use an operator splitting method, the alternating direction method of multipliers (ADMM), to solve SOCP problems. The major advantage of ADMM is that it can split the original problem into a set of problems that involve the solution of linear systems. Additionally, a large number of problems can be formulated in the form of SOCP or be formulated as problems with second-order cone constraints, such as homogeneous QCQP problems [5]. Hence, a large number of convex optimization problems can be solved efficiently with the memristor crossbar and ADMM algorithm.
To the best of our knowledge, this paper presents the first framework for solving SOCP and homogeneous QCQP problems using memristor crossbar techniques. This is expected to be an important step towards the solution of more general convex optimization problems. The proposed solution procedure is an iterative procedure with O(N) complexity in each iteration, and the procedure does not need to update the conductance matrix of memristor crossbar during iterations, thereby significantly reducing the solution complexity. Besides, the procedure only iterates a constant number of times, thus the solution framework can achieve pseudo-O(N) computational complexity. Compared with software-based solvers of SOCP and homogeneous QCQP problems (the CVX tool), the proposed memristor crossbar-based solution framework achieves significant speedup and energy efficiency improvement up to 1.57 × 10 5 X and 1.32 × 10 7 X, respectively. Finally, extensive experimental results demonstrate excellent reliability of the proposed solution framework under process variations.
In the rest of this paper, Section II presents the background on convex optimization and the forms of SOCP and homogeneous QCQP problems, as well as the memristor crossbar stru
This content is AI-processed based on open access ArXiv data.
