An Energy-Efficient Mixed-Signal Neuron for Inherently Error-Resilient Neuromorphic Systems

📝 Abstract

This work presents the design and analysis of a mixed-signal neuron (MS-N) for convolutional neural networks (CNN) and compares its performance with a digital neuron (Dig-N) in terms of operating frequency, power and noise. The circuit-level implementation of the MS-N in 65 nm CMOS technology exhibits 2-3 orders of magnitude better energy-efficiency over Dig-N for neuromorphic computing applications - especially at low frequencies due to the high leakage currents from many transistors in Dig-N. The inherent error-resiliency of CNN is exploited to handle the thermal and flicker noise of MS-N. A system-level analysis using a cohesive circuit-algorithmic framework on MNIST and CIFAR-10 datasets demonstrate an increase of 3% in worst-case classification error for MNIST when the integrated noise power in the bandwidth is ~ 1 {\mu}V2.

💡 Analysis

This work presents the design and analysis of a mixed-signal neuron (MS-N) for convolutional neural networks (CNN) and compares its performance with a digital neuron (Dig-N) in terms of operating frequency, power and noise. The circuit-level implementation of the MS-N in 65 nm CMOS technology exhibits 2-3 orders of magnitude better energy-efficiency over Dig-N for neuromorphic computing applications - especially at low frequencies due to the high leakage currents from many transistors in Dig-N. The inherent error-resiliency of CNN is exploited to handle the thermal and flicker noise of MS-N. A system-level analysis using a cohesive circuit-algorithmic framework on MNIST and CIFAR-10 datasets demonstrate an increase of 3% in worst-case classification error for MNIST when the integrated noise power in the bandwidth is ~ 1 {\mu}V2.

📄 Content

∑ w1 1 i1 i2 inputs output Adder Sigmoidal Thresholding Multipliers w0 x F(x) O= F(x) w2 w3 i3 in wn

Fig. 1. Multiplier-based neuron model: MAC with thresholding An Energy-Efficient Mixed-Signal Neuron for Inherently Error-Resilient Neuromorphic Systems Baibhab Chatterjee, Student Member, IEEE, Priyadarshini Panda, Student Member, IEEE,
Shovan Maity, Student Member, IEEE, Kaushik Roy, Fellow, IEEE and Shreyas Sen, Member, IEEE School of Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana – 47907, USA E-mail: {bchatte, pandap, maity, kaushik, shreyas}@purdue.edu Abstract—This work presents the design and analysis of a mixed-signal neuron (MS-N) for convolutional neural networks (CNN) and compares its performance with a digital neuron (Dig- N) in terms of operating frequency, power and noise. The circuit- level implementation of the MS-N in 65 nm CMOS technology exhibits 2-3 orders of magnitude better energy-efficiency over Dig- N for neuromorphic computing applications - especially at low frequencies due to the high leakage currents from many transistors in Dig-N. The inherent error-resiliency of CNN is exploited to handle the thermal and flicker noise of MS-N. A system-level analysis using a cohesive circuit-algorithmic framework on MNIST and CIFAR-10 datasets demonstrate an increase of 3% in worst-case classification error for MNIST when the integrated noise power in the bandwidth is ~ 1 µV2.
Keywords—artificial neural network, low-energy, mixed-signal. I. INTRODUCTION The energy-efficiency of the human brain is orders of magnitude better than that of a modern-day von-Neumann computer [1], which necessitates exploring other architectures for applications with different frequency or precision requirements. Neuromorphic computing, inspired by brain, uses artificial neural networks (ANN) for applications such as classification and pattern recognition. As shown later, Dig-Ns [2-4] in an ANN offer a wide frequency of operation, but are not energy-efficient in lower frequencies due to static leakage currents of high number of transistors. On the other hand, the use of mixed-signal circuitry for computing has been a topic of debate for more than two decades [5-6], as the efficiency gained in terms of energy is often overshadowed by the effects of noise and variability. However, due to the presence of multiple distributed connections from input to output in an ANN, it offers inherent error-resiliency towards noise/variability and hence the low power consumption of MS-N can be properly leveraged. MS-N configurations for non-learning spiking neural networks (SNN) are explored in [1, 7] that perform current- switching in large-signal mode. Convolutional neural networks (CNN), however, require a multiply and accumulate (MAC) model for existing learning algorithms (e.g. back-propagation). MAC based architectures implement summation of incoming signals through multiple synapses with different weights followed by an activation function for thresholding (Fig. 1). Though large-signal current-mode MAC structures [8] are available, a small-signal implementation with a fixed current can achieve a better power-bandwidth trade-off. In this work, we present a compact resistor-less differential MS-N with PMOS load which operate in small-signal mode, and is energy-efficient over a large range of bias currents. This paper is organized as follows: Section II presents the architecture of the small-signal MS-N and compares its energy- efficiency with a synthesized Dig-N. Section III describes the system-level framework on MNIST and CIFAR-10 datasets that is used to establish a relationship between the quantization/ thermal/switch noise in the system and the classification error for digit/image recognition applications. Section IV concludes the paper by summarizing our key contributions.
II. MS-N ARCHITECTURE Fig. 2(a) shows the N-bit, differential-amplifier based MS-N architecture with n synaptic weights. The N-bit weights are coming from a digital memory while the MAC operation is performed in an analog fashion, hence the name MS-N. The weights activate switches at the gate of the input transistors while a fixed bias current flows through the k-th synapse (for all k = 1,2,3, … ,n), enabling the small signal operation. Had the bias currents of the individual bits in the k-th synapse been controlled by the weights, the circuit would have operated in large-signal mode, hence (a) changing bandwidth with weight (b) requiring resistive loads for linearity of multiplication. Since the gain of the k-th branch needs to be 1 for supporting multiple synapses in the design, lower currents in the large-signal mode would have had high area penalty for the resistive load. The output of the MS-N is modeled in Eq. (1). Vout = σ(∑gmwkVk) (1) where σ is the transfer function of a di

This content is AI-processed based on ArXiv data.