Hardware implementation of photonic neuromorphic autonomous navigation

Reinforcement learning (RL) is a core technology enabling the transition of artificial intelligence (AI) from perception to decision-making, but its deployment on conventional electronic hardware suffers from high latency and energy consumption imposed by the von Neumann architecture. Here, we propose a photonic spiking twin delayed deep deterministic policy gradient (TD3) reinforcement learning architecture for neuromorphic autonomous navigation and experimentally validate it on a distributed feedback laser with a saturable absorber (DFB-SA) array. The hybrid architecture integrates a photonic spiking Actor network with dual continuous-valued Critic networks, where the final nonlinear spiking activation layer of the Actor is deployed on the DFB-SA laser array. In autonomous navigation tasks, the system achieves an average reward of 58.22 plus-minus 17.29 and a success rate of 80% plus-minus 8.3%. Hardware-software co-inference demonstrates an estimated energy consumption of 0.78 nJ/inf and an ultra-low latency of 191.20 ps/inf, with co-inference error rates of 0.051% and 0.059% in task scenarios with and without obstacle interference, respectively. Simulations for error-activated channels show full agreement with the expected responses, validating the dynamic characteristics of the DFB-SA laser. The architecture shows strong potential for integration with large-scale photonic linear computing chips, enabling fully-functional photonic computation and low-power, low-latency neuromorphic autonomous navigation.

💡 Research Summary

**
The paper presents a novel photonic spiking reinforcement‑learning framework that combines the Twin‑Delayed Deep Deterministic Policy Gradient (TD3) algorithm with a photonic spiking neural network (PSNN) to achieve neuromorphic autonomous navigation. The authors address the fundamental bottleneck of conventional electronic von Neumann processors—high latency and energy consumption caused by the separation of memory and computation—by moving both linear and nonlinear operations into the optical domain.

System Architecture
The RL agent consists of a PSNN‑based Actor and two conventional artificial‑neural‑network (ANN) Critics. The Actor receives the robot’s state (LiDAR‑derived obstacle distances, goal position, etc.), encodes it into spike trains, processes it through multiple leaky‑integrate‑fire (LIF) layers, and finally aggregates the temporal spikes to produce continuous linear and angular velocity commands via a tanh scaling. The Critic networks evaluate the state‑action pair and provide Q‑value estimates for policy updates. The TD3 algorithm is employed because its twin‑Critic design, target‑policy smoothing, and delayed policy updates improve stability and sample efficiency for continuous‑control tasks.

Hardware‑Aware Design
To make the Actor compatible with existing photonic linear‑computing platforms (e.g., microring resonator weight banks, Mach‑Zehnder interferometer meshes), the authors constrain all linear weights to be positive and eliminate bias terms during software pre‑training. This enables a direct mapping of the learned weight matrices onto photonic matrix‑vector multiplication chips. The only non‑linear operation—the spiking activation layer—is realized in hardware using a distributed‑feedback laser with a saturable absorber (DFB‑SA) array. The DFB‑SA laser exhibits neuron‑like dynamics: a threshold current, temporal integration of input pulses, and a refractory period, thereby providing an optical analogue of the LIF activation function.

Hardware‑Software Co‑Inference
The workflow proceeds as follows: (1) the Actor network is pre‑trained in software with a surrogate gradient method (single‑time‑step, T = 1) while respecting the positive‑weight, bias‑free constraints; (2) the trained linear layers are mapped onto a photonic linear processor; (3) the final spiking activation is off‑loaded to the DFB‑SA laser array; (4) input vectors are converted to optical signals, injected into the laser, and the resulting optical spikes are detected, digitized, and fed to the software Critic for gradient computation. This hybrid inference scheme allows the authors to measure the true physical latency and energy of the optical activation while keeping the rest of the RL pipeline in a conventional digital environment.

Experimental Validation
The authors evaluate the system in a Gazebo simulation of a mobile robot equipped with LiDAR, performing map‑less dynamic obstacle avoidance and goal‑directed navigation. In pure‑software simulations the system achieves an average cumulative reward of 58.22 ± 17.29 and a navigation success rate of 80 % ± 8.3 %. When the spiking activation layer is executed on the DFB‑SA hardware, the same performance metrics are reproduced, confirming that the optical neuron does not degrade learning quality. Measured hardware metrics are striking: an estimated inference energy of 0.78 nJ per inference and a latency of 191.20 ps per inference, yielding error rates of only 0.051 % (with obstacles) and 0.059 % (without obstacles). The authors also compare the measured laser responses to simulations based on the Yamamura model, finding full agreement and thereby validating the physical model of the DFB‑SA device.

Significance and Outlook
This work demonstrates the feasibility of integrating photonic linear computing with a laser‑based spiking nonlinearity to build a fully‑functional optical RL agent capable of real‑time autonomous navigation. The ultra‑low latency (sub‑nanosecond) and ultra‑low energy consumption (sub‑nanojoule) are orders of magnitude better than state‑of‑the‑art electronic neuromorphic chips, suggesting a path toward energy‑constrained autonomous systems such as drones, micro‑robots, and edge‑mounted vehicles.

However, the current prototype is limited to a modest 24 × 128 × 128 linear matrix size and only the Actor’s activation is optical; the Critics remain digital. Scaling to larger networks will require addressing optical routing losses, thermal stability of the lasers, and multi‑wavelength synchronization. Future research directions include (i) expanding both Actor and Critic to fully photonic implementations on larger matrix‑vector processors, (ii) developing robust bias‑free photonic weight banks with higher resolution, (iii) integrating multi‑channel DFB‑SA arrays for parallel policy evaluation, and (iv) deploying the system on physical robotic platforms to validate real‑world robustness.

In summary, the paper provides a compelling proof‑of‑concept that photonic spiking neurons, when combined with modern RL algorithms, can deliver neuromorphic autonomous navigation with unprecedented speed and energy efficiency, opening a new frontier for photonic AI hardware.