Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application

With the increasing complexity of mobile device applications, these devices are evolving toward high agility. This shift imposes new demands on mobile sensing, particularly in achieving high-accuracy and low-latency. Event-based vision has emerged as a disruptive paradigm, offering high temporal resolution and low latency, making it well-suited for high-accuracy and low-latency sensing tasks on high-agility platforms. However, the presence of substantial noisy events, lack of stable, persistent semantic information, and large data volume pose challenges for event-based data processing on resource-constrained mobile devices. This paper surveys the literature from 2014 to 2025 and presents a comprehensive overview of event-based mobile sensing, encompassing its fundamental principles, event \textit{abstraction} methods, \textit{algorithm} advancements, and both hardware and software \textit{acceleration} strategies. We discuss key \textit{applications} of event cameras in mobile sensing, including visual odometry, object tracking, optical flow, and 3D reconstruction, while highlighting challenges associated with event data processing, sensor fusion, and real-time deployment. Furthermore, we outline future research directions, such as improving the event camera with advanced optics, leveraging neuromorphic computing for efficient processing, and integrating bio-inspired algorithms. To support ongoing research, we provide an open-source \textit{Online Sheet} with recent developments. We hope this survey serves as a reference, facilitating the adoption of event-based vision across diverse applications.

💡 Research Summary

This survey paper provides a comprehensive review of event‑based vision for mobile embodied perception, covering literature from 2014 to 2025. It begins by motivating the need for high‑accuracy, low‑latency sensing on high‑agility platforms such as drones, autonomous robots, and smart‑city agents, and explains why conventional frame cameras, LiDAR, and mmWave radar fall short in terms of frame rate, latency, dynamic range, or power consumption. The authors then describe the fundamental operating principle of event cameras: asynchronous detection of logarithmic intensity changes at individual pixels, producing a stream of events defined by pixel location, timestamp, and polarity when a configurable threshold is exceeded. This mechanism yields microsecond‑level temporal resolution, sub‑millisecond perception latency, a 140 dB dynamic range, and low power draw (~0.5 W), making event cameras uniquely suited for fast, power‑constrained mobile agents.

A detailed hardware overview follows, including the CMOS sensor stack, on‑chip thresholding circuitry, and read‑out architecture. The paper surveys commercial products (e.g., Prophesee Metavision, Samsung ISOC) and benchmark datasets (EV‑Flow, DSEC, MVSEC) that have become standard for evaluating event‑based algorithms.

The core of the survey is organized around four pillars: abstraction, algorithm, acceleration, and application. In the abstraction section, five major event representations are compared—individual events, event packets, event frames, time‑surfaces, and 3‑D spatio‑temporal grids. The authors discuss each representation’s memory footprint, computational complexity, and ability to preserve spatial‑temporal continuity, highlighting trade‑offs such as the low latency of event packets versus the compatibility of event frames with existing vision pipelines.

The algorithmic pillar is broken into six processing stages: denoising, filtering & feature extraction, matching, mapping, and high‑level perception (visual odometry, optical flow, 3‑D reconstruction). State‑of‑the‑art methods are reviewed, ranging from classic spatio‑temporal filters and event‑based corner detectors to deep neural networks for event denoising (e.g., EVDenoiseNet) and transformer‑based feature encoders. The survey contrasts geometry‑driven approaches (e.g., EKF, ICP) with learning‑driven counterparts, evaluating them on accuracy, latency, and power consumption.

Acceleration strategies are examined in both hardware and software domains. Hardware acceleration includes FPGA pipelines, ASIC designs (DynapSE, Intel Loihi), and emerging neuromorphic chips that can ingest raw events directly, thereby eliminating host‑CPU bottlenecks and achieving end‑to‑end latencies below 0.5 ms. Software acceleration covers CUDA‑based parallelism, OpenCL, and specialized event‑processing libraries (evsdk, ESIM). The authors present optimization techniques such as event batching, memory pooling, and kernel fusion that reduce CPU utilization to under 30 % while maintaining real‑time throughput.

Application areas are illustrated with concrete use cases. For high‑speed drone navigation, event‑based visual odometry and optical flow achieve centimeter‑level positioning with sub‑millisecond update rates. In autonomous driving, event cameras complement LiDAR and IMU to provide robust perception under low‑light and high‑dynamic‑range conditions. Humanoid robots benefit from event‑based object tracking and 3‑D mapping for rapid hand‑eye coordination. Smart‑city deployments exploit the sparsity of event streams to lower communication bandwidth in large‑scale sensor networks. In each scenario, the survey emphasizes the necessity of sensor fusion, temporal synchronization, and online calibration to meet stringent accuracy (mm) and latency (ms) requirements.

Finally, the paper outlines four future research directions: (1) advanced optics and high‑resolution pixel designs to improve spatial fidelity; (2) neuromorphic sparse transformers and other event‑native deep learning models for efficient inference; (3) bio‑inspired memory architectures that emulate synaptic variability and sparse coding, enabling ultra‑low‑power ASICs; and (4) holistic hardware‑software co‑design frameworks that jointly optimize camera, processor, and memory interfaces. An open‑source “Online Sheet” is provided to keep the community updated with the latest developments.

Overall, the survey positions event‑based vision as a pivotal technology for next‑generation mobile embodied perception, offering a clear roadmap for researchers and engineers to develop accurate, low‑latency, and energy‑efficient perception systems on resource‑constrained, high‑agility platforms.