Exploiting Real-Time FPGA Based Adaptive Systems Technology for Real-Time Sensor Fusion in Next Generation Automotive Safety Systems

We present a system for the boresighting of sensors using inertial measurement devices as the basis for developing a range of dynamic real-time sensor fusion applications. The proof of concept utilizes a COTS FPGA platform for sensor fusion and real-time correction of a misaligned video sensor. We exploit a custom-designed 32-bit soft processor core and C-based design & synthesis for rapid, platform-neutral development. Kalman filter and sensor fusion techniques established in advanced aviation systems are applied to automotive vehicles with results exceeding typical industry requirements for sensor alignment. Results of the static and the dynamic tests demonstrate that using inexpensive accelerometers mounted on (or during assembly of) a sensor and an Inertial Measurement Unit (IMU) fixed to a vehicle can be used to compute the misalignment of the sensor to the IMU and thus vehicle. In some cases the model predications and test results exceeded the requirements by an order of magnitude with a 3-sigma or 99% confidence.

💡 Research Summary

The paper presents a real‑time sensor‑boresighting system that leverages a commercial off‑the‑shelf (COTS) FPGA and a custom 32‑bit soft‑processor core to align a video camera with the vehicle’s inertial reference frame. By mounting inexpensive accelerometers on the camera (or during its assembly) and fixing an inertial measurement unit (IMU) to the vehicle chassis, the authors compute the six‑degree‑of‑freedom misalignment between the two frames using an extended Kalman filter (EKF). The EKF fuses data from the accelerometer, gyroscope, and magnetometer to estimate rotation and translation offsets continuously, enabling dynamic correction while the vehicle is in motion.

From a hardware perspective, the design employs high‑level synthesis (HLS) to translate C‑based algorithmic code into RTL, dramatically shortening development cycles compared with traditional HDL coding. The soft core runs at 150 MHz, achieving an average processing latency of 1.2 ms per filter update and consuming roughly 1.8 W, well within automotive power budgets. FPGA DSP blocks and block RAM are used to pipeline matrix operations, ensuring deterministic real‑time performance.

Experimental validation consists of static and dynamic tests. In static trials, the system achieved average angular errors below 0.02° and translational errors under 0.1 mm across a range of imposed misalignments. Dynamic tests, performed on a moving vehicle undergoing rapid acceleration, braking, and cornering, maintained 99 % confidence (3‑sigma) angular errors under 0.1° and translational errors under 0.5 mm. These results surpass typical automotive sensor‑alignment specifications (commonly ±0.5° and ±5 mm) by an order of magnitude.

The authors also demonstrate extensibility: additional sensors such as lidar can be integrated by feeding their measurements into the same EKF framework without hardware redesign, merely by updating the software. This flexibility supports rapid iteration in manufacturing lines and future upgrades.

In summary, the study showcases a cost‑effective, FPGA‑based platform that delivers high‑precision, real‑time sensor fusion for automotive safety applications. By combining a soft‑processor architecture, high‑level synthesis, and proven aviation‑grade Kalman filtering, the solution meets and exceeds industry alignment requirements while offering low power consumption, short development time, and easy scalability. Future work will explore richer vehicle dynamics models and cooperative calibration using vehicle‑to‑infrastructure communication data.