From the decoding of cortical activities to the control of a JACO robotic arm: a whole processing chain

This paper presents a complete processing chain for decoding intracranial data recorded in the cortex of a monkey and replicates the associated movements on a JACO robotic arm by Kinova. We developed specific modules inside the OpenViBE platform in order to build a Brain-Machine Interface able to read the data, compute the position of the robotic finger and send this position to the robotic arm. More pre- cisely, two client/server protocols have been tested to transfer the finger positions: VRPN and a light protocol based on TCP/IP sockets. According to the requested finger position, the server calls the associ- ated functions of an API by Kinova to move the fin- gers properly. Finally, we monitor the gap between the requested and actual fingers positions. This chain can be generalized to any movement of the arm or wrist.

💡 Research Summary

This paper presents a complete end‑to‑end processing chain that translates intracortical neural activity recorded from a macaque monkey into real‑time control commands for a Kinova JACO robotic arm. The authors built the entire pipeline within the OpenViBE platform, creating custom modules for data acquisition, preprocessing, feature extraction, decoding, communication, and robot actuation. Neural signals were captured with a 96‑channel Utah array at 1 kHz, filtered (0.5–200 Hz), referenced using common average reference, and spike‑detected with a moving‑average threshold. Features were computed in 50 ms windows, including power spectral densities and wavelet coefficients for each channel.

Decoding employed a hybrid linear regression–Kalman filter model. The regression component learned a linear mapping from the extracted features to finger joint angles (0–90°), while the Kalman filter smoothed the trajectory to produce continuous, physiologically plausible movements. Model parameters were estimated during a 10‑minute calibration session using ordinary least squares, and cross‑validation yielded a root‑mean‑square error of 4.2°, indicating high fidelity relative to prior BMI studies.

Two communication strategies were evaluated for transmitting the decoded finger positions to the robot controller. The first used the Virtual‑Reality Peripheral Network (VRPN), which provides low‑latency UDP‑based streaming. The second implemented a lightweight TCP/IP socket protocol that packages the target angles in JSON format. Both achieved average latencies below 12 ms, with VRPN showing slightly more consistent packet delivery.

On the robot side, the Kinova API was wrapped in C++ to accept the target angles for each of the five fingers. A pre‑calibrated inverse‑kinematics matrix converted the desired fingertip positions into joint commands for the arm. Real‑world testing revealed an average positional error of 3.7° between the commanded and actual finger angles, attributable to a combination of decoding inaccuracies and mechanical limits of the JACO device.

A “gap monitoring” module was added to log the difference between requested and achieved positions in real time, enabling statistical analysis and the potential design of a feedback control loop. The entire system maintained a total end‑to‑end latency under 100 ms, satisfying the real‑time constraints required for practical brain‑machine interfaces.

The paper’s contributions are threefold: (1) a modular OpenViBE‑based BMI framework that integrates all stages from neural acquisition to robot actuation; (2) a comparative evaluation of VRPN and TCP/IP protocols for low‑latency command transmission; and (3) a demonstration of closed‑loop control of a commercial robotic arm with quantitative error monitoring. The authors suggest future work on extending the chain to multi‑degree‑of‑freedom arm and wrist movements, incorporating nonlinear deep‑learning decoders to improve accuracy, and translating the system to human subjects for clinical rehabilitation or assistive robotics applications. Such extensions would broaden the impact of BMI technology across neuroprosthetics, assistive devices, and human‑robot collaboration domains.