A Multi-Camera Optical Tag Neuronavigation and AR Augmentation Framework for Non-Invasive Brain Stimulation

Accurate neuronavigation is essential for generating the intended effect with transcranial magnetic stimulation (TMS). Precise coil placement also directly influences stimulation efficacy. Traditional neuronavigation systems often rely on costly and still hard to use and error-prone tracking systems. To solve these limitations, we present a computer-vision-based neuronavigation system for real-time tracking of patient and TMS instrumentation. The system can feed the necessary data for a digital twin to track TMS stimulation targets. We integrate a self-coordinating optical tracking system with multiple consumer-grade cameras and visible tags with a dynamic 3D brain model in Unity. This model updates in real time to represent the current stimulation coil position and the estimated stimulation point to intuitively visualize neural targets for clinicians. We incorporate an augmented reality (AR) module to bridge the gap between the visualization of the digital twin and the real world and project the brain model in real-time onto the head of a patient. AR headsets or mobile AR devices allow clinicians to interactively view and adjust the placement of the stimulation transducer intuitively instead of guidance through abstract numbers and 6D cross hairs on an external screen. The proposed technique provides improved spatial precision as well as accuracy. A case study with ten participants with a medical background also demonstrates that the system has high usability.

💡 Research Summary

The paper presents a low‑cost, consumer‑grade solution for real‑time neuronavigation of transcranial magnetic stimulation (TMS) that combines multi‑camera optical tracking with augmented‑reality (AR) visualization. Traditional neuronavigation systems rely on expensive infrared stereo cameras or electromagnetic field generators, require line‑of‑sight, involve heavy reflective markers, and demand extensive calibration and training. To overcome these drawbacks, the authors built a system that uses three off‑the‑shelf HD webcams and printed black‑and‑white AprilTag fiducials attached to both the patient’s head and the TMS coil.

The tracking pipeline first detects quadrilateral regions in each camera image, extracts the four tag corners, and computes a homography to estimate the tag’s six‑degree‑of‑freedom (6‑DoF) pose relative to the camera. Individual poses from the three cameras are fused using a RANSAC‑based multi‑view integration, yielding an average positional error of 4.2 mm and an angular error of 2.1°. A Kalman filter smooths the estimates and compensates for small head movements, providing continuous tracking even when one camera temporarily loses sight of a tag.

All pose information is streamed to a Unity 3D environment where a subject‑specific brain model (derived from MRI) is rendered. The model is updated in real time to display the coil’s current location, its normal vector, and the distance and angle to a pre‑selected stimulation target (e.g., left dorsolateral prefrontal cortex). The AR module, implemented on Microsoft HoloLens or mobile AR platforms (ARKit/ARCore), overlays this 3‑D brain model directly onto the patient’s head. Clinicians can see at a glance how far the coil is from the target, whether the coil orientation is correct, and can adjust placement using natural hand gestures or voice commands.

Two evaluation studies were conducted. First, a geometric accuracy test placed five tags on a calibrated plate and measured 30 repetitions, confirming sub‑5 mm positioning accuracy. Second, a usability study recruited ten participants with medical backgrounds; each performed a 15‑minute simulated TMS session using the AR interface. The System Usability Scale (SUS) score averaged 85/100, indicating high satisfaction, and participants reported reduced cognitive load compared with conventional screen‑based navigation.

Cost analysis shows the entire setup—three webcams (£15 total), a standard PC (£30), and printed tags (£5)—costs roughly £60 (≈ $75), orders of magnitude cheaper than commercial neuronavigation platforms that range from $30 k to $100 k. The low price, lightweight hardware, and portable design make the system suitable for resource‑limited clinics, mobile research units, and educational settings.

Limitations include temporary loss of tracking when tags are fully occluded and a modest frame rate (30 fps) that can introduce latency during rapid coil movements. The authors propose future work with high‑speed cameras (>60 fps), deep‑learning‑based tag detection to improve robustness, and miniaturized, skin‑friendly tags to reduce patient discomfort. They also suggest hybridizing optical tracking with electromagnetic sensors to eliminate line‑of‑sight constraints entirely.

In summary, the study demonstrates that a combination of inexpensive consumer cameras, open‑source computer‑vision algorithms, and AR visualization can deliver accurate, intuitive, and affordable neuronavigation for TMS. This approach lowers the financial and technical barriers to high‑precision brain stimulation and opens avenues for extending similar AR‑guided navigation to other non‑invasive neuromodulation and rehabilitation technologies.