Hybrid PS-V Technique: A Novel Sensor Fusion Approach for Fast Mobile Eye-Tracking with Sensor-Shift Aware Correction

 Abstract—This paper introduces and evaluates a hybrid technique that fuses efficiently the eye-tracking principles of photosensor oculography (PSOG) and video oculography (VOG). The main concept of this novel approach is to use a few fast and power-economic photosensors as the core mechanism for performing high speed eye-tracking, whereas in parallel, use a video sensor operating at low sampling-rate (snapshot mode) to perform dead-reckoning error correction when sensor movements occur. In order to evaluate the proposed method, we simulate the functional components of the technique and present our results in experimental scenarios involving various combinations of horizontal and vertical eye and sensor movements. Our evaluation shows that the developed technique can be used to provide robustness to sensor shifts that otherwise could induce error larger than 5°. Our analysis suggests that the technique can potentially enable high speed eye-tracking at low power profiles, making it suitable to be used in emerging head- mounted devices, e.g. AR/VR headsets.

Index Terms— hybrid eye-tracking, photosensor oculography, sensor fusion, sensor shift correction, video oculography

I. INTRODUCTION ye-tracking is expected to become an essential tool for seamless human-computer interaction (HCI) in modern head-mounted devices. For example, in the case of AR/VR headsets, eye-tracking can substantially improve the immersion and the overall user experience by enabling applications like foveated rendering [1], saccade-contingent screen updating [2], touchless interaction [3], and assist on the prevention of eye fatigue [4] and cybersickness [5]. In order to meet the demands of the growing mobile AR/VR ecosystems, two very important requirements for eye-tracking systems aiming to enable such applications are high tracking speed and relatively low power consumption.

I. Rigas and O.V. Komogortsev are with Texas State University, Department of Computer Science, 601 University Dr, San Marcos, TX 78666 USA, (e-mails: rigas.ioann@gmail.com; ok@txstate.edu). H. Raffle is with Google, 1600 Amphitheater Drive, Mountain View, CA 94043, USA, (e-mail: hraffle@google.com).

Most current eye-tracking systems are based on the principle of video oculography (VOG). In a typical VOG implementation [6], the eye is illuminated by one (or more) infrared LED(s), and consecutive images of the eye are captured and processed to extract important features, e.g. pupil center and corneal reflection. The differences in position of these features can be used to estimate eye movement with relative robustness to small sensor movements. The systems based on VOG can provide high accuracy during gaze estimation but they have certain limitations when high speed eye-tracking is needed combined to low power consumption. These limitations arise from the need to capture and process multiple images, a procedure with considerable burden for computational resources. For binocular eye-tracking these demands and the overall cost become further inflated. A number of alternative eye-tracking methods have been explored in the past, with the most prominent being: a) the magnetic scleral coil method [7], b) electrooculography (EOG) [8], and c) photosensor oculography (PSOG) [9]. Among them, PSOG appears to fulfill many of the eye- tracking needs posed by modern headsets. The principle of PSOG is based on the direct measurement of the amount of reflected light from the eye using simple pairs of photo- sensitive sensors. A major advantage of PSOG when compared to VOG is the minimal computational burden (just a few computations to combine sensor outputs) that can enable eye-tracking with high sampling rate and low power consumption. Also, the PSOG does not need any attachment on the eye or skin making it less obtrusive than the magnetic scleral coil method and EOG. Despite these obvious advantages, PSOG has also its Achilles’ heel: it is very sensitive to sensor shifts. Most headsets use head-straps to limit excessive mobility, however, small sensor movements can still occur due to facial expressions or body movements (e.g. during jumping, walking). Such sensor shifts can result in considerable degradation of accuracy for the traditional implementations of PSOG. In this work, we propose a new approach for addressing the limitations of traditional photosensor oculography and video oculography systems by selectively combining the best characteristics from both worlds. The key contributions of this work are: Hybrid PS-V Technique: A Novel Sensor Fusion Approach for Fast Mobile Eye-Tracking with Sensor-Shift Aware Correction Ioannis Rigas, Hayes Raffle, and Oleg V. Komogortsev E

…(Full text truncated)…