Eye-gaze Estimation with HEOG and Neck EMG using Deep Neural Networks

Fu，Zhen1; Wang，Bo1; Chen，Fei2; Wu，Xihong1,3; Chen，Jing1,3

10.23919/EUSIPCO54536.2021.9616059

2021

2219-5491

978-1-6654-0900-1

European Signal Processing Conference

2021-August

1261-1265

23-27 Aug. 2021

Dublin, Ireland

Hearing-impaired listeners usually have troubles attending target talker in multi-talker scenes, even with hearing aids (HAs). The problem can be solved with eye-gaze steering HAs, which require listeners eye-gazing on the target. In a situation where head rotates, eye-gaze is subject to both behaviors of saccade and head rotation. However, existing methods of eye-gaze estimation did not work reliably, since the listener's strategy of eye-gaze varies and measurements of the two behaviors were not properly combined. Besides, existing methods were based on hand-craft features, which could overlook some important information. In this paper, a head-fixed and a head-free experiment were conducted. We used horizontal electrooculography (HEOG) and neck electro-myography (NEMG), which separately measured saccade and head rotation to jointly estimate eye-gaze. Besides traditional classifier and hand-craft features, deep neural networks (DNN) were introduced to automatically extract features from intact waveforms. Evaluation results showed that when the input was HEOG with inertial measurement unit, the best performance of our proposed DNN classifiers achieved 93.3%; and when HEOG was with NEMG together, the accuracy reached 72.6%, higher than that with HEOG (71.0%) or NEMG (35.7%) alone. These results indicated the feasibility to estimate eye-gaze with HEOG and NEMG.

Deep neural network HEOG IMU Neck EMG

其他

英语

EI

National Natural Science Foundation of China[12074012];National Natural Science Foundation of China[61771023];National Natural Science Foundation of China[U1713217];

20220411505834

Audition ; Eye movements ; Hearing aids ; Signal processing

Ergonomics and Human Factors Engineering:461.4 ; Rehabilitation Engineering and Assistive Technology:461.5 ; Information Theory and Signal Processing:716.1 ; Acoustic Devices:752.1 ; Acoustical Instruments:941.1

2-s2.0-85123161739

Scopus

1.Department of Machine Intelligence,Speech and Hearing Research Center,Key Laboratory of Machine Perception (Ministry of Education),Peking University,Beijing,China
2.Department of Electrical and Electronic Engineering,Southern University of Science and Technology,Shenzhen,China
3.Peng Cheng Laboratory,Shenzhen,China

Fu，Zhen,Wang，Bo,Chen，Fei,et al. Eye-gaze Estimation with HEOG and Neck EMG using Deep Neural Networks[C],2021:1261-1265.