RimSense: Enabling Touch-based Interaction on Eyeglass Rim Using Piezoelectric Sensors

Xie，Wentao1,2; Chen，Huangxun3; Wei，Jing4; Zhang，Jin5; Zhang，Qian1

2024-01-12

10.1145/3631456

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies   影响因子和分区

2474-9567

Smart eyewear's interaction mode has attracted significant research attention. While most commercial devices have adopted touch panels situated on the temple front of eyeglasses for interaction, this paper identifies a drawback stemming from the unparalleled plane between the touch panel and the display, which disrupts the direct mapping between gestures and the manipulated objects on display. Therefore, this paper proposes RimSense, a proof-of-concept design for smart eyewear, to introduce an alternative realm for interaction - touch gestures on eyewear rim. RimSense leverages piezoelectric (PZT) transducers to convert the eyeglass rim into a touch-sensitive surface. When users touch the rim, the alteration in the eyeglass's structural signal manifests its effect into a channel frequency response (CFR). This allows RimSense to recognize the executed touch gestures based on the collected CFR patterns. Technically, we employ a buffered chirp as the probe signal to fulfil the sensing granularity and noise resistance requirements. Additionally, we present a deep learning-based gesture recognition framework tailored for fine-grained time sequence prediction and further integrated with a Finite-State Machine (FSM) algorithm for event-level prediction to suit the interaction experience for gestures of varying durations. We implement a functional eyewear prototype with two commercial PZT transducers. RimSense can recognize eight touch gestures on the eyeglass rim and estimate gesture durations simultaneously, allowing gestures of varying lengths to serve as distinct inputs. We evaluate the performance of RimSense on 30 subjects and show that it can sense eight gestures and an additional negative class with an F1-score of 0.95 and a relative duration estimation error of 11%. We further make the system work in real-time and conduct a user study on 14 subjects to assess the practicability of RimSense through interactions with two demo applications. The user study demonstrates RimSense's good performance, high usability, learnability and enjoyability. Additionally, we conduct interviews with the subjects, and their comments provide valuable insight for future eyewear design.

eyewear interaction piezoelectric sensor touch gesture

EI

英语

其他

2-s2.0-85182600238

Scopus

1.CSE,Hong Kong University of Science and Technology,Hong Kong,Hong Kong
2.Southern Univ. of Science and Technology,Shenzhen,China
3.IoT Thrust,Information Hub,Hong Kong University of Science and Technology (Guangzhou),Guangzhou,China
4.CIS,University of Melbourne,Melbourne,Australia
5.Shenzhen Key Laboratory of Safety and Security for Next Generation of Industrial Internet,CSE,Southern University of Science and Technology,Shenzhen,China

Xie，Wentao,Chen，Huangxun,Wei，Jing,et al. RimSense: Enabling Touch-based Interaction on Eyeglass Rim Using Piezoelectric Sensors[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies,2024,7(4).

Xie，Wentao,Chen，Huangxun,Wei，Jing,Zhang，Jin,&Zhang，Qian.(2024).RimSense: Enabling Touch-based Interaction on Eyeglass Rim Using Piezoelectric Sensors.Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies,7(4).

Xie，Wentao,et al."RimSense: Enabling Touch-based Interaction on Eyeglass Rim Using Piezoelectric Sensors".Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7.4(2024).