智能沙盘人机交互系统研发

RESEARCH AND DEVELOPMENT OF HUMAN-COMPUTER INTERACTION SYSTEM BASED ON INTELLIGENT SANDBOX

刁学磊

11849320

硕士

计算机技术领域工程

杨双华

2020-05-30

2020-07-20

哈尔滨工业大学

深圳

现有的智能沙盘教学演示系统仅支持鼠标键盘等传统的交互方式，为进一步提高用户沉浸感和参与感，本文研发了基于手势操作的智能沙盘人机交互系统。智能沙盘人机交互系统采用增强现实技术、三维图形建模相关技术、体感开发技术、手势识别技术。本文将手势识别技术运用到智能沙盘上。用自然直接的手势代替鼠标键盘的输入，在无需佩戴任何交互式设备的情况下，就可获得交互式的增强现实体验。本文在Kinect v1.8的基础上实现了单手六种手势的识别、多手手势识别和动态手势识别，从而实现了丰富的智能沙盘的人机交互功能。可根据用户不同的手势触发不同的天气系统、自然灾害系统，以及判断手势与沙盘平台以及平台上虚拟物体的相对位置，识别动态手势掌心轨迹，进行放置移动沙盘上的虚拟物体等功能。在智能沙盘教学演示系统的基础上，智能沙盘人机交互系统研发了包含地形同步系统、天气系统、自然灾害系统、军事系统、手势交互系统和图形界面系统共六大系统。包含基础功能、场景功能、交互功能、同步功能共四大功能模块。智能沙盘人机交互系统将手势识别技术运用到智能沙盘上，进一步完善了智能沙盘的功能，进一步提高智能沙盘系统的沉浸感，提高用户群体的参与感。激发了用户的学习兴趣和创新热情，培养了用户的创造力、想象力和实践能力。

The existing intelligent sandbox teaching demonstration system only supports traditional interaction methods such as mouse and keyboard. In order to further improve the user's sense of immersion and participation, this paper develops an intelligent sandbox human-computer interaction system based on gesture operation.The intelligent sandbox human-computer interaction system uses augmented reality technology, 3D graphics modeling related technology, somatosensory development technology, and gesture recognition technology. This paper applies gesture recognition technology to intelligent sandbox. Use natural and direct gestures instead of mouse and keyboard input to get an interactive augmented reality experience without wearing any interactive equipment.  Based on Kinect v1.8, this paper realizes the recognition of six gestures with one hand, multi-hand gesture recognition, and dynamic gesture recognition, thereby realizing the rich human-computer interaction function of the intelligent sandbox. It can trigger different weather systems and natural disaster systems according to different gestures of users, determine the relative positions of gestures and sandbox platforms and virtual objects on the platform, identify the palm trajectory of dynamic gestures, and place and move virtual objects on the sandbox. On the basis of the intelligent sandbox teaching demonstration system, the intelligent sandbox human-computer interaction system has developed six systems including terrain synchronization system, weather system, natural disaster system, military system, gesture interaction system and graphical interface system. Including basic functions, scene functions, interactive functions, synchronization functions, a total of four major functional modules.The intelligent sandbox human-computer interaction system applies gesture recognition technology to the intelligent sandbox, which further improves the functions of the intelligent sandbox, further improves the immersion of the intelligent sandbox system, and increases the sense of participation of the user group. Stimulated the user's interest in learning and innovation, cultivated the user's creativity, imagination and practical ability.

智能沙盘 人机交互 Kinect 手势识别

intelligent sandbox human-computer interaction Kinect gesture recognition

中文

联合培养

南方科技大学

刁学磊. 智能沙盘人机交互系统研发[D]. 深圳. 哈尔滨工业大学,2020.