A Smart Mirror Based on Augmented Reality, Gesture Recognition and Deep Learning Techniques

107

Augmented Reality (AR) is becoming more and more common on our mobile devices. AR technology helps users get information in an interactive way, improving convenience and increasing entertainment, attracting more people. Today's mobile devices are already mature enough to use AR technology. However, when we applied AR on a smart mirror, the insensitivity of the screen or the limitation of distance might cause inconvenience to the user's operation. There are few references for the operation to switch AR object in a smart mirror. Therefore, this paper develops a smart mirror based on gesture recognition system. We hope to perform a non-contact operation by gestures to improve the user experience. The model of gesture recognition is trained by deep learning in this thesis. Our smart mirror also adds AR effects, so that users can feel more immersive. Moreover, the system uses gesture to switch AR objects or give a snapshot instruction. Finally, the photo is transmitted to the cloud and generated QR code to the user. Therefore, the system of this work solves the operational problems and improves the user experience by gesture recognition. In this thesis, we verified the models trained by the number of different data sets. The more data sets, the better the accuracy of gesture recognition. We also experimented with the results of gestures recognized by users at different distances from the system. Finally, through the questionnaire survey, the system can improve the user experience through gesture recognition.