Extending Smartphone-Based Hand Gesture Recognition for Augmented Reality Applications with Two-Finger-Pinch and Thumb-Orientation Gestures
Document Type
Article
Publication Date
2022
Abstract
Augmented Reality (AR) on smartphones, while limited compared to their headset-styled counterparts (e.g., Microsoft HoloLens and Magic Leap), may be improved by supporting Hand Gesture Recognition (HGR) without requiring the expensive hardware found on the said headsets. This study improves the overall smartphone support of an existing HGR framework, named Augmented-reality Mobile Device Gestures (AMDG), and adds two new gestures: two-finger-pinch and thumb-orientation gestures, which are alternatives to existing gestures that may help improve overall user experience. A user study is also conducted to evaluate the usability of these two gestures in a realistic scenario. This study finds that the two-finger-pinch gesture is possibly more usable than the air-tap gesture (popularized by HoloLens) for the purpose of moving AR objects in a scene, and that the thumb-orientation gesture is a potentially viable simplification of whole-hand orientation (with some caveats relating to ease of execution and detection).
Recommended Citation
Vidal, Jr., E.C.E., & Rodrigo, M.M.T. (2022). Extending Smartphone-Based Hand Gesture Recognition for Augmented Reality Applications with Two-Finger-Pinch and Thumb-Orientation Gestures. In J.Y.C. Chen, G. Fragomeni, H. Degen, & S. Ntoa (Eds.), HCI International 2022 – Late Breaking Papers: Interacting with eXtended Reality and Artificial Intelligence. HCII 2022. Lecture Notes in Computer Science, vol 13518. Springer. https://doi.org/10.1007/978-3-031-21707-4_15
