Pinch gesture interaction in the peripersonal space using VST smartphone-based HMDs

Abstract

Augmented Reality (AR) applications are known for using unconventional devices and enabling different ways to interact with the virtual elements of the environment. Head-mounted displays (HMDs), for example, enrich the user experience by providing greater immersion than other display devices. Most conventional HMDs are expensive or have restrictions, such as a limited field of view, making them inappropriate for many applications. With the increased computational power on mobile, some HMD models integrate conventional smartphones into the equipment. Smartphone-based HMDs have the advantage of being cheaper, so they can decrease the cost of AR systems. In addition, smartphone-based HMDs have the processing power, field of view, and camera quality required for most AR applications. Many applications that can use smartphone-based HMDs use the peripersonal space, distances of up to 1 meter from the user, as a region to interact with the virtual objects of the environment. As it is a specific region, it is essential to evaluate whether the device’s limitations affect the performance of any interaction within this space. In this work, we created a system that combines a depth sensor and an HMD device to evaluate whether the user’s performance using pinch gestures to interact inside the peripersonal space is uniform when using a smartphone-based HMD in AR applications. The results showed that such equipment is suitable for applications where interaction occurs under these conditions, as the user’s performance in similar tasks is not affected when the interaction occurs within the peripersonal space.