Background
Hand Redirection is a technique that cleverly takes advantage of how the human perceptual system works in order to let virtual reality (VR) applications "control" the user's real hand movement. Through hand redirection, VR applications can manipulate where users reach with their hand in the real world while they reach for a virtual target in the virtual environment. In other words, the goal of hand redirection is to redirect a person's hand while it is in motion - letting a user grasp a virtual object with the virtual hand, while actually grasping a (potentially displaced!) physical object with the real hand.
In a VR system, people can only see a virtual representation of their hand. Normally, this virtual hand is rendered exactly where the physical hand is located. Some hand redirection techniques, however, break with this 1-to-1 mapping of the real and virtual hand without the user noticing it. This works because visual feedback often dominates the human senses when there's a mismatch between the virtual and the real world (a perceptual phenomenon known as visual dominance).
If, for example, a user's virtual hand is slightly shifted to the right while it is in motion, the user unknowingly compensates this shift by moving the real hand further to the left. Doing so every frame results in a discrepancy between the virtual and the real hand and redirects the user's real reach motion towards the right while the virtual hand moves forward. The animation below shows an example. Blue represents the real hand/object and green represents the virtual hand/object.
The approach shown in this animation is called Body Warping and is one type of hand redirection technique. In addition to body warping, also other approaches to redirect the user's hand exist. Another method, for example, maintains a 1-to-1 mapping between the virtual and the real hand but manipulates the way the virtual environment is rotated and positioned relative to the user, or technically manipulates the user's orientation and position within the virtual environment (which effectively does the same trick). By applying a gain factor to increase or decrease a user's head rotations, for example, the virtual environment can be transformed to establish alignment between virtual objects and their (initially dislocated) physical counterparts. Similarly, we can also apply gains to head translations when the user moves to achieve alignment of virtual and real objects. For example, if we want a user to rotate around 90 degrees in the virtual world but around 180 degrees in the real world, we can scale the user's head rotations down by 50%. Such techniques are known as World Warping and are related to the concepts of redirected walking.
While research has shown that hand redirection techniques are useful to build more ergonomic 3D user interfaces and interactions, hand redirection is also a great way to incorporate prop-based haptic feedback into VR applications. In order to receive haptic feedback for the virtual objects in a VR environment, we can leverage tangible props, i.e. physical counterparts, in the real world - a concept known as passive haptic feedback or proxy-based haptics. But creating a 1-to-1 mapping between all virtual objects in a scene with dedicated physical objects in the real environment does not scale well. Instead, it is often desirable to reuse a few physical objects (props) for multiple virtual objects. This can be achieved through Haptic Retargeting.
To implement haptic retargeting, hand redirection techniques can be used. When reaching for a virtual object represented by a prop in the real world, hand redirection is used to redirect a user's hand such that the real hand touches the physical prop while the virtual hand touches the virtual counterpart - even if they are spatially separated!
Unity has become increasingly popular in Human-Computer Interaction (HCI) research in the last years. It is often used as a platform to create VR applications and to perform related user studies (e.g. studying the hand redirection and proxy-based haptic feedback). Additionally, Unity supports most Virtual Reality (VR) and Augmented Reality (AR) devices and frameworks. Because of this, Unity is our first choice for making a hand redirection framework available to the VR development and research community.
On the one hand, the toolkit provides reference implementations of several different hand redirection techniques (such as the most common Body Warping, World Warping, and 3D Interpolation approaches). Using these implementations makes integrating hand redirection in VR applications much more simple.
On the other hand, the toolkit can help developing novel hand redirection approaches. Creating a new hand redirection approach does not only involve finding, implementing, and testing new algorithms but also building a complete test environment. This can be very time-consuming and often ends in hardcoded scenarios that do work well for one approach but do not guarantee an easy extension for other approaches.
With our hand redirection toolkit, we want to provide a modular system that allows users of the toolkit to create new hand redirection techniques and test them in a predefined test environment. Users can compare different approaches on the fly and only have to insert their algorithm into a given framework. They can then directly use their implementation in a couple of example scenarios or create their own scenarios. Moreover, via this repository, they can share their implementations with the community.
As setting up a VR system for quick tests and development can be tedious, our toolkit also provides functionality to simulate (and visualize) body and hand movements by only using the keyboard and mouse. Furthermore, many existing redirection approaches do build upon each other. A unified platform where the most common approaches are implemented can help to ease the development of novel techniques or combinations.
Finally, a set of visualizations is also included in the toolkit, which can be handy while analyzing new approaches, finding bugs, or trying something out.
Since Hand Redirection has no wide adoption in VR applications yet, and is mostly used in research so far, this toolkit on the one hand aims to help researchers in developing and deploying hand redirection approaches. On the other hand, we also invite VR developers interested in hand redirection to make use of the toolkit. In order to use this toolkit, users should be familiar with the development in Unity and VR.
The toolkit for hand redirection was inspired by the Redirected Walking Toolkit (by Azmandian et al.) - a similar toolkit developed for Redirected Walking approaches in VR. For more information, check out our their Toolkit and Paper