Reality Is What You Do (Not What You See)

“Being there” means the capability to act there. 

“Being there” means the capability to act there. 

Your perception of reality is based on what you can do.  When you are inside of a VR environment, the more functionality that you have, the more the experience resembles your every day life.  You believe an object is real when you can interact with it, not just when you see it.  

Presence is defined as a sense of “being there,” or the extent to which virtual environments are perceived as places visited rather than images seen.  If you accept that presence is a design ideal for VR environments, there are systematic ways to increase users’ feelings of it.  Here I review two scientific papers on using body movement to heighten presence.  

Locomotion: Walking in Place vs. Using a Mouse

The degree of presence depends on the match between proprioceptive and sensory data.  Researchers at the University of London asked people to walk in place while they were inside of a virtual experience.  The gaze of the participant in the HMD determined what direction people felt they were walking in.  They compared the walking-in-place experience against the use of a computer mouse for locomotion.  The researchers believed that walking in place offered an advantage because is that it doesn’t require people to use their hands for navigation.  

The hand may be entirely reserved for the purposes for which it is used in everyday reality, that is, the manipulation of objects and activation of controls.
— Slater, Usoh & Steed (1995)
Valve's The Lab uses a teleportation function built into the hand controllers.  The user points a light beam where they want to go and release the trigger to teleport.  In this case, the user will land in the green circle in front of them.

Valve's The Lab uses a teleportation function built into the hand controllers.  The user points a light beam where they want to go and release the trigger to teleport.  In this case, the user will land in the green circle in front of them.

From the Slater, Usoh, M. and Steed, A. (1995) article: 

A fundamental requirement for an effective virtual reality is, therefore, that there is a consistency between proprioceptive information and sensory feedback, and in particular, between the mental body model and the virtual body…Proprioception is “the continuous, but unconscious sensory flow from the movable parts of our body (muscles, tendons,joints) by which their position and tone and motion [are] continually monitored and  and adjusted, but in a way that ishidden from us because it is automatic and unconscious.” (Sacks 1985).  Proprioception allows us to form a mental model that describes the dynamic spatial and relational disposition of our body and its parts. We know where our left foot is (without having to look) by tapping into this body model. We can clap our two hands together (with closed eyes) similarly by relying on this unconscious mental model formed from the proprioceptive data flow.

The control groups (the “pointers”) navigated the environment using a 3D mouse, initiating movement by pressing a button, with direction of movement controlled by pointing. The experimental groups (the “walkers”) used the walking technique. In each case the mouse was also used for grasping objects. The task was to pick up an object, take it into a room, and place it on a particular chair. The chair was placed in such a way that the subjects had to cross a chasm over another room about 20 feet below in order to reach it…With respect to the ease of navigating the environment, subjects in both experiments marginally preferred to use the pointing technique. This result was not surprising: as Brooks et al. [ 1992] noted, with the real treadmill more energy certainly is required to use the whole body in a walking activity, compared to pressing a mouse button or making a hand gesture (or driving a car, with respect to the similar comparison in everyday reality).

This is quite interesting…People found that the mouse was EASIER, but walking was more natural.  More evidence that the best experiences might not be the easiest ones.

Other results showed that “for the “walkers” the greater their association with the virtual body the higher the presence score, whereas for the “pointers” there was no correlation between virtual body association and the presence score. In other words, participants who identified strongly with the virtual body had a greater degree of reported presence if they were in the “walking” group than if they were in the “pointing” group. Association with the virtual body is important.…We argue that the walking technique [helps people match their proprioception to their sensory information,] compared to the pointing technique,and therefore other things being equal should result in a greater sense of presence. However, we found that this is modified by the degree of association of the individual with the virtual body…The virtual body association is significantly positively correlated with a subjective presence for the walkers but not for the pointers, which is certainly consistent with the proposed model.

This is important because it means that Samsung Gear VR experiences where a person is represented as a black hole and has no virtual body, presence is going to be very difficult to create.

Beyond Walking: The Influence of Bending, Standing, and Task Complexity on Presence

The researchers of the walkers vs. pointers study conducted a follow up experiment where they asked people to “walk” in place through a forest.  They varied the height of the trees for some participants, meaning that when there was high (vs. low) variability, people inside of the experiment had to bend down and look up more.  

The results showed a significant positive association between reported presence and the amount of body movement, in particular head yaw, and the extent to which subjects bent down and stood up…The practical importance of the results of this experiment is that since there does seem to be a relationship between body movement and presence, it is a reasonable goal to design interactive paradigms that are based on semantically appropriate whole body gestures. These will not only seem more ‘natural’, but may also increase presence. We further believe that the increase in presence in itself will engender more body movement, which in turn will generate higher presence, and so on.

Interestingly, adding a layer of cognitive effort did not increase user feelings presence.  They manipulated task complexity by asking some participants to count the number of trees that they saw and remember the distribution of diseased trees.  However, there was no increase in presence by having to exert mental effort.  

Head Yaw is Good for Presence

Head Yaw is Good for Presence

Using Walking in Place to Make Stairs and Ladders

If you are doing a Harry Potter wizarding experience, then flying or teleporting might be the best locomotion.  However, if you are trying to do an education simulation, such as training fire fighters, consider integrating more humdrum actions.  

The same idea [of walking-in-place] can be applied to the problem of navigating steps and ladders. One alternative is to use the familiar pointing technique and to “fly.” While in some applications there maybe a place for such magical activity, the very fact that mundane objects such as steps and ladders are in the environment would indicate that a more-mundane method of locomotion be employed. The walking-in-place technique carries over in a straightforward manner to this problem. 

When the collision detection process in the virtual reality system detects a collision with the bottom step of a staircase, continued walking will move the participant up the steps. Walking down the steps is achieved by turning around and continuing to walk. If at any moment the participant’s virtual legs move off the steps (should this be possible in the application), then they would “fall” to the ground immediately below. Since walking backward down steps is something usually avoided, we do not provide any special means for doing this. However, it would be easy to support backward walking and walking backward down steps by taking into account the position of the hand in relation to body line: a hand behind the body would result in backward walking.

Ladders are slightly different; once the person has ascended part of the ladder, they might decide to descend at any moment. In the case of steps, the participant would naturally turn around to descend. Obviously this does not make sense for ladders. Also, when climbing ladders it is usual for the hands to be used. Therefore, in order to indicate ascent or descent of the ladder, hand position is taken into account. While carrying out the walking-in-place behavior on a ladder, if the hand is above the head then the participant will ascend the ladder and descend when below the head. Once again it is a whole-body gesture, rather than simply the use of the hand, that is required in order to achieve the required result in an intuitive manner. If at any time the virtual legs come off the rungs of the ladder, then the climber will “fall” to the ground below.

Key Takeaways to Maximize Presence in VR

  • Presence is defined as the user’s sense of “being there” inside of a simulated environment.  
  • The way that you believe you can interact with your environment is just as important as what you see in VR.
  • Walking-in-place has been proven to be a metaphor for locomotion and navigation that increases presence.
  • There is evidence that using body movements such as walking, bending down, and moving your head also heightens a sense of presence.
  • Cognitive complexity does not increase a sense of presence.  

Further reading

Sacks, Oliver. (1985). The Man Who Mistook His Wife for a Hat. Picador, London.

Slater, M., McCarthy, J., & Maringelli, F. (1998). The influence of body movement on subjective presence in virtual environments. Human Factors: The Journal of the Human Factors and Ergonomics Society, 40(3), 469-477.

Slater, M., Usoh, M., & Steed, A. (1995). Taking steps: the influence of a walking technique on presence in virtual reality. ACM Transactions on Computer-Human Interaction (TOCHI), 2(3), 201-219.