In writing my post about how taking physical action increases presence in VR, I got curious - how is presence being measured? That turned out to be such a big question that it needed a separate post. Here’s a round up of definitions and ways to measure presence. I’ve tried to organize the info in a way that helps people who are actively doing user experience research in VR.
In short, presence is the notion of “being there,” inside of a digital experience. Here’s a longer definition:
1. High resolution information displayed to the participant, in a manner that does not indicate the existence of the display devices.
2. Consistency of the displayed environment across all sensory modalities;
3. The possibility of the individual being able to navigate through and interact with objects in the environment, including interaction with other actors which may spontaneously react to the individual;
4. The individual’s virtual body, their self-representation within the environment, should be similar in appearance or functionality to the individual’s own body, and respond appropriately to the movements of their head, eyes, and limbs;
5. The connection between individual’s actions and effects of those actions should be simple enough for the individual to quickly learn.
Each of these is naturally maximized in the context of a person acting in everyday life (Usoh, Catena, Arman, and Slater 2000).
The concepts of presence, immersion, and performance have been studied in VR for more than 20 years so there are many different sources to draw on. I’m going to discuss how physical, psychological, and behavioral data on presence is collected.
Physical Evidence of Presence
Physical signs that can be objectively measured (rather than reported by the participant) are attractive to researchers. Heart rate is one such metric. A lesser know measure is the Galvanic skin response (GSR). When a human experiences psychological arousal, a broad term meaning they are alert, awake, and attentive, then their skin becomes temporarily better at conducting electricity. The implication for VR is that if people are going through an experience and their skin conductance increases, then you would assume that participants have a stronger sense of “being there” inside of an experience. More details on GSR from the MIT Media Lab:
The skin conductance response is measured from the eccrine glands, which cover most of the body and are especially dense in the palms and soles of the feet. (These are different from the apocrine sweat glands found primarily in the armpits and genital areas.) The primary function of eccrine glands is thermoregulation -- evaporative cooling of the body -- which tends to increase in aerobic activity, so yes, activity can affect conductance. However, the eccrine glands located on the palms and soles have been found to be highly sensitive to emotional and other significant stimuli, with a measurable response that precedes the appearance of sweat…Arousal has been found to be a strong predictor of attention and memory.
In the case of the heart rate or the GSR, they are useful for signaling to designers that something is happening inside of the experience. However, more steps are necessary to determine exactly what is the user’s experience. Measuring heart rate or arousal is different from measuring what people are feeling. To learn that, you will have to ask participants or analyze their behavior.
Assessing Psychological Presence
There are loads of existing survey questions that you could co-opt for your own user research. The Slater, Usoh and Stead (1995) and Witmer and Singer (1998) questionnaires are both very highly cited. Consider what the aims of your study are when deciding what type of survey questions to use with your participants.
Witmer and Singer (1998) has 32-questions and covers the domains of Control, Sensory Input, Distraction, and Realism. Example questions:
A. How much were you able to control events?
B. How compelling was your sense of objects moving through space?
C. How aware were you of your display and control devices?
Questions from Slater, Usoh, and Stead (1995) are below:
1. Please rate your sense of being in the virtual environment, on a scale of 1 to 7, where 7 represents your normal experience of being in a place.
2. To what extent were there times during the experience when the virtual environment was the reality for you?
3. When you think back to the experience, do you think of the virtual environment more as images that you saw or more as somewhere that you visited?
4. During the time of the experience, which was the strongest on the whole, your sense of being in the virtual environment or of being elsewhere?
5. Consider your memory of being in the virtual environment. How similar in terms of the structure of the memory is this to the structure of the memory of other places you have been today? By ‘structure of the memory’ consider things like the extent to which you have a visual memory of the virtual environment, whether that memory is in color, the extent to which the memory seems vivid or realistic, its size, location in your imagination, the extent to which it is panoramic in your imagination, and other such structural elements.
6. During the time of your experience, did you often think to yourself that you were actually in the virtual environment?
When conducting user research, it's ideal to film the participant so you have the opportunity to view the footage afterward. You might not know exactly what behavior you are looking for until you’re already on the last participant of the day. It most cases it will work to take an inductive approach and learn what behaviors are important by observing people going through your experience.
Examples of behaviors to track would be how people are interacting with menus. Do they have to make multiple attempts to accomplish a task? Are they taking time to read your instructions or do they just dive right in?
- Don't try to reinvent the wheel when it comes to measuring presence - there's plenty of valid and reliable research to draw on.
- Combine various sources of data in order to build the most robust results.
- Physical data typically only tells you that something is happening (increased heart rate or skin conductance), but not what the user's actual experience is.
- Surveys are used to gather psychological measures. An advantage of them is that data collected can easily be compared over time and across experiences / platforms / etc.
- Behavioral data can be tricky to collect because you may not know what you’re looking for, but it’s essential to telling the complete story of the user experience.
Slater, M., McCarthy, J., & Maringelli, F. (1998). The influence of body movement on subjective presence in virtual environments. Human Factors: The Journal of the Human Factors and Ergonomics Society, 40(3), 469-477.
Usoh, M., Catena, E., Arman, S., & Slater, M. (2000). Using presence questionnaires in reality. Presence: Teleoperators and Virtual Environments, 9(5), 497-503.