memory

How Google Earth VR Ruins Childhood Memories

On a recent “Voices of VR” podcast, I heard host Kent Bye talking about using Google Earth VR to reminisce and share stories with friends. Now, I had a very happy childhood, but my first reaction was, “I would never reminisce or share stories about my past in Google Earth VR. That’s just going to corrupt those memories.”

Memory is one the pillars of human cognition, in addition to attention, problem-solving, evaluation, and decision-making.  It is also extremely fragile and easy to influence.  If our experiences and memories are what make each of us unique, then it is sobering to realize how unreliable human memory can be. 

Your brain is not a computer. 

Human memory does not behave like a computer.  Memories are not files on a hard drive just waiting to be loaded. They are not fossils to be re-visited in a perfectly maintained museum.  Just by reminiscing about an event in your life, your memory of that event is impacted. 

Recalling memories changes them.

The act of revisiting our memories alters them. Here’s a good recap from Jeremy Dean at PsyBlog: 

"How can recalling a memory change it? Well, just by recalling a memory, it becomes stronger in comparison to other memories. Let’s run this through an example. Say you think back to one particular birthday from childhood and you recall getting a Lego spaceship. Each time you recall that fact, the other things you got for your birthday that day become weaker in comparison.

The process of recall, then, is actually actively constructing the past, or at least the parts of your past that you can remember.

This is only the beginning though. False memories can potentially be created by this process of falsely recalling the past. Indeed, psychologists have experimentally implanted false memories.  This raises the fascinating idea that effectively we create ourselves by choosing which memories to recall."


And Dean writes of another illustration of how easy it is to manipulate human memory:

"A neat experiment by Goff and Roediger (1998) demonstrates how easily our memory can transform fantasy into reality. Participants were asked either to imagine performing an action or actually asked to perform it, e.g. breaking a toothpick. Sometime later they went through the same process again. Then, later still they were asked whether they had performed that action or just imagined it. Those who imagined the actions more frequently the second time were more likely to think they’d actually performed the actions the first time."


You might think - I can remember details about my childhood much better than if a researcher in a white coat asked me to break a toothpick. Possibly, though we tend to have the best memory for the most recent events. Think about the implications of this false memory study for VR. People could easily believe that what they experienced virtually happened in real life.  It makes me wonder if creating realistic presence in VR should actually be a goal, given how it could affect people’s perceptions of reality.

If you are interested in memory, it has been studied extensively by researchers in criminal justice. Cognitive psychologist Elizabeth Loftus has done tremendous work in false memory and the inaccuracies of eyewitness testimony. In short, eyewitnesses are not reliable due to poor viewing conditions, brief exposure, being under stress, having pre-conceived expectations, biases, stereotypes, and more.  
 

Takeaways for VR Designers:

  • The human brain is not a computer.
  • The act of recalling memories changes them. 
  • Maintain skepticism of the memories that people recount to you (and the memories that you personally retrieve).  We are actively constructing our own experiences everyday.

 

 

 

Why is novelty important in VR/AR?

When you are in a novel situation, your brain processes information differently than when you are in your humdrum routine.  For example, can you remember your commute home last night?  Or what you ate for lunch last Wednesday?  In a normal routine, attention and memory don't encode as well compared to when you are doing something new and novel.  This is why some neuroscientists insist on driving home via a new route every night.  

Being able to change attention and memory processing has important consequences when you are making a VR/AR experience.  What things do you want your user to pay attention to?  What do you want them to remember after the experience?  Here's a model of cognitive processing that you can use to consider how new information is going to be presented to users called "Top-Down Bottom-Up."

Excerpted from: Mind: Journey to the Heart of Being Human’ by Daniel J Siegel:

In the view we will be using here, top-down refers to ways we have experienced things in the past and created generalised summaries or mental models, also known as schema, of those events. For example, if you’ve seen many dogs, you’ll have a general mental model or image of a generic dog. The next time you see a furry canine strolling by, your top-down processing might use that mental model to filter incoming visual input, and you won’t really see the uniqueness of this dog in front of you. You have overlaid your generalised image of ‘dog’ on top of the here-and-now perceptual stream of energy that creates the neural representation of ‘dog’. What you actually have in awareness is that amalgam of the top-down filtering of your experience.

So here, ‘top’ means that prior experience is activated, making it difficult to notice the unique and vibrant details of what is happening here and now. The top-down generalised notion of dog will shade and limit your perception of the actual animal in front of you. The benefit of top-down is that it makes your life more efficient. That’s a dog, I know what it is, I don’t need to expend any more energy than needed on insignificant, non-threatening things, so I’ll take my limited resources and apply them elsewhere. It saves time and energy, and therefore is cognitively efficient. That’s top-down processing.

On the other hand, if you’ve never seen a spiny anteater before, the first time you come across one on the trail, it will capture all of your attention, engaging your bottom-up processing so that you are seeing with beginner’s eyes. These are eyes leading to circuitry in the brain, not shaping and altering ongoing perception through the top-down filters of prior experience. You’ll be taking in as much pure sensation from eyesight as possible, without the top-down filter altering and limiting what you see now based on what you’ve seen before.

These screenshots are from UpLoadVR's playthrough of the game "Accounting" by Squanchtendo.  The game uses different environments, such as a windowless accounting office (image left) to contrast with fantasy worlds where there are talking trees (image right) and other bizarre characters.  When you are in the fantasy world of "Accounting," you are likely using "bottom up" cognitive processing.  People feel excited when they are trying something new.  They take information from the exterior world, move it up a level, add language, move it up, analyze, and the user experience shifts.  Novel experiences tend to be more pleasurable because they rely on sensation and not cognition. 

The most vivid part of the mind bubbles up through sensation and new experience when unencumbered by analytical thought.
— Daniel J. Siegel

More on sensation from Siegel:

Sensation might be as bottom-up as we get. Since we live in a body, our within-mind experience is shaped by the physical apparatus that lets us take in energy flow from the outside world. We have our first five senses of sight, hearing, smell, taste and touch; we have our proprioceptive sense of motion; and we have our ‘interoceptive’ sense of the signals from the interior of the body. These perceptual capacities to sense the outer world and internal bodily world are built upon the physical neural machinery that enables energy to flow. Information is created with these energy patterns, generated as ions flow in and out of membranes and chemicals are released in the pathways of neural activity. As energy flows into the brain from our external sense organs, such as our eyes and ears, or from internal receptors of our body’s muscles, bones and internal organs, we move from sensation to perception – with pure sensation as close as we get to being fully present in the world.

Key Takeaways for VR/AR experience designers

  • Novelty increases awareness, which influences attention and memory
  • Top-down is the analytical system, which relies on prior experience
  • Bottom-up is the sensation-driven system, which looks with the eyes of a beginner 
  • Take a new route home on your commute tonight to experience novelty yourself.  Then look for ways to integrate that type of novelty into your experience.  

 

Further reading: 

Siegel, Daniel J. The Open Mind.  Aeon Magazine.  October 25, 2016

UploadVR full play through of "Accounting"

Stop Forcing Users to Text and Drive

Here's a good read with practical tips for VR designers. Adrienne Hunter of Tomorrow Today Labs just posted "Reducing Cognitive Load in VR: 6 Ways to Improve your VR UX."  

Cognitive load has to do with how much information you are trying to hold inside of your working memory at any given time.  Information is filtered in working memory as a transient step before being encoded in short-term or long-term memory (or forgotten).  

We're really just talking about attention.  People have a finite capacity to pay attention, which is why it's difficult to manage two complicated tasks at once, such as texting and driving.  The more attention that you are paying, the more that your working memory is being used.  That makes it harder to learn or to solve puzzles.  But, it also has substantial effects on decision-making.  Here's a few ways that people made decisions differently when they were under high cognitive load and had to divide their attention:

Conclusion

Only ask your users to do one thing at a time. Setting them up to do a sequence a tasks is preferred to having them do anything simultaneously.