Experience on Demand

by Jeremy Bailenson

Virtual reality is able to effectively blur the line between reality and illusion, pushing the limits of our imagination and granting us access to any experience possible. These experiences, ones that the brain is convinced are real, will soon be available at the click of a button. In Experience on Demand, Jeremy Bailenson draws on two decades spent researching the psychological effects of virtual reality to help readers understand this new medium. He offers expert guidelines for interacting with VR and describes the profound ways this technology can be used to hone our performance, help us recover from trauma, improve our learning and communication abilities, and even enhance our empathic and imaginative capacities so that we treat others, the environment and ourselves better.

Mark Zuckerberg is about to walk the plank. It’s March 2014, and we’re standing in the multisensory room of the Virtual Human Interactive Lab (VHIL) at Stanford University. I’m making last-minute adjustments to his head mounted display (HMD), the bulky, expensive, helmet-like device that is about to take him into another world.

Zuckerberg, at the moment plunged into darkness, is asking me questions about the technical specs of the VR hardware in my lab—what’s the resolution of the displays in front of his eyes? How quickly do the images on those displays update? Unsurprisingly, he is a curious and knowledgeable subject, and he’s clearly done his homework. He’s come today because he wants to experience state-of-the-art virtual reality, and I’m eager to talk to him because I have opinions on the ways in which virtual reality can be used on a social networking site like Facebook.

At Stanford, we are encouraged to face outward and to share our work, not just with academics but with decision makers of all types. I often do this kind of outreach, sharing my lab’s unique capabilities with business executives, foreign dignitaries, journalists, celebrities, and others who are curious about the experience of virtual reality. On this day, I am eager to show Zuckerberg—someone who has demonstrated great philanthropic investment in areas of education, environment, and empathy research—the ways in which our work on VR has direct applications for those same issues. But first I have to show him what my lab can do. I usually start with “the plank”—it’s one of the most effective ways to evoke the powerful sensation of presence that good VR produces. And our VR is very good—one of the best in the world. The floor shakes, we have “haptic” devices that give feedback to the hands, 24 speakers that deliver spatialized sound, and a high-resolution headset with LEDs on the side that allow cameras mounted around the room to track head and body movement as the user walks around the room. All of this information is assimilated to render interactive, digitally created places that allow the user to experience almost anything we can think of—flying through a city, swimming with sharks, inhabiting nonhuman bodies, or standing on the surface of Mars. Anything we decide to program can be rendered in a virtual environment. 

The display comes on and Zuckerberg sees the multisensory room again, except that I and my lab assistants have disappeared. The room he is looking at is discernibly lower-res—a bit like television used to look before High Definition—but the carpeted floor, the doors, the wall in front of him are all there, creating an effective simulacrum of the space he was just standing in. Zuckerberg moves his head around to take it all in, everything smoothly scrolling into his vision as it would in real life. He steps forward and backward, and the illusion projected a few inches in front of his eyes corresponds with the movement of his body. “Trippy,” he says. I lead him to a spot on the floor (I will be constantly at his side “spotting him” during this demonstration, as it’s very easy to bump into real-world things when you’re navigating a virtual space) and instruct my assistant in the control room to start the program. “Let’s do the pit.”

Zuckerberg hears an industrial whine, the floor shudders, and the small virtual platform on which he stands shoots away from the ground. From his perspective, which I can see via a projection screen on the wall, he’s now standing on a small shelf about 30 feet in the air, connected by a narrow plank to another platform about 15 feet away. Zuckerberg’s legs buckle a bit and his hand involuntarily goes to his heart. “OK, that’s pretty scary.” If we were measuring his stress signs we’d see that his heart rate was speeding up and his hands were beginning to sweat. He knows he is standing on the floor of a campus lab, but his dominant sense is telling him that he’s precariously balanced at a deadly distance above the ground. He’s getting a taste of “presence,” that peculiar sense of “being there” unique to virtual reality.

Over the nearly two decades that I’ve been doing VR experiments and demonstrations, I’ve witnessed this scene—when a person is first enveloped by a virtual environment—thousands of times, and I’ve seen a lot of reactions. Some gasp. Some laugh with delight. Depending on what’s being rendered in the program, I’ve also seen people cry out in fear, or throw their hands up to protect themselves as they hurtle toward a wall. An elderly federal judge once dove horizontally into a real table in order to catch an imaginary ledge after he “fell” off the virtual platform. At a demonstration at the Tribeca Film Festival, the rapper Q-Tip crawled across the plank on his hands and knees. Often my subjects just stand slack-jawed with wonder, gazing down, up, and around, amazed to see themselves suddenly surrounded by a digitally rendered world that nevertheless feels, in crucial ways, real.

It’s a strange feeling, standing above the pit. No matter how prepared you think you are going to be the first time, it still takes you by surprise. Consider first that you, a subject immersed in the pit program, know you are participating in a VR demo—it’s not like you accidentally wound up in a VR rig. So you’re anticipating it. In fact, you’ve probably already witnessed—and perhaps even been amused by—another person going through the pit demo before you. You’ve looked at the projection of what they’re seeing as they stand over the ledge. You’ve watched their legs bend as they lower their center of gravity, and seen their arms shoot out to their side for better balance. Watched as they tentatively shuffle across a thin beam that exists only in the program’s code, and in the part of their brain that is interpreting the illusion projected into their eyes. Watched a person standing in the middle of the lab room, wearing an awkward helmet tethered to the ceiling by cables, ever so tentatively lean forward to gaze at a drop-off that doesn’t exist. 

In truth, it’s kind of funny. And yet still, there you are a few moments later wearing the HMD yourself, your feet planted firmly on the same, solid floor you traversed just moments before, now suddenly facing a deadly drop with only a narrow plank to walk across. And it’s not so funny anymore. If you’re like one-third of the subjects in my lab, when I ask you to take a step off the platform into the virtual abyss, you’ll refuse, standing rooted to a spot on the floor.

Zuckerberg does walk the plank, although it’s not easy going. After he makes it to the other platform, I run a program in which his avatar grows a third arm and he has to learn how to move it by twisting his real limbs. Then he flies through the air like Superman. We put him in the body of a senior citizen and point him to a virtual mirror so he can see this strange doppelganger reflect the gestures he is making in the lab room. I load up another program, and he inhabits the body of a shark and swims around a coral reef. “It’s not bad, being a shark,” he says. After a few more minutes he’s had enough—the experience of VR can be overwhelming and even the best equipment today can cause eyestrain and head discomfort after twenty minutes or so. 

For the rest of his two-hour visit to the lab we discuss my research into the psychology of virtual reality, and how it has convinced me that there are many ways the unique power of VR can be applied to make us better people, more empathetic, more aware of the fragility of the environment, and more productive at work. We talk about how VR is going to improve the quality and reach of education, and open up the world for people who can’t afford to travel, transporting users to the tops of mountains, or into Earth’s orbit, or into a calming oceanside setting at the end of a long day. How it’s going to allow us to share experiences like these with our friends or family, even if they live far away. 

Find out more

Excerpted from Experience on Demand by Jeremy Bailenson. Copyright © 2018 by Jeremy Bailenson. Used with permission of the publisher, W.W. Norton & Company, Inc. All rights reserved.

Jeremy Bailsenson is professor of communication at Stanford University and founding director of the Virtual Human Interaction Lab. His writing has appeared in the Washington Post, Slate, and the San Francisco Chronicle. He lives in Redwood City, California.