How to pronounce "pea"

When I was a kid,

I experienced something so powerful,

I spent the rest of my life searching for it,

and in all the wrong places.

What I experienced wasn't virtual reality.

It was music.

And this is where the story begins.

That's me,

listening to the Beatles' "White Album."

And the look on my face is the feeling

that I've been searching for ever since.

Music goes straight to the emotional vein,

into your bloodstream

and right into your heart.

It deepens every experience.

Fellas?

(Music)

This is the amazing McKenzie Stubbert

and Joshua Roman.

Music --

(Applause)

Yeah.

Music makes everything have more emotional resonance.

Let's see how it does for this talk.

The right piece of music at the right time fuses with us

on a cellular level.

When I hear that one song

from that one summer

with that one girl,

I'm instantly transported back there again.

Hey, Stacey.

Here's a part of the story, though, where I got a little greedy.

I thought if I added more layers on top of the music,

I could make the feelings even more powerful.

So I got into directing music videos.

This is what they looked like.

That's my brother, Jeff.

Sorry about this, Jeff.

(Laughter)

Here's me, just so we're even.

Incredible moves.

Should've been a dancer.

(Laughter)

These experiments grew,

and in time, started to look more like this.

In both, I'm searching for the same thing, though,

to capture that lightning in a bottle.

Except, I'm not.

Adding moving pictures over the music added narrative dimension, yes,

but never quite equated the power

that just raw music had for me on its own.

This is not a great thing to realize when you've devoted your life

and professional career to becoming a music video director.

I kept asking myself, did I take the wrong path?

So I started thinking: if I could involve you, the audience, more,

I might be able to make you feel something more as well.

So Aaron Koblin and I began auditioning new technologies

that could put more of you inside of the work,

like your childhood home in "The Wilderness Downtown,"

your hand-drawn portraits, in "The Johnny Cash Project,"

and your interactive dreams

in "3 Dreams of Black."

We were pushing beyond the screen,

trying to connect more deeply

to people's hearts and imaginations.

But it wasn't quite enough.

It still didn't have the raw experiential power of pure music for me.

So I started chasing a new technology

that I only had read about in science fiction.

And after years of searching, I found a prototype.

It was a project from Nonny de la Peña in Mark Bolas's lab in USC.

And when I tried it, I knew I'd found it.

I could taste the lightning.

It was called virtual reality.

This was it five years ago when I ran into it.

This is what it looks like now.

I quickly started building things in this new medium,

and through that process we realized something:

that VR is going to play an incredibly important role

in the history of mediums.

In fact, it's going to be the last one.

I mean this because it's the first medium that actually makes the jump

from our internalization of an author's expression

of an experience,

to our experiencing it firsthand.

You look confused. I'll explain. Don't worry.

(Laughter)

If we go back to the origins of mediums,

by all best guesses,

it starts around a fire, with a good story.

Our clan leader is telling us

about how he hunted the woolly mammoth on the tundra that day.

We hear his words

and translate them into our own internal truths.

The same thing happens

when we look at the cave painting version of the story,

the book about the mammoth hunt,

the play,

the radio broadcast,

the television show

or the movie.

All of these mediums require what we call "suspension of disbelief,"

because there's a translation gap between the reality of the story

and our consciousness interpreting the story

into our reality.

I'm using the word "consciousness" as a feeling of reality that we get

from our senses experiencing the world around us.

Virtual reality bridges that gap.

Now, you are on the tundra hunting with the clan leader.

Or you are the clan leader.

Or maybe you're even the woolly mammoth.

(Laughter)

So here's what special about VR.

In all other mediums,

your consciousness interprets the medium.

In VR, your consciousness is the medium.

So the potential for VR is enormous.

But where are we now?

What is the current state of the art?

Well,

we are here.

We are the equivalent of year one of cinema.

This is the Lumière Brothers film

that allegedly sent a theater full of people running for their lives

as they thought a train was coming toward them.

Similar to this early stage of this medium,

in VR, we also have to move past the spectacle

and into the storytelling.

It took this medium decades

to figure out its preferred language of storytelling,

in the form of a feature film.

In VR today, we're more learning grammar

than writing language.

We've made 15 films in the last year at our VR company, Vrse,

and we've learned a few things.

We found that we have a unique, direct path into your senses,

your emotions, even your body.

So let me show you some things.

For the purpose of this demo,

we're going to take every direction that you could possibly look,

and stretch it into this giant rectangle.

OK, here we go.

So, first: camera movement is tricky in VR.

Done wrong, it can actually make you sick.

We found if you move the camera at a constant speed in a straight line,

you can actually get away with it, though.

The first day in film school,

they told me you have to learn every single rule

before you can break one.

We have not learned every single rule.

We've barely learned any at all,

but we're already trying to break them

to see what kind of creative things we can accomplish.

In this shot here, where we're moving up off the ground, I added acceleration.

I did that because I wanted to give you a physical sensation

of moving up off the ground.

In VR, I can give that to you.

(Music)

Not surprisingly, music matters a lot in this medium as well.

It guides us how to feel.

In this project we made with the New York Times, Zach Richter

and our friend, JR,

we take you up in a helicopter,

and even though you're flying 2,000 feet above Manhattan,

you don't feel afraid.

You feel triumphant for JR's character.

The music guides you there.

(Music)

Contrary to popular belief,

there is composition in virtual reality,

but it's completely different than in film,

where you have a rectangular frame.

Composition is now where your consciousness exists

and how the world moves around you.

In this film, "Waves of Grace," which was a collaboration between Vrse,

the United Nations, Gabo Arora, and Imraan Ismail,

we also see the changing role of the close-up in virtual reality.

A close-up in VR means you're actually close up to someone.

It brings that character inside of your personal space,

a space that we'd usually reserve for the people that we love.

And you feel an emotional closeness to the character

because of what you feel to be a physical closeness.

Directing VR is not like directing for the rectangle.

It's more of a choreography of the viewer's attention.

One tool we can use to guide your attention

is called "spatialized sound."

I can put a sound anywhere in front of you, to left or right,

even behind you,

and when you turn your head, the sound will rotate accordingly.

So I can use that to direct your attention to where I want you to see.

Next time you hear someone singing over your shoulder,

it might be Bono.

(Laughter)

VR makes us feel like we are part of something.

For most of human history, we lived in small family units.

We started in caves,

then moved to clans and tribes, then villages and towns,

and now we're all global citizens.

But I believe that we are still hardwired to care the most

about the things that are local to us.

And VR makes anywhere and anyone feel local.

That's why it works as an empathy machine.

Our film "Clouds Over Sidra" takes you to a Syrian refugee camp,

and instead of watching a story about people over there,

it's now a story about us here.

But where do we go from here?

The tricky thing is that with all previous mediums,

the format is fixed at its birth.

Film has been a sequence of rectangles,

from Muybridge and his horses to now.

The format has never changed.

But VR as a format, as a medium,

isn't complete yet.

It's not using physical celluloid or paper or TV signals.

It actually employs what we use to make sense of the world.

We're using your senses as the paints on the canvas,

but only two right now.

Eventually, we can see if we will have all of our human senses employed,

and we will have agency to live the story in any path we choose.

And we call it virtual reality right now,

but what happens when we move past simulated realities?

What do we call it then?

What if instead of verbally telling you about a dream,

I could let you live inside that dream?

What if instead of just experiencing visiting some reality on Earth,

you could surf gravitational waves on the edge of a black hole,

or create galaxies from scratch,

or communicate with each other not using words

but using our raw thoughts?

That's not a virtual reality anymore.

And honestly I don't know what that's called.

But I hope you see where we're going.

But here I am, intellectualizing a medium I'm saying is experiential.

So let's experience it.

In your hands, you hopefully hold a piece of cardboard.

Let's open the flap.

Tap on the power button to unlock the phone.

For the people watching at home,

we're going to put up a card right now

to show you how to download this experience on your phone yourself,

and even get a Google cardboard of your own to try it with.

We played in cardboard boxes as kids,

and as adults, I'm hoping we can all find a little bit of that lightning

by sticking our head in one again.

You're about to participate

in the largest collective VR viewing in history.

And in that classic old-timey style of yesteryear,

we're all going to watch something

at the exact same time, together.

Let's hope it works.

What's the countdown look like? I can't see.

Audience: ...15, 14, 13, 12, 11, 10, 9,

8, 7, 6, 5, 4, 3, 2, 1

(Birds singing)

(Train engine)

Audience: (Shreiks)

(Video) JR: Let me tell you

how I shot the cover of the New York Times Magazine,

"Walking New York."

I just got strapped on outside the helicopter,

and I had to be perfectly vertical so I could grab it.

And when I was perfectly above --

you know, with the wind, we had to redo it a few times --

then I kept shooting.

(Video) Woman's voice: Dear Lord,

protect us from evil,

for you are the Lord,

the light.

You who gave us life took it away.

Let your will be done.

Please bring peace to the many who have lost loved ones.

Help us to live again.

(Music)

(Video) (Children's voices)

Child's voice: There are more kids in Zaatari than adults right now.

Sometimes I think

we are the ones in charge.

Chris Milk: How was it?

(Applause)

That was a cheap way of getting you to do a standing ovation.

I just made you all stand. I knew you'd applaud at the end.

(Applause)

I believe that everyone on Earth needs to experience

what you just experienced.

That way we can collectively start to shape this,

not as a tech platform

but as a humanity platform.

And to that end, in November of last year,

the New York Times and Vrse made a VR project called "The Displaced."

It launched with one million Google Cardboards

sent out to every Sunday subscriber with their newspaper.

But a funny thing happened that Sunday morning.

A lot of people got them

that were not the intended recipients on the mailing label.

And we started seeing this all over Instagram.

Look familiar?

Music led me on a path

of searching for what seemed like the unattainable

for a very long time.

Now, millions of kids just had the same formative experience

in their childhood

that I had in mine.

Only I think this one

surpasses it.

Let's see

where this

leads them.

Thank you.

(Applause)