How to pronounce "ems"

Someday, we may have robots as smart as people,

artificial intelligence, AI.

How could that happen?

One route is that we'll just keep accumulating better software,

like we've been doing for 70 years.

At past rates of progress, that may take centuries.

Some say it'll happen a lot faster

as we discover grand new powerful theories of intelligence.

I'm skeptical.

But a third scenario

is what I'm going to talk about today.

The idea is to port the software from the human brain.

To do this, we're going to need three technologies to be good enough,

and none of them are there yet.

First, we're going to need lots of cheap, fast, parallel computers.

Second, we're going to need to scan individual human brains

in fine spatial and chemical detail,

to see exactly what cells are where, connected to what, of what type.

And third, we're going to need computer models

of how each kind of brain cell works --

taking input signals, changing interval state

and sending output signals.

If we have good enough models of all the kinds of brain cells

and a good enough model of the brain,

we can put it together to make a good enough model of an entire brain,

and that model would have the same input-output behavior as the original.

So if you talk to it, it might talk back.

If you ask it to do things, it might do them.

And if we could do that, everything would change.

People have been talking about this idea for decades,

under the name of "uploads."

I'm going to call them "ems."

When they talk about it, they say,

"Is this even possible?

If you made one, would it be conscious? Or is it just an empty machine?

If you made one of me, is that me or someone else?"

These are all fascinating questions that I'm going to ignore ...

(Laughter)

because I see a neglected question:

What would actually happen?

I became obsessed with this question.

I spent four years trying to analyze it,

using standard academic tools, to guess what would happen,

and I'm here to tell you what I found.

But be warned --

I'm not offering inspiration, I'm offering analysis.

I see my job as telling you what's most likely to happen

if we did the least to avoid it.

If you aren't at least a bit disturbed by something I tell you here,

you're just not paying attention.

(Laughter)

OK, the first thing I can tell you

is that ems spend most of their life in virtual reality.

This is what you might look like if you were using virtual reality.

And this is what you might see:

sunlight glinting off of water, you might hear gulls flying above,

you might even feel the wind on your cheeks or smell seawater,

with advanced hardware.

Now, if you were to spend a lot of time here,

you might want a dashboard

where you could do things like make a phone call,

move to a new virtual world,

check your bank account.

Now, while this is what you would look like in virtual reality,

this is what an em would look like in virtual reality.

It's computer hardware sitting in a server rack somewhere.

But still, it could see and experience the same thing.

But some things are different for ems.

First, while you'll probably always notice that virtual reality isn't entirely real,

to an em, it can feel as real to them as this room feels to you now

or as anything ever feels.

And ems also have some more action possibilities.

For example, your mind just always runs at the same speed,

but an em can add more or less computer hardware to run faster or slower,

and therefore, if the world around them seems to be going too fast,

they can just speed up their mind,

and the world around them would seem to slow down.

In addition, an em can make a copy of itself at that moment.

This copy would remember everything the same,

and if it starts out with the same speed, looking at the same speed,

it might even need to be told, "You are the copy."

And em could make archive copies,

and with enough archives,

an em can be immortal --

in principle, though not usually in practice.

And an em can move its brain, the computer that represents its brain,

from one physical location to another.

Ems can actually move around the world at the speed of light,

and by moving to a new location,

they can interact more quickly with ems near that new location.

So far, I've been talking about what ems can do.

What do ems choose to do?

To understand that, we'll need to understand three key facts.

First, ems by definition do what the human they emulate would do

in the same situation.

So their lives and behavior are very human.

They're mainly different because they're living in a different world.

Second, ems need real resources to survive.

You need food and shelter or you'll die.

Also, ems need computer hardware, energy, cooling, or they can't exist.

For every subjective minute that an em experiences,

someone, usually that em, had to work to pay for it.

Third, ems are poor.

(Laughter)

The em population can grow quicker than the em economy,

so that means wages fall down to em subsistence levels.

That means ems have to be working most of the time.

So that means this is what ems usually see:

beautiful and luxurious, but desks --

they're working most of the time.

Now, a subsistence wage scenario, you might think, is exotic and strange,

but it's actually the usual case in human history,

and it's how pretty much all wild animals have ever lived,

so we know what humans do in this situation.

Humans basically do what it takes to survive,

and this is what lets me say so much about the em world.

When creatures are rich, like you,

you have to know a lot about what they want

to figure out what they do.

When creatures are poor,

you know that they mostly do what it takes to survive.

So we've been talking about the em world from the point of view of the ems --

now, let's step back and look at their whole world.

First, the em world grows much faster than ours,

roughly a hundred times faster.

So the amount of change we would experience in a century or two,

they would experience in a year or two.

And I'm not really willing to project this age much beyond that,

because plausibly by then something else will happen, I don't know what.

Second, the typical emulation runs even faster,

roughly a thousand times human speed.

So for them, they experience thousands of years in this year or two,

and for them, the world around them is actually changing more slowly

than your world seems to change for you.

Third, ems are crammed together in a small number of very dense cities.

This is not only how they see themselves in virtual reality,

it's also how they actually are physically crammed together.

So at em speeds, physical travel feels really painfully slow,

so most em cities are self-sufficient,

most war is cyber war,

and most of the rest of the earth away from the em cities

is left to the humans, because the ems really aren't that interested in it.

Speaking of humans,

you were wanting to hear about that.

Humans must retire, at once, for good.

They just can't compete.

Now, humans start out owning all of the capital in this world.

The economy grows very fast, their wealth grows very fast.

Humans get rich, collectively.

As you may know, most humans today don't actually own that much

besides their ability to work,

so between now and then, they need to acquire sufficient assets,

insurance or sharing arrangements,

or they may starve.

I highly recommend avoiding this outcome.

(Laughter)

Now, you might wonder,

why would ems let humans exist?

Why not kill them, take their stuff?

But notice we have many unproductive retirees around us today,

and we don't kill them and take their stuff.

(Laughter)

In part, that's because it would disrupt the institutions we share with them.

Other groups would wonder who's next,

so plausibly, ems may well let humans retire in peace during the age of em.

You should worry more that the age of em only lasts a year or two

and you don't know what happens next.

Ems are very much like humans,

but they are not like the typical human.

The typical em is a copy of the few hundred most productive humans.

So in fact, they are as elite, compared to the typical human,

as the typical billionaire, Nobel Prize winner,

Olympic gold medalist, head of state.

Ems look on humans perhaps with nostalgia and gratitude,

but not so much respect,

which is, if you think about it, how you think about your ancestors.

We know many things about how humans differ in terms of productivity.

We can just use those to predict features of ems --

for example, they tend to be smart, conscientious, hard-working,

married, religious, middle-aged.

These are features of ems.

Em world also contains enormous variety.

Not only does it continue on with most of the kinds of variety that humans do,

including variety of industry and profession,

they also have many new kinds of variety,

and one of the most important is mind speed.

Ems can plausibly go from human speed

up to a million times faster than human speed,

and down to a billion times slower than human speed.

Faster ems tend to have markers of high status.

They embody more wealth. They win arguments.

They sit at premium locations.

Slower ems are mostly retirees,

and they are like the ghosts of our literature.

If you recall, ghosts are all around us --

you can interact with them if you pay the price.

But they don't know much, they can't influence much,

and they're obsessed with the past, so what's the point?

(Laughter)

Ems also have more variety in the structure of their lives.

This is your life: you start and you end, really simple.

This is the life of an em,

who every day splits off some short-term copies

to do short-term tasks and then end.

We'll talk more about those short term versions in a moment,

but they are much more efficient

because they don't have to rest for the next day.

This em is more opportunistic.

They make more copies of themselves when there's more demand for that.

They don't know which way the future's going.

This is an em designer,

who conceives of a large system

and then breaks recursively into copies who elaborate that,

so ems can implement larger, more coherent designs.

This an emulation plumber

who remembers that every day, for the last 20 years,

they only ever worked two hours a day, a life of leisure.

But what really happened is, every day they had a thousand copies,

each of whom did a two-hour plumbing job,

and only one of them went on to the next day.

Objectively, they're working well over 99 percent of the time.

Subjectively, they remember a life of leisure.

(Laughter)

This, again, is you. You start and you end.

This could be you if at the start of party,

you took a drug that meant you would not remember that party

ever after that day.

Some people do this, I'm told.

Toward the end of the party,

will you say to yourself, "I'm about to die, this is terrible.

That person tomorrow isn't me, because they won't remember what I do."

Or you could say, "I will go on tomorrow. I just won't remember what I did."

This is an em who splits off a short-term copy

to do a short-term task and then end.

They have the same two attitude possibilities.

They can say, "I'm a new short-term creature with a short life. I hate this."

Or "I'm a part of a larger creature who won't remember this part."

I predict they'll have that second attitude,

not because it's philosophically correct, but because it helps them get along.

Today, if the president says we must invade Iraq,

and you say, "Why?"

and they say, "State secret,"

you're not sure if you can trust them,

but for ems, a copy of the president and a copy of you can go inside a safe,

explain all their secret reasons,

and then one bit comes out from your copy to yourself,

telling you if you were convinced.

So now you can know there is a good reason.

I know you guys are all eager to evaluate this world.

You're eager to decide if you love it or hate it.

But think: your ancestors from thousands of years ago

would have loved or hated your world

based on the first few things they heard about it,

because your world is really just weird.

So before judging a strange future world, you should really learn a lot about it,

maybe read a whole book about it,

and then, if you don't like it, work to change it.

Thank you.

(Applause)