For our entire lives, we’ve been living
through a computing revolution.
Many of you here in this room
have contributed to it,
with leaps forward in compute,
connectivity, mobile technologies,
and now AI.
For my part, I've dedicated
my entire career to augmented reality,
fusing the real world
with computing experiences.
I started this journey
about 25 years ago for my PhD.
It might sound groundbreaking,
but check out these early prototypes.
The technology was really
primitive back then,
but the potential felt limitless
when we were experimenting
in that university lab.
What I didn't know at that time
was many of the fundamental
innovations for my work
would come from AI researchers
in a different lab.
While I was helping
computers see the world,
these AI researchers were helping
computers reason about the world.
Since then, large language models
and multimodal reasoning
have enabled richer
language and image understanding.
These models are now fast enough
for live conversations
where the AI can act on your behalf
and, most importantly,
with your permission.
And augmented reality and virtual
reality has moved computing
from the rectangular screen
to the 360 immersive display
to now even the world itself
becoming the display.
We now refer to this broad collection
of experiences as extended reality or XR.
Until now,
these innovations have occurred
separately and in silos.
Well here comes act two
of the computing revolution.
AI and XR are converging,
unlocking radical new ways
to interact with technology on your terms.
Computers will become
more lightweight and personal.
They will share your vantage point,
understand your real-world context
and have a natural interface
that's both simple and conversational.
Countless people and companies
are innovating in this space,
including many on the TED stage this week.
We're excited to contribute
to this momentum with Android XR.
It's an operating system
we're building with Samsung
that brings XR hardware together
with Gemini, our AI assistant,
to augment human intelligence.
It can support a broad range of devices,
from headsets to glasses
to form factors we haven't
even dreamed of yet.
OK, let me show you where we’re heading
by inviting my colleague
Nishtha to the stage.
Welcome, Nishtha.
(Applause)
Nishtha Bhatia: Hi.
Shahram Izadi: These ordinary-looking
glasses are packed full of technology.
A miniaturized camera and microphones
give the AI the ability
to see and hear the world.
Speakers let you listen to the AI
and play music or even take calls.
And these glasses also have a tiny,
high-resolution in-lens display
that's full color
that I'm holding in my hand.
It's very, very small.
These glasses work with your phone
streaming back and forth,
allowing the glasses
to be very lightweight
and access all your phone apps.
And if you're wondering,
I'm actually wearing the glasses too.
They're actually helping me see
all of you in the audience
because they have
prescription lenses inside them.
But they’re also displaying
my speaker notes for me as well.
(Laughter)
For this demo,
you'll see what Nishtha is seeing
on the screen behind her.
And this is the very first time
we're showing these glasses in public.
So this is going to be a live demo
of conceptual hardware, early software.
What could go wrong?
Nishtha, best of luck.
No pressure.
NB: Amazing.
Alright, let’s just make sure
that these glasses are shown
on the screen behind us.
OK.
[Glasses screen off]
Awesome.
(Applause)
Now I'm going to launch Gemini.
Gemini: Hello there.
NB: Hi, Gemini.
Can you start us off
with something fun and creative,
and write a haiku for what
I'm seeing right now?
G: Sure.
Faces all aglow.
Eager minds await the words.
Sparks of thought ignite.
SI: Some anxious
but happy faces as well, yeah.
As you can see, the AI sees
what Nishtha sees,
hears what she hears
and is responding in real time.
But that’s just table stakes.
I think everyone in this audience
has seen demos like this before.
Let's actually step it up a notch
with something we call Memory.
For a rolling contextual window,
the AI remembers what you see
without having to be told
what to keep track of.
NB: So you guys may have noticed
I snuck a peek back
at the shelf a moment ago.
I wasn't paying attention,
but let's see if Gemini was.
Hey, did you happen to catch
the title of the white book
that was on the shelf behind me?
G: The white book
is “Atomic Habits” by James Clear.
NB: That is absolutely right.
So let's try something harder.
I keep losing my hotel key card.
Do you know where I last left the card?
G: The hotel key card
is to the right of the music record.
NB: Great.
SI: For someone as forgetful as me,
that's a killer app.
OK, this AI is inherently multimodal.
So it's using visual data
and natural language
to digest complex queries in real time
while remembering your past context.
Nishtha, let's see some more.
NB: Yeah, let’s bring this book back.
So I don't really have time
to read it right now.
But I wonder if Gemini might be able
to help me summarize what's going on.
Like, I don't know.
What does this diagram mean?
It looks complex.
G: This diagram
is labeled “The Habit Line”
and illustrates the process
of habit formation.
It shows how automaticity
increases with repetitions
until a habit is formed.
NB: OK.
SI: Now imagine being able to communicate
and understand any language.
The AI can shift its output
to speak different languages
with dialects and accents
that feel human and natural.
NB: Yeah, let’s go
to this sign over here.
Can you translate this to English for me?
G: The sign states:
“Private property, no trespassing.”
SI: OK, let's prove this is a live demo.
Does anyone in the audience
have a suggestion
for a different language to translate to?
Audience: Farsi.
SI: Farsi.
We tried Farsi, it failed last time,
but let's try it again.
NB: Do you want to try Farsi?
SI: Because I do speak Farsi.
It's my mother tongue, so thank you.
NB: Gemini,
let’s just give this a shot.
Can you translate this sign
to Farsi for us?
G: (Speaking Farsi)
SI: Great, awesome.
It speaks Farsi.
NB: That’s amazing.
So, as Shahram mentioned,
you all may have seen
translation demos like this before,
but what's new now is that in addition
to just saying things
in a different language,
I can also speak to Gemini
in another language.
I know Hindi, so let's give this a shot.
(Speaks Hindi)
G: (Speaks Hindi)
NB: So Gemini said
you all look focused and excited,
and it has a better accent than I do.
(Laughter)
SI: Alright, now let's see how
the AI can connect the physical world
with your digital content and take action.
NB: Yeah, let’s get
some music going in here.
OK, Gemini, why don't you
play us a track from this record?
GB: Sure, here’s
“Bad Dreams” by Teddy Swims.
(Music)
SI: Perfect.
In a few seconds,
the AI recognized the album,
looked up the tracklist
and took action using the phone apps.
OK, Nishtha, it does look
like the demo gods are with us.
Maybe with you more than me.
Let's do one last demo
I know you're keen to give.
NB: Yes, this is
my first time in Vancouver,
and I love going on walks.
So why don't you navigate me
to a park nearby with views of the ocean?
G: OK, I am starting
navigation to Lighthouse Park,
which has magnificent views
of the Pacific Ocean.
Is there anything else
I can assist you with?
NB: Honestly, with these
directions and a 3D map,
I should be all set, and hopefully
I won’t look like a tourist.
Thank you all.
(Applause)
SI: Thank you, Nishtha, that was awesome.
OK, we've seen glasses.
Now let's turn our attention to the other
side of the spectrum: headsets.
You've seen these types of devices before,
but when we first brought AI to a headset,
it completely caught me by surprise.
For this demo, we're going to use
the Project Moohan headset
that Samsung is launching later this year.
Compared to glasses,
headsets give you an infinite display
for getting work done
or immersing yourself in a movie,
or maybe a TED Talk at some point.
Let me bring up my colleague Max
to show us even more.
Hey, Max.
Max Spear: Hello.
(Applause)
SI: And the same thing
is going to go here.
You'll see exactly what
Max is seeing behind on the screen.
Go for it, Max.
MS: Let’s do it.
You'll notice we start
grounded in the real world.
And I'm going to control the entire system
with my eyes, hands and voice.
But where things get really interesting
is when we invite Gemini in
as this conversational companion
that can come with us anywhere.
G: Hello.
MS: Hey, Gemini, can you bring up
my trip planner for me, please?
G: Of course.
Opening up your trip planner.
MS: Cool, but I left these windows
really disorganized.
Can you help with that?
G: Of course I can help with that.
SI: No clicks, no keyboards.
It's just a conversation.
And the AI is taking action.
OK, some more audience participation.
Someone shout out a name
of a place you want to visit.
Audience: Melbourne.
Audience: Cape Town.
SI: Let's go to Cape Town.
Max: OK, sounds fun.
Can you please take me to Cape Town?
G: Certainly.
Let me help with organizing the windows.
SI (Laughs)
MS: Awesome.
And can you also take me to Cape Town?
(Laughter)
G: I can certainly do that.
Let me take you to Cape Town.
MS: And we’re very organized
as we go there, perfect.
SI: As you can see, the AI
is taking Max's requests,
figuring out how best to answer it,
opening up the Maps app.
And from there, he can actually explore
anywhere in the world in this 3D view.
MS: OK, this is pretty incredible.
Viewing the world from this angle,
I can even zoom in to city levels.
But what's really interesting
is having an AI here
who can see what I see.
Can you tell me more about
the significance of this place?
G: I can indeed provide
information about Table Mountain.
Table Mountain holds
a profound significance,
deeply rooted in both its natural splendor
and its rich cultural history.
It has long been a spiritual
and cultural symbol
for the Indigenous
Khoekhoe and San people.
MS: OK, very good, Gemini.
I'll come back to this later.
It also does look
like a table, that's handy.
SI: OK, Max, let's bring this
closer to home.
How about exploring winter sports,
seeing that we're in Vancouver.
MS: OK, let’s get
some inspiration for Shahram.
Can you please show me 360 videos of this?
[Powder snowboarding]
G: Sure, here are some 360-degree
powder snowboarding videos.
MS: OK, I’m kind of new to snowboarding,
so hopefully this video
will help me learn.
Although this looks
already way more intense
than I think I'm ready for.
What was the name of that trick?
G: That’s a backside 540 with a grab.
SI: The AI knows
exactly what Max is looking at,
even in rich video content,
and can answer any question in real-time.
Max, are you sure this is Whistler?
MS: You know, Shahram,
I actually have no idea.
I've never been here before,
but let's find out.
What run do you think this is?
G: It’s probably one
of the ungroomed runs in summit bowl
or the top section of West Ridge
on Mount Bachelor,
as they are close
to the summit and face north
towards Three Sisters and Broken Top,
the prominent mountains visible
in the background.
MS: OK, Mount Bachelor, Oregon, got it.
SI: OK, let's have
a little bit of fun now.
Let's have AI narrate this video.
Somebody in the audience,
shout out a fun narration style
like, nature documentary
or something like that.
Audience: Morgan Freeman.
SI: We can't do actors,
but we'll get to that at some point.
Any other suggestions?
Audience: Movie trailer.
Audience: Horror movie.
SI: Horror movie.
That's going to be -- it might be gory,
but let's go for it.
MS: Slightly dark, but let’s do it.
Can you please describe what you're seeing
as if you're an overly enthusiastic
character in a horror movie?
G: Oh, darling,
this is simply marvelous.
A desolate mountain scape.
How delightfully ominous.
Prepare for a chilling dance
with the unknown.
Where every gust of wind whispers tales
of icy terror and lurking shadows.
MS: OK, Gemini, that’s great.
Return to more positive
things another day.
SI: OK, Max, I know there's one last demo,
and you've been apparently
practicing your farming skills?
MS: Yes, Shahram, I’ve been excited
to get back to my rural roots
and also embrace
one of my favorite things,
which is playing games.
But it becomes a whole new,
next-level experience
when you have an AI by your side,
both to help keep you company as you play,
but also to help you learn how to play
these kind of games in the first place.
So, for example, what do you think
I should do next here?
G: It looks like this is early
in the game in “Stardew Valley.”
You need to earn money,
so try planting some seeds.
First, use your hoe to till the soil,
and then plant your seeds.
It looks like you have some parsnips.
Since it isn't raining,
you'll also need to water them
with your watering can.
MS: Seems like a lot, and maybe
robots will do this for us one day,
but for now, I'm going to have
to return back to my farm later.
For one last thing, can you please
give a playful two-sentence summary
of all the ground we covered today?
G: We soared from
organizing a digital realm
to traversing snowy mountains,
and even found time to farm
some virtual parsnips.
Quite the adventure today.
MS: Quite the adventure, indeed.
Thanks, Gemini.
(Applause)
SI: Fantastic, thank you Max.
We're entering an exciting new phase
of the computing revolution.
Headsets and glasses
are just the beginning.
All this points to a single
vision of the future,
a world where helpful AI
will converge with lightweight XR.
XR devices will become
increasingly more wearable,
giving us instant access to information.
While AI is going to become more
contextually aware, more conversational,
more personalized,
working with us on our terms
and in our language.
We're no longer augmenting our reality,
but rather augmenting our intelligence.
Thank you so much.
(Applause)