How to pronounce "modes"

So today, I would like to talk with you

about bionics,

which is the popular term

for the science of replacing part of a living organism

with a mechatronic device, or a robot.

It is essentially

the stuff of life meets machine.

And specifically, I'd like to talk with you

about how bionics is evolving

for people with arm amputations.

This is our motivation.

Arm amputation causes a huge disability.

I mean, the functional impairment is clear.

Our hands are amazing instruments.

And when you lose one, far less both,

it's a lot harder to do the things

we physically need to do.

There's also a huge emotional impact.

And actually, I spend as much of my time in clinic

dealing with the emotional adjustment of patients

as with the physical disability.

And finally, there's a profound social impact.

We talk with our hands.

We greet with our hands.

And we interact with the physical world with our hands.

And when they're missing,

it's a barrier.

Arm amputation is usually caused by trauma,

with things like industrial accidents,

motor vehicle collisions

or, very poignantly, war.

There are also some children who are born without arms,

called congenital limb deficiency.

Unfortunately, we don't do great

with upper-limb prosthetics.

There are two general types.

They're called body-powered prostheses,

which were invented just after the Civil War,

refined in World War I and World War II.

Here you see a patent

for an arm in 1912.

It's not a lot different

than the one you see on my patient.

They work by harnessing shoulder power.

So when you squish your shoulders, they pull on a bicycle cable.

And that bicycle cable can open or close a hand or a hook

or bend an elbow.

And we still use them commonly,

because they're very robust

and relatively simple devices.

The state of the art

is what we call myoelectric prostheses.

These are motorized devices

that are controlled

by little electrical signals from your muscle.

Every time you contract a muscle,

it emits a little electricity

that you can record with antennae or electrodes

and use that to operate the motorized prosthesis.

They work pretty well

for people who have just lost their hand,

because your hand muscles are still there.

You squeeze your hand, these muscles contract.

You open it, these muscles contract.

So it's intuitive, and it works pretty well.

Well how about with higher levels of amputation?

Now you've lost your arm above the elbow.

You're missing not only these muscles,

but your hand and your elbow too.

What do you do?

Well our patients have to use

very code-y systems

of using just their arm muscles

to operate robotic limbs.

We have robotic limbs.

There are several available on the market, and here you see a few.

They contain just a hand that will open and close,

a wrist rotator and an elbow.

There's no other functions.

If they did, how would we tell them what to do?

We built our own arm at the Rehab Institute of Chicago

where we've added some wrist flexion and shoulder joints

to get up to six motors, or six degrees of freedom.

And we've had the opportunity to work with some very advanced arms

that were funded by the U.S. military, using these prototypes,

that had up to 10 different degrees of freedom

including movable hands.

But at the end of the day,

how do we tell these robotic arms what to do?

How do we control them?

Well we need a neural interface,

a way to connect to our nervous system

or our thought processes

so that it's intuitive, it's natural,

like for you and I.

Well the body works by starting a motor command in your brain,

going down your spinal cord,

out the nerves and to your periphery.

And your sensation's the exact opposite.

You touch yourself, there's a stimulus

that comes up those very same nerves back up to your brain.

When you lose your arm, that nervous system still works.

Those nerves can put out command signals.

And if I tap the nerve ending

on a World War II vet,

he'll still feel his missing hand.

So you might say,

let's go to the brain

and put something in the brain to record signals,

or in the end of the peripheral nerve and record them there.

And these are very exciting research areas,

but it's really, really hard.

You have to put in

hundreds of microscopic wires

to record from little tiny individual neurons -- ordinary fibers

that put out tiny signals

that are microvolts.

And it's just too hard

to use now and for my patients today.

So we developed a different approach.

We're using a biological amplifier

to amplify these nerve signals -- muscles.

Muscles will amplify the nerve signals

about a thousand-fold,

so that we can record them from on top of the skin,

like you saw earlier.

So our approach is something we call targeted reinnervation.

Imagine, with somebody who's lost their whole arm,

we still have four major nerves

that go down your arm.

And we take the nerve away from your chest muscle

and let these nerves grow into it.

Now you think, "Close hand," and a little section of your chest contracts.

You think, "Bend elbow,"

a different section contracts.

And we can use electrodes or antennae

to pick that up and tell the arm to move.

That's the idea.

So this is the first man that we tried it on.

His name is Jesse Sullivan.

He's just a saint of a man --

54-year-old lineman who touched the wrong wire

and had both of his arms burnt so badly

they had to be amputated at the shoulder.

Jesse came to us at the RIC

to be fit with these state-of-the-art devices, and here you see them.

I'm still using that old technology

with a bicycle cable on his right side.

And he picks which joint he wants to move with those chin switches.

On the left side he's got a modern motorized prosthesis

with those three joints,

and he operates little pads in his shoulder

that he touches to make the arm go.

And Jesse's a good crane operator,

and he did okay by our standards.

He also required a revision surgery on his chest.

And that gave us the opportunity

to do targeted reinnervation.

So my colleague, Dr. Greg Dumanian, did the surgery.

First, we cut away the nerve to his own muscle,

then we took the arm nerves

and just kind of had them shift down onto his chest

and closed him up.

And after about three months,

the nerves grew in a little bit and we could get a twitch.

And after six months, the nerves grew in well,

and you could see strong contractions.

And this is what it looks like.

This is what happens when Jesse thinks

open and close his hand,

or bend or straighten your elbow.

You can see the movements on his chest,

and those little hash marks

are where we put our antennae, or electrodes.

And I challenge anybody in the room

to make their chest go like this.

His brain is thinking about his arm.

He has not learned how to do this with the chest.

There is not a learning process.

That's why it's intuitive.

So here's Jesse in our first little test with him.

On the left-hand side, you see his original prosthesis,

and he's using those switches

to move little blocks from one box to the other.

He's had that arm for about 20 months, so he's pretty good with it.

On the right side,

two months after we fit him with his targeted reinnervation prosthesis --

which, by the way, is the same physical arm,

just programmed a little different --

you can see that he's much faster

and much smoother as he moves these little blocks.

And we're only able to use three of the signals at this time.

Then we had one of those little surprises in science.

So we're all motivated to get motor commands

to drive robotic arms.

And after a few months,

you touch Jesse on his chest,

and he felt his missing hand.

His hand sensation grew into his chest again

probably because we had also taken away a lot of fat,

so the skin was right down to the muscle

and deinnervated, if you would, his skin.

So you touch Jesse here, he feels his thumb;

you touch it here, he feels his pinky.

He feels light touch

down to one gram of force.

He feels hot, cold, sharp, dull,

all in his missing hand,

or both his hand and his chest,

but he can attend to either.

So this is really exciting for us,

because now we have a portal,

a portal, or a way to potentially give back sensation,

so that he might feel what he touches

with his prosthetic hand.

Imagine sensors in the hand

coming up and pressing on this new hand skin.

So it was very exciting.

We've also gone on

with what was initially our primary population

of people with above-the-elbow amputations.

And here we deinnervate, or cut the nerve away,

just from little segments of muscle

and leave others alone

that give us our up-down signals

and two others that will give us a hand open and close signal.

This was one of our first patients, Chris.

You see him with his original device

on the left there after eight months of use,

and on the right, it is two months.

He's about four or five times as fast

with this simple little performance metric.

All right.

So one of the best parts of my job

is working with really great patients

who are also our research collaborators.

And we're fortunate today

to have Amanda Kitts come and join us.

Please welcome Amanda Kitts.

(Applause)

So Amanda, would you please tell us how you lost your arm?

Amanda Kitts: Sure. In 2006, I had a car accident.

And I was driving home from work,

and a truck was coming the opposite direction,

came over into my lane,

ran over the top of my car and his axle tore my arm off.

Todd Kuiken: Okay, so after your amputation, you healed up.

And you've got one of these conventional arms.

Can you tell us how it worked?

AK: Well, it was a little difficult,

because all I had to work with was a bicep and a tricep.

So for the simple little things like picking something up,

I would have to bend my elbow,

and then I would have to cocontract

to get it to change modes.

When I did that,

I had to use my bicep

to get the hand to close,

use my tricep to get it to open,

cocontract again

to get the elbow to work again.

TK: So it was a little slow?

AK: A little slow, and it was just hard to work.

You had to concentrate a whole lot.

TK: Okay, so I think about nine months later

that you had the targeted reinnervation surgery,

took six more months to have all the reinnervation.

Then we fit her with a prosthesis.

And how did that work for you?

AK: It works good.

I was able to use my elbow

and my hand simultaneously.

I could work them just by my thoughts.

So I didn't have to do any of the cocontracting and all that.

TK: A little faster?

AK: A little faster. And much more easy, much more natural.

TK: Okay, this was my goal.

For 20 years, my goal was to let somebody

[be] able to use their elbow and hand in an intuitive way

and at the same time.

And we now have over 50 patients around the world who have had this surgery,

including over a dozen of our wounded warriors

in the U.S. armed services.

The success rate of the nerve transfers is very high.

It's like 96 percent.

Because we're putting a big fat nerve onto a little piece of muscle.

And it provides intuitive control.

Our functional testing, those little tests,

all show that they're a lot quicker and a lot easier.

And the most important thing

is our patients have appreciated it.

So that was all very exciting.

But we want to do better.

There's a lot of information in those nerve signals,

and we wanted to get more.

You can move each finger. You can move your thumb, your wrist.

Can we get more out of it?

So we did some experiments

where we saturated our poor patients with zillions of electrodes

and then had them try to do two dozen different tasks --

from wiggling a finger to moving a whole arm

to reaching for something --

and recorded this data.

And then we used some algorithms

that are a lot like speech recognition algorithms,

called pattern recognition.

See.

(Laughter)

And here you can see, on Jesse's chest,

when he just tried to do three different things,

you can see three different patterns.

But I can't put in an electrode

and say, "Go there."

So we collaborated with our colleagues in University of New Brunswick,

came up with this algorithm control,

which Amanda can now demonstrate.

AK: So I have the elbow that goes up and down.

I have the wrist rotation

that goes -- and it can go all the way around.

And I have the wrist flexion and extension.

And I also have the hand closed and open.

TK: Thank you, Amanda.

Now this is a research arm,

but it's made out of commercial components from here down

and a few that I've borrowed from around the world.

It's about seven pounds,

which is probably about what my arm would weigh

if I lost it right here.

Obviously, that's heavy for Amanda.

And in fact, it feels even heavier,

because it's not glued on the same.

She's carrying all the weight through harnesses.

So the exciting part isn't so much the mechatronics,

but the control.

So we've developed a small microcomputer

that is blinking somewhere behind her back

and is operating this

all by the way she trains it

to use her individual muscle signals.

So Amanda, when you first started using this arm,

how long did it take to use it?

AK: It took just about probably three to four hours

to get it to train.

I had to hook it up to a computer,

so I couldn't just train it anywhere.

So if it stopped working, I just had to take it off.

So now it's able to train

with just this little piece on the back.

I can wear it around.

If it stops working for some reason, I can retrain it.

Takes about a minute.

TK: So we're really excited,

because now we're getting to a clinically practical device.

And that's where our goal is --

to have something clinically pragmatic to wear.

We've also had Amanda able to use

some of our more advanced arms that I showed you earlier.

Here's Amanda using an arm made by DEKA Research Corporation.

And I believe Dean Kamen presented it at TED a few years ago.

So Amanda, you can see,

has really good control.

It's all the pattern recognition.

And it now has a hand that can do different grasps.

What we do is have the patient go all the way open

and think, "What hand grasp pattern do I want?"

It goes into that mode,

and then you can do up to five or six different hand grasps with this hand.

Amanda, how many were you able to do with the DEKA arm?

AK: I was able to get four.

I had the key grip, I had a chuck grip,

I had a power grasp

and I had a fine pinch.

But my favorite one was just when the hand was open,

because I work with kids,

and so all the time you're clapping and singing,

so I was able to do that again, which was really good.

TK: That hand's not so good for clapping.

AK: Can't clap with this one.

TK: All right. So that's exciting

on where we may go with the better mechatronics,

if we make them good enough

to put out on the market and use in a field trial.

I want you to watch closely.

(Video) Claudia: Oooooh!

TK: That's Claudia,

and that was the first time

she got to feel sensation through her prosthetic.

She had a little sensor at the end of her prosthesis

that then she rubbed over different surfaces,

and she could feel different textures

of sandpaper, different grits, ribbon cable,

as it pushed on her reinnervated hand skin.

She said that when she just ran it across the table,

it felt like her finger was rocking.

So that's an exciting laboratory experiment

on how to give back, potentially, some skin sensation.

But here's another video that shows some of our challenges.

This is Jesse, and he's squeezing a foam toy.

And the harder he squeezes -- you see a little black thing in the middle

that's pushing on his skin proportional to how hard he squeezes.

But look at all the electrodes around it.

I've got a real estate problem.

You're supposed to put a bunch of these things on there,

but our little motor's making all kinds of noise

right next to my electrodes.

So we're really challenged on what we're doing there.

The future is bright.

We're excited about where we are and a lot of things we want to do.

So for example,

one is to get rid of my real estate problem

and get better signals.

We want to develop these little tiny capsules

about the size of a piece of risotto

that we can put into the muscles

and telemeter out the EMG signals,

so that it's not worrying about electrode contact.

And we can have the real estate open

to try more sensation feedback.

We want to build a better arm.

This arm -- they're always made for the 50th percentile male --

which means they're too big for five-eighths of the world.

So rather than a super strong or super fast arm,

we're making an arm that is --

we're starting with,

the 25th percentile female --

that will have a hand that wraps around,

opens all the way,

two degrees of freedom in the wrist and an elbow.

So it'll be the smallest and lightest

and the smartest arm ever made.

Once we can do it that small,

it's a lot easier making them bigger.

So those are just some of our goals.

And we really appreciate you all being here today.

I'd like to tell you a little bit about the dark side,

with yesterday's theme.

So Amanda came jet-lagged,

she's using the arm,

and everything goes wrong.

There was a computer spook,

a broken wire,

a converter that sparked.

We took out a whole circuit in the hotel

and just about put on the fire alarm.

And none of those problems could I have dealt with,

but I have a really bright research team.

And thankfully Dr. Annie Simon was with us

and worked really hard yesterday to fix it.

That's science.

And fortunately, it worked today.

So thank you very much.

(Applause)