03x08 - Mind Reading

Episode transcripts for the TV show "Mind Field". Aired: January 2017 to October 2019.*
Watch/Buy Amazon


"Mind Field" takes a journey into the mysterious depths of the human psyche and investigates the strange and surprising terrain of the Mind Field.
Post Reply

03x08 - Mind Reading

Post by bunniefuu »

Mind reading?

Of course not.

I love reading.

Look, mind reading might sound
like pseudoscientific--

pardon my language--

bullshoot.

But its scientific counterpart,
thought identification,

is very much a real thing.

It's based in neuroimaging
and machine learning,

and what's really cool is
that experiments in mind reading

aren't just about spying
on what someone is thinking.





They're about figuring out
what thoughts are even made of.

I mean, when I think
of something,

what does that mental picture
actually look like?

What resolution is it in?

How high fidelity
is a memory,

and how do they change
over time?

Well, in this episode,

I'm going to look at how
reading people's minds

can help us answer
these questions.

My journey begins right here
at the University of Oregon.

I'm meeting with Dr. Brice Kuhl
from the Kuhl lab.

He's a neuroscientist
who uses neuroimaging

and machine learning to figure
out what people are thinking

without them telling him.

So tell me what
you're doing here.

Well, I'm in the cognitive
neuroscience program here,

and I study human memory.

My lab primarily uses
neuroimaging methods,

so we do a lot of work using

functional magnetic
resonance imaging,

or fMRI.

And how do you use
fMRI to investigate memories?

We're looking at the pattern
of neural activity.

When you form a memory,
there's a certain pattern.

And we can record
that pattern

and then test whether
that pattern is reinstated

or reactivated at a later point,
like when you're remembering it.

Does that mean we can look at
the patterns of brain activity

and deduce what it is that is
being remembered, or recalled,

or even just thought?

Yes, and so we call that
decoding.

So it basically takes
your input pattern

as some pattern of activity
that we record

while you're remembering
something.

And we make a prediction
about what you're remembering.

You can see how this sounds
like mind reading.

[laughs]
Yes. It sounds like that.

So, Brice, what are you going
to do to me today?

So, what we're going
to be doing today

is uncharted territory
for us.

So we're going to be trying out
a kind of new variant

of the experiment on you.

So I can't guarantee
any particular results.

But it represents
where the field is

and where
we're trying to go.

Today, you're going to
participate in an experiment

where you'll be studying faces.

So we're going
to have you study

12 pictures of celebrities.

People I already am
familiar with.

-People that you know, yeah.
-Okay.

And you're going to try
to remember those pictures.

Then we're going to have you go
into the MRI scanner.

Try to bring that picture
to mind as vividly as possible.

And we're going to be recording
your brain activity

as you try to imagine
these pictures.

We're going to try
to build the face.

Essentially draw a picture of
what you're remembering.

-A picture?
-A picture.

An actual picture
that we can print out

and I could, like,
hang on my wall.

[laughs]
If you wanted.

[Michael] The first step
is for me to memorize

the 12 specific
celebrity photographs

Brice will later try
to detect me thinking about.

I sat down to do this
graduate student, Max.

The success of his predictions
depend, in part,

on my ability
to recall these faces

as vividly as possible
while inside the fMRI.

All right, so...

[sighs]

I think I have a pretty good
memory of all of those.

-Great.
-I feel the stakes are high.

With the celebrity faces
hopefully memorized,

it's time for the next step:

going through
the metal detector

and into the fMRI,

where Brice will record
and monitor my brain activity,

and then later feed it into his
algorithm to rebuild the faces.

This will be the first time
he's attempted

to reconstruct faces
from long-term memory,

which is very difficult,
because we're relying

on how clearly I can remember
the celebrity photos

I saw an hour ago.

I love its eyes.
Look at that.

[woman]

Wouldn't the kid be like,
"It's going to eat me"?

An fMRI monitors the activity
within the brain

by dividing it up
into thousands of small cubes

called voxels,
or volumetric pixels.

Each of these voxels contains

hundreds of thousands
of neurons.

Using fMRI,
we are able to detect

blood flow
within these voxels,

which means that that part
of the brain is active.

If I'm shown several pictures
of people with mustaches,

my brain will react
to the features for each face.

But there will be
a common area of my brain

that is engaged
throughout.

That may be the area of my
brain that reacts to mustaches.

So later,
when I imagine a face,

if Brice notices
that area is engaged,

he can predict
that I am thinking

about a mustache.

So right now Michael's
in the scanner,

and he's seeing words appear
on the screen one at a time,

and he's trying
to visualize the face,

remember the face in as much
detail as possible.

What you can see here are
the images that we're acquiring.

We get one of these
brain volumes every two seconds.

So these are refreshing in real
time as we collect the images.

[Michael] With part one
of the fMRI session over,

it's time for part two,
where Brice and his team

will learn the language
of my brain activity,

so they can later
decode by brain scans.

Hi, Michael.
You doing okay still?

[Michael]
Yup.

They'll show me hundreds
of unique faces,

and record how my brain reacts

to certain facial
characteristics.

They will then use
this information

to reconstruct
the celebrity faces

I thought about during
the first phase of the scan.

Really, the more faces that
we can show Michael, the better.

So we're going to basically keep
him in there

as long as he's comfortable.

[Michael]
Two hours was the maximum time

we could get in the fMRI.

But I was able to look
at over 400 faces,

which should be enough to get

some pretty interesting
results.

Hey, Michael, you did it.
That was great.

We're going to come
get you out.

[Michael]
All right.

Yeah, so these just show
some of the pictures

that we were taking
while you were in there.

Some images of your brain.

Now we are going
to crunch some numbers.

Max is going to analyze
your data.

We'll meet up
again tomorrow,

where we'll look
at the results,

where we try to actually
reconstruct the face images

from the brain data
that we just collected.

All right.
Well, see you tomorrow.

All right.
Thanks a lot.

Max, thank you as well.
I can't wait.

You better pull
an all-nighter.

I want this data
to be perfect.

All right, so I am back
at Dr. Kuhl's lab.

Overnight, his team
crunched the data,

and I can't wait to see what
they think they saw me thinking.

How are my results?

I think they look good.

We're going to take a look
in just a moment here.

All right,
I can't wait.

-So can I just take a seat?
-Yeah, have a seat.

All right, so...

first of all...

what am I seeing?
Oh, okay, well,

these are the pictures
I actually memorized.

-That's right.
-And this is what

you've reconstructed
from my imagination.

-That's right.
-Oh, wow. Okay.

[Brice]
Okay, so this is one
of the reconstructions

that was generated.

[Michael]
Interesting.

[Max]
So that's John Cho.

[Michael]
Not bad. Not bad.

-Can we see the side by side?
-Yeah.

[Michael]
I see, you know, similarities

in the kind of facial
expressions in general.

You know, you could almost
see the hairline matching here.

The shape of the face
I also thought was--

It kind of had
a square shape to it.

-Yes. Yes.
-So those are the things

that came out to me.

And so when I was
visualizing

this image of John Cho,

the squareness of the face was
the first, most salient thing.

I just kept thinking,
he was the square guy.

Excellent, all right.

[Brice]
So that's Megan Fox.

[Michael]
Mm-hmm.

You're going to show us the--
side by side.

[Michael]
The side by side. Right.

[Brice]
You can see the picture
you actually saw,

and that's the reconstruction
we generated.

I'll you this.
Megan Fox, I was not able

to have a really clear picture
in my mind.

For some reason, this image
of her was really hard for me

to bring back into my mind.

The sternness in the face was
something that I did pick up on.

So I did sense that there was--
It looked feminine.

And you picked up
on the sternness.

And so together,
that produces a match.

[Michael] Keep in mind
that Brice and his team

have read these
from my memory.

But when I remember a face,

do I picture every detail
simultaneously

with photographic accuracy?

Or do I just attend to a few
at a time?

By reading my mind,
they may be seeing

how bad my memory is,
and how it works.

-Me! Me!
-[Brice laughs]

Okay, so that is your
reconstruction

of me thinking about
this image of myself.

[Brice]
That's right.

Where'd the beard go?

[Brice] I don't know.
I was hoping you could tell me.

[Michael]
For instance, this is a picture
of me remembering my own face.

It really doesn't look like me,
but the question is:

how good am I
at picturing myself?

I don't think of my own face
that often,

so the strangeness
in the result

may be as much about flaws
in my own memory

and mental picture of myself
as flaws in the technology.

So that's Jennifer Lawrence,
I believe.

[Michael]
That's Jennifer Lawrence?

It looks like it's Jennifer
Lawrence's much older uncle.

[all chuckle]

Nothing here was too
mind-blowingly close.

But this is something that
you're just starting out trying

these sort of long-term
memories.

What Brice and his team
read in my mind

might have been more accurate
if they'd shown me thousands

rather than hundreds of images
in the fMRI,

because then the algorithm
would have learned

the language of my brain
more thoroughly.

But regardless,
the quality of my memories

would have still
been an issue.

I mean, look what happens
when memory

is cut out of the equation
entirely.

Brice also read
my brain activity

when I was looking
at faces in the fMRI.

not just imagining them.

And those results
were much closer

than those reconstructed
from my memory.

Okay, so, what am I looking at
right here?

[Brice]
Okay, so what you're seeing here

in the top row,
these are images that you saw

while you were in the scanner.

Below that, in this bottom row,
these are the reconstructions

that we draw from the patterns
of brain activity we collected.

-This is from the source image.
-Right.

[Michael]
These are from my brain.

-[Brice] Right.
-[Michael] They're pretty close.

Yeah, overall they were
pretty close.

So not perfect.

These are-- you can see there's
some variability in these.

But this is consistent
with what we've found before,

that the reconstructions
that we generated,

when you're viewing
the faces,

there is some correspondence
between the actual face.

So this is kind of
a sanity check,

that we can actually
reconstruct the images

-when you're viewing them.
-Right, right.

They're pretty good.

Well, Brice, Max,
thank you so much

for letting me
be a part of this.

I hope my data's useful.

Thank you.
It's been a lot of fun.

It's always useful for us
to think about these things.

Dr. Brice Kuhl's memory research
is showing that it's possible

for a computer
to read someone's mind.

To figure out
what they're thinking.

But a lot of progress
still needs to be made.

I mean, if you want to know

what I'm thinking right now,
for example,

it's still easier to just ask me
to tell you.

But what if I can't
tell you?

Dr. Yukiyasu Kamitani
is a researcher,

professor and pioneer
exploring the frontier

behind the wall of sleep.

I've come here
to Kyoto University

to meet with him and to see
what it's like

to read not what someone
is thinking,

but what someone
is dreaming.

Kamitani sensei,
I'm Michael.

-Hi, I'm Yuki.
-Yuki, nice to meet you.

[Michael]
For the last ten years,

Dr. Kamitani has been
at the forefront

of machine mind reading.

The subject is, you know,
ready to go in.

Similar to Brice Kuhl,

his early experiments explored
reconstructing images

shown to subjects in an fMRI
based on their brain activity.

In Kamitani's case,

the images were
black-and-white shapes,

and the reconstructions
were strikingly accurate.

Recently, Kamitani has focused
on using deep neural networks

and machine learning

to decipher subjects'
brain activity

while they view
much more complex photographs.

What you're seeing is the
result of a deep neural network

processing the brain activity
of a subject

looking at the photograph.


This could have myriad
applications in the future,

for example,
in criminal investigations

and interpersonal
communication.

[Kamitani]
This is far from perfect.

But I think you still see some,
you know, eyes and, you know...

[Michael]
Well, yeah.

And colors too.

[Kamitani]
Yeah, to some extent, yeah.

His most current work, however,
is about the subconscious.

He's attempting something
extremely ambitious:

recording our dreams.

Would you call yourself
a sleep researcher,

or a vision researcher?

Maybe a brain decoder.

A brain decoder.

That's a pretty cool
job description.

Can you show me anything from
what you're doing with dreams?

[Kamitani]
Mm-hmm, yeah.

Dr. Kamitani's work
on dream decoding

begins with a similar process
to Dr. Kuhl's:

showing the test subject
thousands of images

while they are in an fMRI

in order to learn
what the brain looks like

when it is thinking
of certain things.

Once the machine-learning
algorithm is pretty good

at identifying what images
the subject is thinking about,

the subject is placed
in an fMRI

with an EEG cap
on their head,

and invited to fall asleep.

When the EEG waves indicate
that the person is dreaming,

the algorithm predicts
which kinds of things

the subject is most likely
dreaming about.

Right now, the algorithm
looks for 20 categories.

Things like buildings,
transportation,

and characters
in a language.

Researchers then awaken
the subject,

ask them what they were
dreaming about,

and see if the algorithm's
prediction

and the person's
recollection match.

Here is actual data from
one of Kamitani's experiments.

Below is a word cloud
of categories.

The name of each category
get bigger or smaller

in real time
based on the probability

that they are present
in the subject's current dream.

Now, as you can see,
activity is currently strongest

for the category "character,"
meaning written language.

At this point
the subject was awoken,

and this is what
they reported.

That's pretty spooky.

-[laughs]
-Right? I mean, you--

you spied on their dream.

Yeah, in a way.
But...

the accuracy's
not that great, so...

Well, the accuracy's
not that great but, you know,

my normal accuracy for guessing
people's dreams is zero.

Right.

While continuing
his research

into predicting
the content of dreams,

Dr. Kamitani is embarking
on his newest project:

actually reconstructing images
from our dreams.

So you've brought
some of the reconstructions

that your lab has created...

Mm-hmm.

...of dreams.

Right, they all look like dreams
about blobs.

[Kamitani]
Yeah.

I mean, I want to just
take a step back and...

appreciate that what we're
looking at on this screen

are, in a way, some of the first
photographs of a dream.

Mm-hmm.

We are looking
at the earliest phase

of revolutionary research.

One day, we may able
to have images,

or even record movies,
of our own dreams.

And Dr. Kamitani is the only
person in the world

doing this so far.

He's a lone explorer journeying
into our subconscious.

So this work hasn't even
been published yet.

No.

-Thank you for showing it to me.
-[laughs]

The insights that researchers
like Dr. Kuhl and Dr. Kamitani

might be capable of achieving
in the future

because of mind reading

are difficult
to fully fathom.

But let's slow down
for a second,

because we're talking
about a technology

that can know us
better than we know ourselves.

Should we really
be doing this?

Well, to address
that question,

I'm going to meet
with an expert in ethics,

neuroscience
and artificial intelligence:

Julia Bossmann.

She's the director of strategy
at Fathom Computing,

a council member
of the World Economic Forum,

an alum of Ray Kurzweil's
Singularity University,

and a former president
of the Foresight Institute,

a think t*nk specializing
in future technologies

and their impacts.

Julia, thanks for taking some
time to chat.

-Yeah, of course.
-You are the perfect person

for me to bring
these questions to.

-Mm-hmm.
-And they're deep questions.

But I think they're
extremely important,

and they're becoming
more and more pressing.

I think we're living in such
an interesting time right now,

because we're in this time
where brains and machines

are actually moving
closer together.

So when it comes
to being able

to look at brain activity,

where are
the ethical lines here?

How private should
my internal thoughts be?

Like with any powerful
technology,

it depends on the hands
that wield it.

All these new technologies

are things that can make whoever
uses them more powerful.

So we want to not blame
the technology, but we want to--

how is it being used,

and who is using it?

So how do we make sure
that this technology

is in the right hands?

So I think it's very important
to involve people

who act on policy and law

to understand what is coming
in the future.

I am hopeful about
the collaborative aspect of it.

Let's talk about
the good things now.

I mean, what are
the applications here?

Yeah, so if we think about

the late Stephen Hawking,
for example,

if he had a way of richer
interfacing with the world

or with computers,
we can only imagine

what he could have
shared with us.

Those with locked-in syndrome,
right?

They are there.
They know that they are there.

But we just need something
to look into their brain

to see what it is
that they are trying to say,

-or what they're feeling.
-Right, exactly.

So, what do you say
to people

that have that kind of fear
of technology,

of us surrendering our true
natural selves to technology?

There is something enticing
about getting to the next level

of what some people might call
a human evolution

or civilization development,
and so on.

In a way, we are already not
living natural lives, right?

Because then most of us would
die before the age of,

I don't know, 30 or 40.

We would have all kinds
of diseases.

We would not wear
this clothing.

We wouldn't have eyeglasses
or contact lenses.

We wouldn't have antibiotics.

[Julia]
We are already kind of

very futuristic cyborgs
if we compare ourselves

to the human that was living
10,000 years ago

and was genetically
almost identical

with who we are now.

[Michael]
Yeah, we really are.

In order to understand
cognition,

right now we basically have
to either just ask people

to talk about
what they're thinking,

or observe their behavior.

But reading thoughts directly
would be a lot better.

That is how Dr. Kuhl
is studying memory,

and it's how Dr. Kamitani
is studying sleep and dreams.

But even though the technology
has a long way to go,

it's easy to see
how ethical questions

could become an issue.

Well, here's the thing:

there is no such thing
as a totally wild human.

We are co-evolving
with technology.

Humans and technology today
are inseparable.

Now, it's true that we need
to be careful

about every new thing we do,

but we cannot change the fact
that they will happen.

It's a story we've lived through
again and again.

You know, we could have
sat around forever

debating whether or not
a speed limit should exist

and who should have
the authority to enforce it.

But we didn't.

Instead, we went ahead
and invented cars,

and responsibly figured out
the details as we went along.

Ethical questions
about new technologies

do the most good when
they facilitate the technology,

not when they needlessly
hinder progress.

So follow your dreams.

And, as soon as you can,
show them to me.

And, as always,
thanks for watching.
Post Reply