Matthias Neisser has developed some very creepy software, but he’d really prefer I not frame it that way. He’s grown tired of allusions to dictators and propagandists. He’s sick of questions about manipulation and identity.
This is not quite what Neisser had in mind for Face2Face, and it still isn’t. He sees potential use cases in Hollywood, where film-dubbing remains disastrous, as well as in Silicon Valley, where the visiting Stanford professor currently lives. He thinks it could be particularly useful in virtual reality, where there’s a growing desire to reconstruct faces.
And yet, Neisser can’t quite escape the creep factor, try as he might. At the Hirshhorn last Thursday, for instance, his demo preceded a lecture by the new media artist Josh Kline; Kline uses face-substitution software similar to Neisser’s to critique the anxieties of modern life. “It’s about the power to hijack identities,” he said, “to take over someone’s face, someone’s voice.”
The Intersect sat down with Neisser to discuss creepiness, deception, and the source of both. (Spoiler alert: We have very different ideas on that last one!) This interview has been edited for length and clarity. FYI, the exhibit Neisser was in town to discuss is on display at the Hirshorn until March 2017.
People always say the same thing when they see this technology. First they say it’s cool, then they say it’s “frightening” or “creepy.” How do you think the culture is going to overcome that initial, visceral reaction?
I’ve been in computer graphics for a few years now. And so to me, this reaction is always a bit surprising, because a lot of the stuff we see all the time is synthetically generated. I use the Avatar example a lot: In that movie, and a lot of movies right now, everything is synthetic. I think it’s important that people actually know that. You have to be aware that whatever you see in a video isn’t necessarily real.
That’s one thing in a huge Hollywood blockbuster like Avatar, though — people probably don’t have that expectation for a piece of news footage on YouTube, say.
But maybe they should. The only difference with our technology is that it allows you to edit an existing video. For a movie, you have to build 3D models first, you have a lot of art, there’s a lot of overhead — we do eliminate the need for that, we kind of make it easy. But the outcome is the same with our technology as it would be for someone who had enough resources to generate a synthetic video.
So say for instance you have a government who wants to deceive you — our technology doesn’t make a difference to them, because they have the resources anyway, right? But we’re opening that editing ability they have to everybody.
You’ve said something interesting, along these lines — that “the technology is advancing, and we have to make sure it goes in the right direction.” Who is “we”? Who’s responsible for this?
Well in an academic environment, you would say the community, which is a very vague term. We mean the community of other researchers. I think there is some responsibility on this side still.
But I think a lot of responsibility comes from companies, too, and from investors. A good example right now is Elon Musk — he invested in a company called Open AI, and they’re trying to figure out how to develop machine learning to make it publicly accessible so everybody can use it and monitor it. It’s basically a huge effort to make sure the machines don’t destroy us. It sounds very premature, but that’s the fear that a lot of people have. And I can see that it happens because the media — no offense.
No offense taken!
The media exacerbates it. That’s the job. That’s how you create a viewership, right? It’s a fair game. But that kind of leads to fear. I think it’s important that technology becomes accessible to everybody at some point, and that people know about it and understand it exists. If you didn’t know about our program, and somebody else edited a video and sent it to you, that would be pretty bad! But if you know that every video is fake then it doesn’t make a real difference.
I get the impression that you’re pretty sick of people asking you about this technology’s darker implications. I mean, you live in Silicon Valley, you’re very optimistic, you are obviously super-jazzed about it …
Yeah — it gets a bit old. It’s always the same question. We’re obviously not trying to make any political statements or whatever, or trying to steal identities — we’re really here to help out specific commercial applications. But the media tries to make some story out of it, which makes sense —
But it’s not just media, right? The piece that you’re in D.C. to talk about is by an artist who is fundamentally critical of this technology and its implications for identity and civic good. And there are other academics, other commentators — any discussion of its possible applications seems like a valid criticism.
I think artists in this case are not necessarily representative of public opinion. Like the question is, what sells in public and with viewers? So people write whatever somebody wants to read. It’s kind of entertainment. And art is kind of the same way — if you’re an artist and you don’t have any followers, any fans, than you have a problem, right? So — I kind of think that’s how it goes.
I don’t know. I can understand that some generations are kind of, you know, worried — but we’re not doing anything evil. We’re not trying to help people fake things. We’re really on the other side, trying to help people figure out what’s going on and what we can do with digital data. Digital data is just bits and bytes. You can edit that, right. So how do you make sure that is actually the right thing?
How long do we have before we have to confront some of these questions?
Well — we’ve been working on this for about a year, I would say. It’s all relatively new. This field is that is moving so quickly — there’s so much cool stuff happening right now, both in terms of sensor technology and machine learning and optimization methods.
I like thinking about the smartphone — smartphones changed a lot, and changed the world a lot, in a relatively short period of time. Everybody has a smartphone right now, and these are computers that used to be in big desktops 10 years ago. And they are really, really good and they can do a lot of great stuff — but now since the smartphone market, at least in the U.S., is pretty saturated, people are thinking “what’s the next development?”
So what’s next for you, specifically? What are you working on?
We are working on several commercial applications in the context of virtual reality right now. A lot of people have asked us to fix the eye-tracking problem with video-conferencing — when you want to look into the webcam, but in fact you’re looking at the Skype screen, and it looks really weird. That’s one project. Another involves goggle removal in VR. If you want to have a conversation in VR, but you have the VR headset, it’s kind of disturbing any reasonable conversation.
Beyond that, we’re really interested in editing full bodies in videos. That is really hard. The tracking is just not accurate enough. And if you edit a body in a video, you also have to edit what’s behind the body … and [that] is pretty complicated because you may have never seen it. That’s going to be very interesting right now from a research perspective.
Currently I think computer science is one of the most exciting fields out there right now. If you look at the universities you have so many people want to major in CS. You can actually help to shape the future. If you think about how much impact you can have — that’s a big difference compared to some other fields.
Journalism, for instance?
No, in journalism you can impact a lot of things too. Just write a positive story, okay?
It’s gonna be a Q&A, so you actually have a lot of control! In print, anyway. I couldn’t speak for video.
Correction: Due to a transcription error, Niesser’s name was initially misspelled. Your author is totally unfamiliar with the German language, but glad she learned something today — sorry about that!
Liked that? Try these!
