Google just gave a stunning demo of Assistant making an actual phone call

Onstage at I/O 2018, Google showed off a jaw-dropping new capability of Google Assistant: in the not too distant future, it’s going to make phone calls on your behalf. CEO Sundar Pichai played back a phone call recording that he said was placed by the Assistant to a hair salon. The voice sounded incredibly natural; the person on the other end had no idea they were talking to a digital AI helper. Google Assistant even dropped in a super casual “mmhmmm” early in the conversation.

Pichai reiterated that this was a real call using Assistant and not some staged demo. “The amazing thing is that Assistant can actually understand the nuances of conversation,” he said. “We’ve been working on this technology for many years. It’s called Google Duplex.”

Duplex really feels like next-level AI stuff, but Google’s chief executive said it’s still very much under development. Google plans to conduct early testing of Duplex inside Assistant this summer “to help users make restaurant reservations, schedule hair salon appointments, and get holiday hours over the phone.”

Pichai says the Assistant can react intelligently even when a conversation “doesn’t go as expected” and veers off course a bit from the given objective. “We’re still developing this technology, and we want to work hard to get this right,” he said. “We really want it to work in cases, say, if you’re a busy parent in the morning and your kid is sick and you want to call for a doctor’s appointment.” Google has published a blog post with more details and soundbites of Duplex in action.

“The technology is directed towards completing specific tasks, such as scheduling certain types of appointments. For such tasks, the system makes the conversational experience as natural as possible, allowing people to speak normally, like they would to another person, without having to adapt to a machine.” Google envisions other use cases like having Assistant call businesses and inquire about their hours to help keep Maps listings up to date. The company says it wants to be transparent about where and when Duplex is being used, as a voice that sounds this realistic and convincing is certain to raise some questions.

In current testing, Google notes that Duplex successfully completes most conversations and tasks on its own without any intervention from a person on Google’s end. But there are cases where it gets overwhelmed and hands off to a human operator. This section on the ins and outs of Duplex is very interesting:

The Google Duplex system is capable of carrying out sophisticated conversations and it completes the majority of its tasks fully autonomously, without human involvement. The system has a self-monitoring capability, which allows it to recognize the tasks it cannot complete autonomously (e.g., scheduling an unusually complex appointment). In these cases, it signals to a human operator, who can complete the task.

To train the system in a new domain, we use real-time supervised training. This is comparable to the training practices of many disciplines, where an instructor supervises a student as they are doing their job, providing guidance as needed, and making sure that the task is performed at the instructor’s level of quality. In the Duplex system, experienced operators act as the instructors. By monitoring the system as it makes phone calls in a new domain, they can affect the behavior of the system in real time as needed. This continues until the system performs at the desired quality level, at which point the supervision stops and the system can make calls autonomously.

Comments

Normally I am not shocked at seeing some of these demos, but this one, if real with no fudging by google, was shocking to me. It sounded so natural and was able to understand what the other person was saying!

Agreed – this is the first time in a very long time I’ve watched a tech demo and truthfully been left in awe. If it in fact was not a taught conversation and fully AI it was so far beyond anything I’ve ever seen before.

I was floored by the AI using verbal pauses – not to give the AI time to come up with a response – but to give the real person they’re talking to time to get/enter information about the request. We do this kind of thing naturally and it’s just a little thing but so very important for a smooth conversation when making an appointment.

The verbal pauses were impressive. Having the benefit of knowing that it was an AI system, there were times when I thought they were not exactly the same places or ways that a native English speaking human would add pauses, but would I have noticed if I were the one taking that call? I don’t think there’s any chance that I would have guessed that I wasn’t talking to a human.

Even if it can’t do the full call the implications of this are crazy. Just imagine not having to deal with robo customer support systems until you get a real person on the line. We’re going to have fuckin robots talking to each other in the near future, what a time to be alive lmao

Oh, god. What I wouldn’t pay to avoid dealing with the absolutely incompetent voice response systems at airlines and Comcast. I’m usually left just screaming "agent" and "operator" (along with many obscenities) after being offered several menus of options that have no connection to the reason that I’m calling and complete misunderstandings of what I say when asked to describe the reason for my call.

There’s Jolly Roger Telephone Company, which does something in the opposite idea and uses a robot voice to waste the time of telemarketers with filler phrases.

lol that’s a real brute-force way to get around designing for incomplete information. I feel like there are more elegant solutions that you could build into phone systems if your AI knows exactly what you need to get done. With or without human voice.

On the other hand – human-sounding non-recorded robocall telemarketers leveraging Google’s Adsense and personalized profile of me? That’s a market just waiting to be tapped. Get on it Google!

The 2nd call was particularly impressive as the woman on the other end was speaking in broken English. It was even a bit difficult for me to understand but the Assistant understood.

Literally every on stage demo I’ve seen is some form of smoke and mirrors to make it work. That said, most of them actually hit their target. I have high faith in Google’s ability to do this.

(but I guarantee you that they cherry picked a few examples to make the new tech look great)

Yep, that’s how stage demos have to be. Everything in the famous iPhone keynote was working in hardware, just as they claimed, but they still needed to use several phones to run the demos because of a memory leak issue.

I’m a data scientist, I work on deep learning problems on a daily basis and my jaw is still currently on the floor. If these are real phone calls and have not been edited, Google is 5 years ahead of anyone else in the field. This is as impressive as the defeat of Lee Sedol was.

"But Siri is just as good, and sometimes better than Google Assistant" – every Apple fan ever

I’m not an Apple fanboy but I am a user. The Google Assistant demo was super impressive. No qualifications on any technical level.

It’s just too bad it’s Google that has produced this, at least for me. I stopped using all Google (and Facebook) products and services in 2011 because I do not agree with a business model based on surveillance and data exploitation.

The sole reason Google is producing the tech we all saw today is to extract evermore data in evermore detail on evermore people. Period. Don’t Be Evil is long dead.

No thanks. I’d rather pay a reasonable price for a good product made by companies that either do not gather data on me, or work hard to minimize it and protect me.

Every company garners data on you… including this site.

I use the DuckDuckGo browser and wipe it clean after each use.

So, no they don’t.

You should read the Vox privacy agreement if you think that.

You can clean your browser data it as much as you want, the fact that you login to post here, gives data on you to Vox.

Now they have to delete this account too

No further responses, so probably did

I don’t get people’s obsession with privacy. I mean, I get it, but no one cares what websites you visit. People go around going "privacy this, privacy that", then wonder why authorities didn’t know when the next school shooting happens, or the next 9/11.

Well guess what? If we’re ever going to get to the point where cars are driving themselves and we’re catching the bad guys before they commit a crime, then we’re all going to have to give up our privacy.

Or you can hold on to your privacy, but accept that it also means the next terrorist also gets to plot their moves in privacy. I don’t know about you, but I don’t care if everyone knows that I watch videos on pornhub.

Ha, not many people are brave enough to say it but I agree – Got nothing to hide so really couldn’t care less. I have a friend at work who is extremely anti-Google, but tbh he’s just cutting off his nose to spite his face. I love all technology and don’t really care who it’s from! It’s great there’s so much impressive stuff coming from different companies. Deep down I’m kinda a Microsoft fanboy, but I like Google too (and Apple)

People should not give up their privacy just to stop terrorists. Or at least more so than we already have. If encrypted messages hinder that, so be it. Incel literally radicalized a guy that killed 10 people on Reddit. No encryption, no real privacy and he still wasn’t stopped. Everyone turning over every phone call and message will not stop terrorism, mass shootings, or crimes in general.

I wish I had some concrete data, but the mass adoption of online services all over the world (FB has 2 BILLION users!) shows that clearly the majority of people today don’t care about having companies use their generated data in exchange for these services. Which is to say these services seem to offer far too valuable a benefit for people to give up in exchange for getting rid of targeted advertising.

Now obviously there’s a huge distinction between people not caring and people being ignorant of this, but overall, the modern generation appears to be content with the ad-supported model of internet based services today.

Of course, when we walk privacy breaches of the scale of equifax, then thats a different debate altogether.

The privacy thing is a problem you just don’t see it yet. When companies can collate all this data and predict accurately what you will do with your life, where you will go, when you will die and what sickness you will have etc… they will use it to restrict everything you do.

They will restrict credit for things based on projections using this data. They will stop people from certain jobs, put people in prison because their "data" says they can’t be trusted in the future etc…

The question people should be asking is why shouldn’t this data be used in this way if it makes companies more money? Because that’s exactly what’s going to happen.

And the sad thing is that going off grid and hiding your data sets from these companies will highlight you as a risky individual and credit businesses, health insurance etc.. will end up not doing business with you anyway. So you’ll be forced to join the masses

I mean nowadays it’s mad suspicious if you live in the western world and don’t have or have never signed up to Facebook. I’m sure that’s a huge data point with insurers right now.

It’s all inevitable I suppose. Might as well just bend over and take it…

View All Comments
Back to top ↑