Skip to main content

The Coast Guard wants to fight prank calls with voice recognition

The Coast Guard wants to fight prank calls with voice recognition


A new technology for an old problem

Share this story

Donald Trump Hosts Chinese President Xi Jinping At Mar-A-Lago
Photo by Joe Raedle/Getty Images

The National Coast Guard is facing a surprising threat: prank calls. The agency has been flooded with fake distress calls in recent months, sometimes as often as every day. Since each call is a potential life at risk, the Coast Guard has a duty to respond. Depending on available resources, that could mean scrambling a C-130 aircraft or helicopter, sometimes even diverting airspace from nearby airports — all for a bad joke.

“We’re just getting more hoaxes every day,” says Lt. Gianfranco Palomba, who’s been tasked with stopping the calls. “We’re seeing a direct impact, not just on man-hours but on assets.”

“We’re just getting more hoaxes every day.”

The Coast Guard believes there are a handful of hoaxers making as many as 10 calls apiece — but tracking them down is harder than you might think. The calls typically come in through the Coast Guard’s VHF radio channel, the maritime equivalent of 911, picked up by a network of antennas along the coast. Unlike 911, there’s no identifying information like a phone number to trace back a given caller. A dense enough network would let you triangulate the source of a call, by checking signal strength, but with more than 50 miles between some stations, the Coast Guard’s VHF antennas aren’t nearly dense enough. Palomba is also looking for ways to spot hoaxers on social media, but says so far, the method has turned up few hard leads.

Instead, Palomba and his team have turned to something more exotic: voice recognition. The calls themselves give Coast Guard agents a clear sample of the subject’s voice, even if all they’re saying is “Mayday.” That might not be enough to find out who’s calling, and Palomba acknowledges it probably wouldn’t be enough to serve a search warrant — but it could be a way to match the same caller the next time they phone in a hoax. “Even in situations where someone has called in a hoax on 10 different occasions, that affected unit is still having to respond to them,” Palomba says.

Forty seconds to build a voiceprint

Historically, voice recognition has often been overlooked as a biometric in favor of fingerprinting and facial recognition, leaving lots of research but few off-the-shelf solutions for the Coast Guard to use. Automated voice recognition systems have more often focused on translating speech to text, from early experiments at Bell Labs and DARPA through modern projects like Siri and Cortana. Identification of voices, meanwhile, has historically been done by humans. Forensic analysts at the FBI and US intelligence agencies can study audio recordings for clues, determining how old a speaker is or the kind of room they’re calling from. But distinguishing one voice from another is a more subtle art, and researchers are only recently learning how to automate it.

Some of that technology is already in place at large call centers, which use it to fend off fraud. A company called Nuance — which currently operates voice biometric systems for Barclays, HSBC and others — can build a voice print from only 40 seconds of speech. Within government, Citizenship and Immigration Services is looking for a similar technology to employ in its own call centers, as a way of verifying callers’ identities. Similar algorithms have also found their way into consumer devices: Last week, Google unveiled a similar function for Google Home, allowing the device to identify who’s speaking based on a locally stored voiceprint. At the same time, researchers have also developed advanced voice-spoofing software, allowing sophisticated hoaxers speak with someone else’s voice.

“They instinctively attempt to hide their identity”

It’s still unclear whether those systems can help the Coast Guard. Forty seconds is plenty of time for banking fraud, but a fake Mayday signal could be much shorter. Call centers have another advantage: for whatever reason, banking fraudsters tend to use their own voice. “We all speak differently, not just because of the size and shape of our larynx, but because of where we grew up, because of our personality,” Nuance engineer Brett Beranek says. “That makes voice biometrics a little more complex, but that complexity gives us a lot more data to play with.” Since banking fraud calls often involve complex withdrawal schemes, the callers typically don’t bother maintaining a different voice throughout the process.

But when someone calls into the Coast Guard, that kind of deception is much easier. CMU professor Rita Singh has been examining the Coast Guard’s hoax calls since 2014, as part of a separate research partnership. While Singh can reliably tell the caller’s gender and age, it can be difficult to go farther. At the same time, many of the callers seem to naturally slip into an alternate voice, making things much harder for any automated voice-readers. “Although probably not aware of the biometric potential of their voice, they instinctively attempt to hide their identity by disguising it,” Singh wrote in a recent paper. “They try to sound like a real (albeit fictitious) person other than themselves.”

In the meantime, the Coast Guard is still looking for some edge on the prank callers. Ideally, they’d be able to build something that would identify any voice previously used to make a hoax call, which would let responders limit the damage from hoax calls even without catching anyone. But the agency hasn’t yet found a commercial system that will meet that standard, and is still looking for recommendations and best practices from academia and other agencies in government. In the meantime, Palomba is looking towards a less technical fix — a high-profile arrest and trial to remind everyone of the consequences of fake reports. “People still see that wiggle room, just because they haven’t been reminded of a really big prosecution lately,” Palomba says. “We’re looking for that deterrent effect.”