Smartphones let you control a lot using only your voice. Whether you're on an iPhone or an Android phone, you can ask for directions, perform a search, start a call, or dictate a message. But once you actually open up an app, voice control disappears: for the most part, apps all still work exclusively through touch.
Mix analyzes speech and translates it into an action
Nuance wants to change that. Nuance is the company behind the language detection software that powers Samsung's S Voice and, supposedly, Siri as well. It's now opening up that technology to all developers — big and small — allowing them to bake it into their apps.
It all works through a new tool, called Nuance Mix, that's supposed to give developers an easy way to build full voice control into their apps. For the most part, developers just have to tell Mix what kind of commands their users will give. Then when the app hears those commands, Nuance will be able to convert them into the specific actions that a person wants to perform and send that information back down to the app.
The idea is that someone could say to a social media app, "Update my status," "Post a new status," "Change my status," or even "I want to post something," and Nuance's technology will be able to figure out that they're all the same command. That's partly because developers are required to prime Mix with 15 to 20 samples of every command, but it's also because Mix will then generate alternatives on its own and continue to learn based on what people using the app actually end up saying.
Hardware is what makes Mix really interesting
Nuance Mix isn't limited to apps: it can be used on hardware, too, and that's where it really starts to get interesting. Mix could be used as the primary way for interacting with smart home products or Internet of Things devices. "A lot of these devices don't necessarily have a screen," says Kenn Harper, a senior director at Nuance. "So a thermostat may have a tiny screen or a couple of buttons, but if you want to tell if you're traveling for a couple of days ... how would you do that without speech?"
Right now, most smart home devices are controlled through smartphones, and it seems like that's unlikely to change. But Harper's vision is appealing. Why shouldn't you be able to dictate commands directly to a thermostat if you're in the same room as it? That makes even more sense as you start to consider interactions with devices outside your home, which won't be connected to your smartphone.
Can Nuance defeat existing voice assistants?
While Nuance is offering language processing here and now, it's worth questioning whether it'll really be the one to dominate voice control inside of apps and smart home devices in the future. Apple and Google could always open up Siri and Google Now directly, allowing their native tools to be used inside of apps, which may end up being easier for developers. Both Apple and Google are behind smart home platforms as well, and one could imagine their voice assistants extending there, too. (Apple is believed to use Nuance's technology at the moment, but that may not always be the case.)
Nuance Mix is launching today in beta. That'll give developers a chance to try it out, though most are going to want to wait before building it into anything. Nuance isn't disclosing pricing yet, but it's likely that many — if not most or all developers — will have to pay for Mix once it hits an official release. Pricing options are "not going to be a one size fits all" approach, however, so small developers will theoretically be able to pay less than big ones.