Google Live Transcribe could be a big help for people who are deaf or hard of hearing

Google is announcing two new features for Android phones today: Live Transcribe and Sound Amplifier. They’re both designed as accessibility features to help people who are deaf or hard of hearing.

Live Transcribe is an app that automatically transcribes speech in near-real time, allowing people to communicate in situations where they might not otherwise be able to. Sound Amplifier is designed for those with hearing loss, allowing you to tweak sound settings to improve your ability to hear — sort of like a hearing aid app for your phone. Sound Amplifier was originally announced back in May at Google I/O, but it’s finally shipping now.

Both will be available preinstalled on Pixel 3 phones in Accessibility Settings and also available via the Google Play Store for other phones. However, Live Transcribe is being released as a limited beta, and Sound Amplifier requires phones running Android 9 Pie, which limits their availability.

Of the two, I was more impressed with Live Transcribe. When you open the app, it simply starts writing out what it hears in large, easy-to-read text. It works remarkably well, including adding punctuation and even understanding some context. The example Google showed us is that it was able to correctly capitalize the phrase “I am buying a new jersey in New Jersey.” Unfortunately, it does require an internet connection to work.

Although it’s pretty straightforward, Live Transcribe does have some subtle features that make it a little easier to use in the moment. A blue circle in the corner pulses slightly to show the ambient noise level so that you can visually see if you need to move the microphone closer to the speaker for it to work. You can also hit a small button to bring up a keyboard to type out replies if you need to. Finally, if somebody starts speaking after a period of silence, Live Transcribe will vibrate the phone to let you know to look at it to see what is being said.

You can set a primary and secondary language, toggling between them with a quick tap — though, in my testing, Live Transcribe was able to correctly transcribe short phrases in another language without switching. It supports up to 70 different primary languages, and it’s relatively easy to switch between a dark and light theme and also change the text size. In Android, you can create a button for Accessibility that appears in the main button bar, giving quick access to settings and apps like Live Transcribe.

Google says that keeping it simple and easy-to-use was a major goal and one of the reasons it chose not to include the option to save transcriptions. I imagine privacy concerns may also have factored in. Google noted that, in addition to not saving transcripts, it’s also not storing audio or transcriptions on its servers, nor is it using any of that data to improve its algorithms.

The other feature Google is introducing is called Sound Amplifier. Inside Accessibility Settings, it’s a new option that takes you to a screen with a bunch of sliders for adjusting various sound settings. You plug in a pair of wired headphones (yes, it only works with wired headphones), and then the phone processes incoming audio to make it easier to hear.

Apple has a similar feature that works with AirPods, which would probably be more convenient because you can more easily set the phone next to the speaker since there are no wires involved. iPhones can also work directly with some hearing aids, allowing you to adjust settings on the phone.

Although there are just a few sliders — Boost, Fine tuning, Mic volume, and Sound reduction intensity, along with adjustments for left and right ears — Google says that behind them are “thousands of parameters.” It works locally, without the need for an internet connection

I am not hard of hearing (beyond the usual for a 40-year-old who used to go to rock concerts), so I had a difficult time adjusting the sliders to really dial in major improvements. But adjusting the various sliders did significantly change the sound profile of what I was hearing. You also have the option of using an external microphone if you need, and Google says that this feature will be available to developers to build into their own apps via the Dynamic Processing Effect API.

Google says that it hasn’t ruled out creating apps for iOS, but it’s focused on Android for now. Sound Amplifier should be available in the Play Store for Android today, and people who want to try out Live Transcribe will need to sign up to be notified when it’s more widely available.

Correction: the original version of this piece mischaracterized support for short phrases in other languages as “translate” instead of transcribe. We regret the error.

Comments

It could be helpful if they improve their recognition. I assume this is the same process used for the call screening and voicemail transcription which is a great thing to have but has been far from perfect. In some cases the misrecognized words change the meaning of the message (such is the nature of language).

This is awesome! I have some hearing loss, so use subtitles on everything to help me out. Having that option everywhere now will give me more choices to engage with other people. Very happy to see this released!

Ok, I tried it, and it needs more work. It barely picked up any speech from videos, and what it did was then broken because of all the missing or misheard words. In conversation, it misheard a lot of what we talked about, maybe an accent thing?

Great idea, and good starting place, but it’s going to need more work to be functionally useful.

If it’s anything like the YouTube auto captions it’ll be shite. If it’s better than whatever tech they use there maybe it’d be an idea to use it throughout their products

They really need to start using the local processing power of phones to run these stuff offline.

Wow, this has such quick access. Just appears as a little stick man on the navigation bar. Always there. Awesome.

Why not allow the captured session to be locally saved, or sent out as an email? It looks like a potential rival to the Nuance speech to text tools.

Nuance Dragon is more about transcribing your own speech.

This one is more about transcribing other people’s speeches – the other people may not want their speeches to be on the record just by conversing with you using this app out of necessity. Although the phone can still screen-capture the transcription or outright sound-record the speech itself, guess they figure this is a step too far.

(Ex: some people may lie and say they need this app out of necessity in order to converse with you, and then record the transcription of your speech without your permission for whatever sly reasons.)

View All Comments
Back to top ↑