Just like every other tech company that got caught with its hand in the cookie jar this year — hey there, Amazon, Apple, Google, and Facebook — we recently learned that Microsoft had been quietly letting human contractors listen to your Skype translations and Cortana voice recordings. That’s right: they’re not just AI.
Our processing of personal data for these purposes includes both automated and manual (human) methods of processing. Our automated methods often are related to and supported by our manual methods.
To build, train, and improve the accuracy of our automated methods of processing (including AI), we manually review some of the predictions and inferences produced by the automated methods against the underlying data from which the predictions and inferences were made. For example, we manually review short snippets of a small sampling of voice data we have taken steps to de-identify to improve our speech services, such as recognition and translation.
When you talk to Cortana or other apps that use Microsoft speech services, Microsoft stores a copy of your audio recordings (i.e., voice data) [...] This may include transcription of audio recordings by Microsoft employees and vendors, subject to procedures designed to prioritize users’ privacy, including taking steps to de-identify data, requiring non-disclosure agreements with vendors and their employees, and requiring that vendors meet the high privacy standards set out in European law and elsewhere.
It’s true that systems built using machine learning, like a majority of modern voice recognition and natural language processing ones, generally need to be audited by humans in order to improve — it’s not clear how a machine would tell a false positive unless a human points it out, annotates the data, and feeds it back into the system. And to Microsoft’s credit, it offers a privacy dashboard where you can retroactively delete your voice data.
(Also, Cortana seems like it’s on the outs.)
But the scandal, with all of these tech companies, was that they didn’t think to make it clear that humans (read: outsourced contractors) would be listening to extremely personal details like people speaking their exact street address, confidential medical info, or sex noises into a voice assistant’s microphone — and let us proactively opt out, if we decide that’s something we don’t want to bring into our homes.
Apple says it’ll have a future update that lets its customers opt out. Will other companies do the same?