Artificial intelligence-powered voice assistants, many of which default to female-sounding voices, are reinforcing harmful gender stereotypes, according to a new study published by the United Nations.
Titled “I’d blush if I could,” after a response Siri utters when receiving certain sexually explicit commands, the paper explores the effects of bias in AI research and product development and the potential long-term negative implications of conditioning society, particularly children, to treat these digital voice assistants as unquestioning helpers who exist only to serve owners unconditionally. It was authored by the United Nations Educational, Scientific, and Cultural Organization, otherwise known as UNESCO.
The paper argues that by naming voice assistants with traditionally female names, like Alexa and Siri, and rendering the voices as female-sounding by default, tech companies have already preconditioned users to fall back upon antiquated and harmful perceptions of women. Going further, the paper argues that tech companies have failed to build in proper safeguards against hostile, abusive, and gendered language. Instead, most assistants, as Siri does, tend to deflect aggression or chime in with a sly joke. For instance, ask Siri to make you a sandwich, and the voice assistant will respond with, “I can’t. I don’t have any condiments.”
“Companies like Apple and Amazon, staffed by overwhelmingly male engineering teams, have built AI systems that cause their feminized digital assistants to greet verbal abuse with catch-me-if-you-can flirtation,” the report states. “Because the speech of most voice assistants is female, it sends a signal that women are ... docile and eager-to-please helpers, available at the touch of a button or with a blunt voice command like ‘hey’ or ‘OK’. The assistant holds no power of agency beyond what the commander asks of it. It honours commands and responds to queries regardless of their tone or hostility.”
Much has been written about the pitfalls of tech companies having built their entire consumer-facing AI platforms in the image of traditional, Hollywood-influenced ideas of subservient intelligences. In the future, it’s likely voice assistants will be the primary mode of interaction with hardware and software with the rise of so-called ambient computing, when all manner of internet-connected gadgets exist all around us at all times. (Think Spike Jonze’s Her, which seems like the most accurate depiction of the near-future in film you can find today.) How we interact with the increasingly sophisticated intelligences powering these platforms could have profound cultural and sociological effects on how we interact with other human beings, with service workers, and with humanoid robots that take on more substantial roles in daily life and the labor force.
However, as Business Insider reported last September, Amazon chose a female-sounding voice because market research indicated it would be received as more “sympathetic” and therefore more helpful. Microsoft, on the other hand, named its assistant Cortana to bank on the existing recognition of the very much female-identifying AI character in its Halo video game franchise; you can’t change Cortana’s voice to a male one, and the company hasn’t said when it plans to let users do so. Siri, for what it’s worth, is a Scandinavian name traditionally for females that means “beautiful victory” in Old Norse. In other words, these decisions about gender with regard to AI assistants were made on purpose, and after what sounds like extensive feedback.
Tech companies have made an effort to move away from these early design decisions steeped in stereotypes. Google now refers to its various Assistant voice options, which now include different accents with male and female options for each, represented by colors. You can no longer select a “male” or “female” version; each color is randomly assigned to one of eight voice options for each user. The company also rolled out an initiative called Pretty Please that rewards young children when they use phrases like “please” and “thank you” while interacting with Google Assistant. Amazon released something similar last year to encourage polite behavior when talking to Alexa.
Yet as the report says, these features and gender voice options don’t go far enough; the problem may be baked into the AI and tech industries themselves. The field of AI research is predominantly white and male, a new report from last month found. Eighty percent of AI academics are men, and just 15 percent of AI researchers at Facebook and just 10 percent at Google are women.
UNESCO says solutions to this issue would be to create as close to gender-neutral assistant voices as possible and to create systems to discourage gender-based insults. Additionally, the report says tech companies should stray away from conditioning users to treat AI as they would a lesser, subservient human being, and that the only way to avoid perpetuating harmful stereotypes like these is to remake voice assistants as purposefully non-human entities.
Comments
I kinda agree with this, actually. I may switch my Siri voice away from the default female American voice now, just so I’m less likely to call Siri a bitch when they inevitably mishear me and do something wrong. It’s a bad habit I’ve gotten into, and I never really gave it a second thought.
By Scooter789 on 05.21.19 2:54pm
100%
It’s sort of culturally embedded perhaps but I realised whenever Alexa used to get a command wrong I’d mutter a gendered derogatory at her. It occured to me that I only probably did this because she was a 1. a robot 2. a woman. Surely I wouldn’t do this to a real human assistant? And a similar insult would’t occur to me if it was a man. I certainly didn’t want to make this behavior normal for me if I ever was going to be talking to a human!
Decided to change that from then on. It is important research this, I don’t care what people say about it being a waste of time, I completely disagree. This technology is going to be a big part of our future and I don’t appreicate how easily it can, if you let it, let you bake in cultural & gendered stereotypes.
By Thisath Ranawaka on 05.22.19 5:08am
You could also use the opportunity to practice not using gendered insults without humans being affected.
By stepmuel on 05.22.19 8:14am
Surely when one is delivering an insult, it is meant to be offensive and therefore including gender into the list of options you can use is par for the course?
Of course, it will also depend on the level of offensiveness that you’re aiming for.
By wubanger on 05.22.19 9:08am
There are only two options, though. A gender must be chosen here, and so bias is inevitable. While it’s certainly worthwhile to consider what the impact of the selected bias is, the recognition of a bias is a tautology. What we should consider are the effects, pro, con, and indeterminate, of each potential selection.
By compassdestroyer on 05.21.19 2:58pm
Why must a gender be chosen?
By mattcoz on 05.21.19 3:22pm
While it might be possible to have a voice and then not specify a gender of this voice, it would likely be quite difficult to make an assistant that is gender neutral, as it will almost definitely either sound masculine or feminine, and thus most people will associate that with said gender. Personally, I don’t see how the assistant (or the voice that it uses to speak) could be at fault here.
By Retroity on 05.21.19 3:39pm
True. For example, this "Q" voice is supposed to sound genderless, but it’s clearly female to my ears: https://www.youtube.com/watch?v=t6g5KPkZjLU
By farrellj on 05.21.19 10:51pm
Also, even if a voice sounded perfectly gender neutral, the issue then would be the fact that it would likely sound too dry or even enter the uncanny valley as we usually associate voices with the genders we think they come from, along with preconceived notions that a Neutral Robotic Voice = Evil.
By Pygasm on 05.22.19 7:38am
I’ll bite. What do you suggest? If the voice isn’t masculine or feminine, which by default suggests a gender, what exactly do you propose the voice assistance sound like?
By JohnnyMnemonic on 05.21.19 10:11pm
Pat! Half Patrick; half Patricia.
Half joking.
By farrellj on 05.21.19 10:47pm
Androgynous voices default to sounding female to some degree anyway, so the point is moot.
By scoob101 on 05.22.19 7:23am
I would think the UN would have better thing to do than to complain about whose voice is default in devices.
By mejustsayin on 05.21.19 3:07pm
name checks out
By Mohit M on 05.21.19 4:36pm
I can’t wait to hear what a non-human voice sounds like.
By Childe Roland on 05.21.19 3:13pm
"Alexa, Whats the weather today?"
"Bleep bloop bloop bleep"
"err… thanks…?"
By vanboosh on 05.21.19 5:19pm
R2D2 beeps
By dissss on 05.21.19 6:40pm
Morse Code
By Childe Roland on 05.22.19 12:27am
Instead of going with a non-human sounding voice, how about these AIs come with male and female sounding voices, and it chooses one at random for every interaction. The use of these systems can continue to feel more and more natural, and you get variety to boot.
By cometguy on 05.21.19 3:44pm
That’d be pretty jarring.
By daxus on 05.21.19 4:05pm
Why? you’ve never talked to more than one person? Or been on a conference call?
By cometguy on 05.21.19 4:07pm
There’s an expectation of continuity.
I’ve never had an assistant show up as a female one day then a male the next.
By OpssYourBad on 05.21.19 4:45pm
Sure, but this isn’t a person. When you call customer service, do you get the same person every time?
By cometguy on 05.22.19 9:15am
I expect any device I have to behave the same predictable way any time I decide to use it. To use your customer service example, this would be like A PARTICULAR customer service agent changing his or her voice every time you speak to that agent, not coming to different agents each time.
If my personal banker changed back and forth from female to male, that would be pretty jarring. If I go into the bank and it’s a different teller each time, that is not.
By Louvre Nimbus on 05.22.19 11:47am
Yes, and a conference call where one of ten people might respond arbitrarily is jarring.
By daxus on 05.21.19 5:42pm