How to train your AI: Uncovering and understanding bias in AI algorithms
Machine learning relies on “training” an algorithm on texts and then producing their intended output, which vary from algorithm to algorithm. Professor James Zou notes that a problem arises when the texts used to train artificial intelligence (AI) systems contain racial and gender biases. In a recent gathering of the Clayman Faculty Fellows, Zou presented his work utilizing this flaw in AI to study bias in texts.
Zou and his team “audit” AI systems to understand the potential biases embedded within these machine learning algorithms. He does this by systematically giving AI systems puzzles to solve. AI systems are about matching, using semantic parallels to create geometric parallels and form vectors. He compared it to SAT analogies. He gives the example Woman: Queen, Man: King, with the AI algorithm providing the final term. Here were some examples of how he found gender biases to be embedded in the matching algorithms.
He: Brother, She: Sister
He: Blue, She: Pink
He: Doctor, She: Nurse
She: Pregnancy, He: kidney stones
He: computer programmer, She: homemaker
That last one got a (virtual) gasp from the audience, and the point was driven home: it really matters how these algorithms are utilized. One could imagine a software program recommending future careers to young adults and having some problematic results.
It might be reassuring to know that there is a way to reduce the bias in stereotyping within AI. Zou uncovers the geometric direction of the stereotypes (cool, right?) and subtracts it from the individual words. Stereotypes can’t be completely eliminated, but it is a simple and effective way of reducing bias in AI algorithms.
Zou then shifted to discuss how it is possible to take this “bug” in machine learning and use it as a feature to reveal biases and how they evolve over time. If you limit the sample of text the AI is trained on by decade, you can see trends in bias. He showed how the most prevalent adjectives describing Asians changed from “envious, barbaric, aggressive” in the 1910s to “inhibited, passive” in the 1990s. He noted that this tracks with Asian-American sentiment, and gives an insight into the shift in anti-Asian stereotypes.
He also looked at gender, noting that the most common adjectives for women went from “charming, placid, delicate” in 1910 to “emotional, protective, attractive” in the 1990s, becoming more physical and emotional. Zou points to the utility of this process - it is able to quantify human stereotypes, especially across time or in time periods where it’s difficult to obtain other measures.
He ended the talk by discussing the implications. Currently, companies scramble to find data on which to train their AI, often purchasing highly-prized data that is particularly specialized to their intended audience. However, specialized data sets mean an even greater likelihood of biased results (often by design). If one data set is better for white college-aged males, the training text will look different than the data for middle-aged Black women. He states that the overall quality of service experienced by the individual customer actually decreases over time when these types of data are used to train AI.
So what should we use for training? Dr. Zou said there aren’t texts that are devoid of bias. Girls and pink are embedded into our modern lexicon. However, specialized and demographic-specific data seem to be worse at embedding stereotypes into algorithms. The talk surfaced questions that left me wondering what kinds of texts our society would need to produce to have He: computer programmer lead to She: computer programmer.
Zou is a professor in biomedical data science as well as in computer science and electrical engineering. His group works on developing different machine learning AI systems, especially motivated by questions about human disease and human health.
