AI-generated sexism and “the synthetic mirror”

Clarote & AI4Media / Better Images of AI / User/Chimera / CC-BY 4.0

When Melissa Heikkilä, a Senior Tech Reporter for MIT Tech review, image prompted the AI avatar app Lensa, she encountered distinctive results from her male counterparts. While the men received digital portraits as explorers or inventors, her generated images were ‘cartoonishly pornified.’ Generative visual AI models have a reoccurring history of bias and often extremely stereotypical or dangerous training. Stable Diffusion powers Lensa and thousands of other image generators and was built using LAION-5B, an enormous open-source data set. LAION-5B, like all generative trainer data sets, was constructed by amassing images from the internet. This points to a larger online crisis surrounding the proliferation of biases involving sexist and racist stereotypes. Heikkilä notes that while not all image generative models use open-sourcing data (e.g. Google’s Imagen and Open AI’s DALL-E), “they are built in a similar way”.

What efforts can be made to detoxify the data fueling generative AI? The outstanding consideration is how LAION and other training sets hold up a mirror to the world and our unspoken biases. Melanie Mitchell, a Professor at the Santa Fe Institute, charged that the associations made from this data within generative models are concerning as much as they are sweepingly generic. Quoted in Bloomberg, “When they start then generating new language, they rely on those associations to generate the language, which itself can be biased in racist, sexist and other ways.”

This last month, however, Google Gemini hinted at the concerns of overcorrecting for these biases. According to Wired, Gemini produced historically inaccurate images of ‘excessive’ representation, such as “exclusively Black people in traditional Viking garb” or “Indigenous people in colonial outfits.” Google has paused Gemini’s people generation feature until they could correct this issue. Jack Krawczyk, a senior director at Gemini, posted on X: “We design our image generation capabilities to reflect our global user base, and we take representation and bias seriously. We will continue to do this for open-ended prompts (images of a person walking a dog are universal!) Historical contexts have more nuance to them and we will further tune to accommodate that.” Gemini’s data training corrections, while shown to exhibit inaccuracies, is one approach to a more significant representational conflict of internet image training data. In the coming months, this conversation will indeed unfold into further efforts to find a curated picture that represents humanity and to remove our ugliest, deep-seated projections.

Shanon Vallor, a philosopher of technology at the Edinburgh Futures Institute and our most recent podcast guest, coined The AI Mirror as her recent book title, advancing the idea that AI’s potential is marred by its influence of the past in making associations from existing data. “…today’s powerful AI technologies reproduce the past. Forged from oceans of our data into immensely powerful but flawed mirrors, they reflect the same errors, biases, and failures of wisdom that we strive to escape.” While generative AI (and other robust emerging systems) offers an extraordinary recipe to enrich human flourishing, we must also take ownership of its ingredients. Challenging open-source as the best available approach for development is one step under which compelling research continues. While we are predisposed to the internet data we have, increasing the public conversation online to take ownership of our content will allow us to turn the tide.

Further listening + reading:

originally published in February 2024 under the University of Cambridge newsletter “The Good Robot Podcast”