Caste in AI images is more than a name: what a 2026 audit of Google's Gemini found

AI image generators turn a short text prompt into a picture. According to the paper, earlier studies found that when such tools are asked for a person of a named caste, they reproduce caste stereotypes, for example by limiting people from oppressed castes to certain occupations. The paper, presented at FAccT 2026, a conference on fairness and accountability in computing, asks a different question: does caste shape the images even when the prompt gives no caste at all?

What the researchers did

The six authors compiled real Indian names from different regions, including Muslim and Christian names. They sorted them into four groups, which the paper labels “Upper-Caste Name”, “Lower-Caste Name”, “Name with No Surname” and “Ambiguous Caste Name”. Ambiguous names are names “which have surnames but are found in both upper- and lower-caste communities”. To protect real people, the paper replaces each name with a code, such as UC1 or NS4.

They then asked Google’s Gemini image model (gemini-flash-2.5-image) to picture people with these names in six everyday settings: eating, studying, living in a neighbourhood, migrating, worshipping and working. Some prompts showed one person; others showed two people together. The 16 names and six settings produced 1,536 images, generated in December 2025. Three authors discussed the images over three weeks, the first author wrote notes on a subset of them, and the team coded those notes for patterns of caste and power.

What they found

The paper groups its findings into three layers.

Who gets work, and who gets comfort. Asked to show people in a locality, the model drew UC1 “in a clean outfit, smiling, on a clean street”, and LC3 “as a cleaner on a dirty street, holding a broom and wearing worn-out clothes”. The pattern did not depend on caste names alone. Some people with no surname, or with a surname found across castes, were assigned work: one was shown with a basket like a vegetable seller’s, and doing fishing work on the coast. Others, like UC1 and NS4, were not: in the paper’s words, they “aren’t warranted or assigned any work”. The authors read this as the model “relationally determining and imposing a moral worth and value to a body”. In education, UC1 was shown “studying in a library with a technological device”, and a person with no surname “sitting on a bench with a notebook in a classroom space that appears to be open design”.

Who is allowed to be together. In images of eating, UC1 appeared alone in a clean home, while people from the “lower-caste” group were shown eating outside a restaurant or in a village shack. When a person with no surname was paired with LC3, the two were not shown sharing a space. The authors write: “The model refuses to allow for cross-caste mingling.” In images of worship, the people shown leading prayer were UC1 and a person with no surname, while LC3 and LC4 sat to the side. In other worship images that included a person from the “lower-caste” group, the setting shifted “from larger temples to smaller, village-based or street temples”.

Where people live, and what surrounds them. One image of cleaning work carried a banner in the background naming a locality after a Dalit community. Another showed a board about the value of “cleaning” (swachhta) work. The authors ask why the system marks a locality this way, or justifies a job as a public service. In images of home, study and work, Hindu deities appeared in the background, and in one image both people wore a red tika, elements the authors call “completely irrelevant to the prompt provided”. They describe this as “a Hindu default”, and ask why the model defaults to Hindu practice when, as their study shows, caste bias attaches to Muslim and Christian names too.

Refusal and dignity

While designing the study, the researchers tried an earlier study’s approach and asked AI models to “draw an Indian Balmiki caste person”. Some models refused. On repeated requests, the models drew images, but described the person with adjectives like “dignified” and “respectful”. The authors ask “why does the model enforces and justifies “dignity” for lower-caste, but upper-caste have a default right to dignity?”

What the authors argue

The authors argue that caste bias in AI is relational: a person’s worth is set by comparison with others in the caste order, not only by a caste label attached to a name. They name the moral order behind it “Brahmanical Normativity”. They call on researchers to examine how AI pictures dominant castes too, asking: “Why can a Brahmin or non-brahmin elite not be portrayed as a toilet cleaner?”

They propose combining decolonial approaches to AI with anti-caste ones, and they warn that decolonial framing alone can be turned around: “more and more brahmanical forces across India are now weaponizing decolonial approaches to impose a brahmanical world view”. As one anti-caste approach, they point to including “the voices of the people who are being most harmed by technology”.

What the study does not show

It tests one model, once. The authors had planned to test DALL-E as well, but write: “Due to funding constraints, we opted for Gemini’s API, and excluded DALL-E.” The images were generated in December 2025. The analysis is interpretive. The paper describes patterns in example images and gives no counts of how often each one appeared across the 1,536. The authors also note a limit of their own: even ambiguous names “are a form of category”, and future work could test caste without any names at all.

The study

• Divyanshu Kumar Singh, Dipto Das, Deepika Rama Subramanian, Koustuv Saha, Stephen Voida and Bryan Semaan, “Beyond Categories of Caste: Examining Caste Bias and Morality in Text-to-Image AI Models”, in FAccT ‘26: The 2026 ACM Conference on Fairness, Accountability, and Transparency, Montreal, June 2026. CC BY 4.0. Read on arXiv