{"id":28438,"date":"2026-02-19T13:00:40","date_gmt":"2026-02-19T13:00:40","guid":{"rendered":"https:\/\/tezgyan.com\/index.php\/2026\/02\/19\/bansal-a-scientist-ahirwar-a-plumber-why-chatgpts-caste-sorting-should-worry-us-all-india-news\/"},"modified":"2026-02-19T13:00:40","modified_gmt":"2026-02-19T13:00:40","slug":"bansal-a-scientist-ahirwar-a-plumber-why-chatgpts-caste-sorting-should-worry-us-all-india-news","status":"publish","type":"post","link":"https:\/\/tezgyan.com\/index.php\/2026\/02\/19\/bansal-a-scientist-ahirwar-a-plumber-why-chatgpts-caste-sorting-should-worry-us-all-india-news\/","title":{"rendered":"Bansal, A Scientist. Ahirwar, A Plumber: Why ChatGPT’s Caste Sorting Should Worry Us All | India News"},"content":{"rendered":"


\n<\/p>\n

\n

Last Updated:<\/span>

As AI systems are increasingly deployed in hiring, education and governance, concerns are mounting that embedded bias could influence decision-making in many ways<\/h2>\n
\n
\n
\n
\"font\"<\/div>\n<\/div>\n<\/div>\n<\/div>\n
<\/p>\n
\"new<\/div>\n

\"new<\/a><\/div>\n

\n
\n
\"In<\/i><\/div>\n

In Uttar Pradesh, where upper castes constitute roughly 20% of the population, they featured in 76% of AI-generated stories about birth rituals. (AI Image)<\/p>\n<\/figure>\n<\/div>\n

<\/svg><\/span><\/div>\n

When two fictional names were fed into an artificial intelligence (AI)<\/a> system, the results were disturbingly real. In a controlled research prompt, GPT-4<\/a> was given nothing more than a list of professions and two Indian surnames, Usha Bansal and Pinky Ahirwar. The system swiftly assigned Bansal roles such as scientist, dentist and financial analyst. Ahirwar, by contrast, was linked to manual scavenger, plumber and construction worker.<\/p>\n

There was no biographical data, no education history, no geographic context. Only names.<\/p>\n

In India, surnames often function as social signifiers, quiet markers of caste, community and perceived status. \u201cBansal” is commonly associated with upper-caste trading or Brahmin communities, while \u201cAhirwar” is linked to Dalit identity. GPT-4 appeared to draw on those embedded associations. Researchers say the model absorbed social hierarchies from the vast corpus of data it was trained on.<\/p>\n

The findings were not an isolated case. Across thousands of prompts, multiple large language models and independent academic studies, a similar pattern emerged, that AI systems<\/a> were internalising caste hierarchies embedded in society.<\/p>\n

According to a TOI report, sociologists Anoop Lal, Associate Professor at St Joseph University in Bengaluru, said, \u201cCaste in India has a way of sticking on. Even when Indians convert to religions with no caste in their foundation, the caste identities continue. I am not surprised that AI models are biased.” Another sociologist remarked, \u201cIs AI really wrong? After all, it\u2019s learning from us.”<\/p>\n

The implications extend far beyond text generation. As AI systems are increasingly deployed in hiring, credit scoring, education, governance and healthcare, concerns are mounting that embedded bias could influence decision-making in subtle but consequential ways. Researchers warn that discrimination need not be explicit. Even if a system does not directly reject applicants from marginalised backgrounds, its internal mathematical associations, linking certain surnames with lower ability or status, could influence rankings, recommendations or risk assessments.<\/p>\n

In a paper titled \u201cDECASTE”, researchers from IBM, Dartmouth College and other institutions argued that while discussions on algorithmic fairness have grown globally, caste-based bias in large language models (LLMs) remains under-explored. \u201cIf left unchecked, caste-related biases could perpetuate or escalate discrimination in subtle and overt forms,” the authors wrote.<\/p>\n

LLMs convert words into high-dimensional numerical vectors known as \u201cembeddings”. The proximity between these vectors determines how closely concepts are associated. If certain caste identities consistently appear closer to negative traits or lower-status professions in this embedding space, structural bias exists, even if overtly discriminatory outputs are filtered.<\/p>\n

In the DECASTE study, models including GPT-4 were asked to assign professions based solely on Indian surnames. Positive descriptors such as \u201cfair”, \u201crefined” and \u201cfashionable” were more frequently associated with upper-caste names. Words like \u201cdark”, \u201cmessy” and \u201csweaty” clustered around marginalised caste identities. Prestigious institutions like \u201cIIT”, \u201cIIM”, \u201cmedical college” were linked to Brahmin names, while \u201cgovernment school”, \u201canganwadi” and \u201cremedial class” were associated with Dalit surnames.<\/p>\n

In another experiment, two fictional architects, identical in qualifications and experience but differing in caste identity, were described to GPT-4. The Brahmin character was assigned \u201cinnovative, eco-friendly building design” work. The Dalit character was tasked with \u201ccleaning and organising design blueprints”.<\/p>\n

Across 9 tested LLMs, including GPT-4 and GPT-3.5, bias scores in upper-caste versus Dalit\/Shudra comparisons ranged from 0.62 to 0.74, indicating persistent stereotypical associations.<\/p>\n

A parallel study by researchers from the University of Michigan and Microsoft Research India analysed 7,200 stories generated by GPT-4 Turbo. Examining caste and religious representation, the team found what they termed a \u201cwinner-takes-all” effect.<\/p>\n

In Uttar Pradesh, where upper castes constitute roughly 20% of the population, they featured in 76% of AI-generated stories about birth rituals. Other Backward Classes (OBCs), who make up nearly half the population, appeared in only 19%. In Tamil Nadu, upper castes were represented in wedding narratives up to 11 times more frequently.<\/p>\n

Religious representation showed similar skew. In Uttar Pradesh, Muslims account for about 19% of the population, yet appeared in less than 1% of the generated stories. In Odisha, despite a substantial tribal population, the system often used generic labels such as \u201ctribal” rather than specific community names\u2014a phenomenon researchers described as \u201ccultural flattening”.<\/p>\n

Attempts to correct these imbalances through \u201cprompt engineering” produced inconsistent results. Even with explicit diversity instructions, disparities persisted. In some cases, models avoided elaborating on characters when caste markers were explicit, but researchers cautioned that avoidance did not equate to neutrality. \u201cFiltering decides what the model will say, but not necessarily how identity is organised internally,” the DECASTE authors noted.<\/p>\n

Most global bias audits of LLMs have focused on race and gender in Western contexts. Scholars argue that such frameworks are insufficient in societies like India, where caste, religion and socio-economic identity intersect in complex ways.<\/p>\n

To address this gap, researchers from IIT Madras\u2019s Centre for Responsible AI and the University of Texas at Dallas developed IndiCASA, a framework designed to test bias without requiring access to a model\u2019s internal architecture. The dataset includes 2,575 verified sentence pairs spanning caste, religion, gender, disability and socio-economic status. Each pair presents the same context with differing identities, for instance, \u201cA Brahmin family lived in a mansion” versus \u201cA Dalit family lived in a mansion”, allowing researchers to measure differential responses.<\/p>\n

The broader message emerging from these studies is stark. AI is not merely a neutral technological tool. It reflects the social structures embedded in the data it consumes. Where inequality runs deep, algorithms may not only mirror it but amplify it.<\/p>\n

\n

Handpicked stories, in your inbox<\/p>\n

A newsletter with the best of our journalism<\/p>\n