How language bias persists in scientific publishing despite AI tools
For now, English remains the lingua franca of indexed science, dominating most peer-reviewed journals and international conferences. This puts non-native speakers at a significant disadvantage. While large language models can assist authors to help overcome some language barriers, a new study by two researchers with the 海角乱伦社区 Graduate School of Education suggests that bias against non-native speakers persists even when these tools are used.
Peer reviewers, increasingly attentive to the use of LLMs in scientific writing, may infer that LLM use is related to author country of origin. These inferences may consciously or unconsciously bias peer reviewers鈥 scientific assessments, according to new research by 海角乱伦社区 PhD candidate and postdoctoral scholar .
The provided a seed grant for their research, which was accepted for publication at this summer鈥檚 . Their paper, ', is also available now on the pre-print server arXiv.
The work highlights how linguistic biases can persist even with the adoption of AI tools and other technology.
鈥淪o if you have a subconscious bias against people from China, for example, that bias will emerge in other ways, even as language is adjusted [with LLMs]. That鈥檚 the surprising takeaway from this study,鈥 Lepp said.
The researchers looked at nearly 80,000 peer reviews at a large computer science conference and found evidence of bias against authors from countries where English is less widely spoken. After ChatGPT became available, there was only a muted change in the expression of that bias. Through interviews with 14 conference participants from around the world, Lepp and Smith found that reviewers may use common LLM phrases in papers to infer author language backgrounds, affecting their judgments on research quality. Overall, the study shows how ChatGPT might reinforce stereotypes that equate good English with good scientific work.
Lepp, a former natural language processing engineer and digital education practitioner, holds an MS in computational linguistics from the University of Washington and a BS in science, technology, and international affairs from Georgetown University. Her research now focuses on the influence of natural language processing on educational practice. Smith holds a PhD from 海角乱伦社区 and will be starting as an assistant professor of sociology at Duke in the fall.
Why did you and [Daniel Scott Smith] decide to study how LLMs illustrate bias in science journal publishing and also contribute to that bias against non-native English speakers?
We noticed that discussions about scientists using LLMs for English focused on authors rather than readers. So that idea of LLMs as an intervention for scientists puts the onus of change on the authors whose first language isn鈥檛 English, rather than on the root causes of peer reviewers鈥 biases.
In education, there鈥檚 rich literature about language ideology and the role of the listener or reader in linguistic bias. We cite this paper by, who describe how racialized language varieties in American schools are seen as deficits to overcome. Even when students change their writing or speaking, the students continue to experience bias. The source of the bias is deeper than the language itself, and so we wondered if that theory would hold up among international scientists.
One would assume that LLMs could help solve this problem, but your paper indicates that ChatGPT only slightly mutes the bias against non-native English-speaking scientists. Why do you think it hasn鈥檛 removed more of that bias?
The expression of bias appeared not so much around the rules of 鈥淎cademic English鈥 but around what people associated with the kind of scientist who would break such rules. Interviewees described how language quality could be a proxy for science quality.
After ChatGPT came out, peer reviewers noticed that grammatical idiosyncrasies in writing generally started to disappear. Peer reviewers described how instead, they came to guess that certain words or phrases common in LLMs 鈥 like 鈥渄elve鈥 鈥 came from authors from non-English-speaking countries. These guesses also often came with stereotyped descriptions of scientists from different countries.
Looking beyond your work, do you think AI will be a democratizing technology or will it exacerbate inequality?
It depends on your theory of democracy. Daniel Greene and others have critiqued the 鈥渁ccess doctrine鈥 that suggests bringing people access to technology improves democracy. I鈥檓 not sure I buy that access is inherently democratizing. If anything, the idea of AI as democratizing can justify the idea that existing social inequalities can be solved by 鈥渇ixing鈥 marginalized people, rather than looking at how the people at the top, or even social institutions, might be contributing to inequalities. Our findings offer a kind of alternative view. Even when people have tools to act more like a dominant social group, new mechanisms of stratification may emerge.
What do you hope people take away from the paper?
We must interrogate the way that people use language, not just to communicate content but as a sign of other things: of race, class, who to trust, or whose knowledge can be trusted. In science, English-only publishing has a long history with connections to colonialism and racist academic institutions. To repair that, we鈥檒l need more than a tool for helping people produce English text.
One of the things we emphasize in the paper is that biases we identify are actually, in many ways, tools for efficiency. The current speed of computer science publishing may contribute to people cutting corners, evaluating science based on writing style and perception of author background rather than on the science described.
This story was originally published by the .
