Dialectal Bias in Bengali: An Evaluation of Multilingual Large Language Models Across Cultural Variations

Abstract

Large Language Models (LLMs) have transformed human-centric AI applications on the Web, yet they often exhibit stereotypes and biases, especially in sensitive contexts like cultural differences in low-resource languages such as Bengali. In this work, we investigate cultural bias in LLMs by evaluating their performance in Bengali cultural dialects of Hindu and Muslim majority. We evaluated widely used Web-enabled models, including ChatGPT, Gemini, and Microsoft Copilot, using a curated data set to analyze their handling of culturally specific terms and approaches to mitigating social biases. By addressing bias in language technologies that underpin the modern Web, our study contributes to advancing human-centered NLP and LLM auditing. Through a detailed exploration of bias causes and evaluation methods, our goal is to promote fairness and inclusion for more than 300 million Bengali speakers in the evolving ecosystem of the Web.