SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs
英文标题：SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs
arXiv:2608.07862v1 Announce Type: new Abstract: Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguistic diversity and culturally grounded safety risks present in other languages. To address this gap,
2.1 KB
483 B
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
arXiv:2608.07862v1 Announce Type: new Abstract: Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguistic diversity and culturally grounded safety risks present in other languages. To address this gap,
中文全文翻译正在补齐，可先查看下方英文原文。
English / 原文
[Submitted on 8 Aug 2026]
Title:SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs
Authors:Debopriyo Banerjee
,
,
,
,
,
,
,
,
,
,
,
,
,
View a PDF of the paper titled SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs, by Debopriyo Banerjee and 13 other authors
Abstract:Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguistic diversity and culturally grounded safety risks present in other languages. To address this gap, we introduce SurakshaEval, a novel safety benchmark composed of human-written prompts spanning real-world scenarios, explicitly designed for ten major Indian languages - Assamese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Punjabi, Tamil, and Telugu, along with English. SurakshaEval includes both generic prompts common across India and region- and language-specific prompts that capture localized sociocultural sensitivities. We benchmark a broad range of state-of-the-art LLMs on SurakshaEval, establish baseline safety performance, and identify recurring failure modes, including over-refusal, missed detection of implicit bias, and insufficient contextual awareness in regionally sensitive settings. Our results show that even strong multilingual LLMs struggle to reliably meet nuanced safety requirements when operating in Indic languages, particularly in native scripts. These findings highlight the urgent need for safety evaluation frameworks that incorporate region-specific data and structured assessment protocols, enabling the development and deployment of AI systems that operate securely, ethically, and in alignment with diverse societal values. Our code and data are available at this https URL. Warning: This paper contains text that may be offensive or unsafe.
Submission history
From: Debopriyo Banerjee [
]
[v1]
Sat, 8 Aug 2026 02:03:33 UTC (7,491 KB)