超越准确率:视觉语言模型用于 Meme 仇恨言论检测的定性分析
英文标题:Beyond Accuracy: A Qualitative Analysis of Vision-Language Models for Hate Speech Detection in Memes
论文指出,Meme 的匿名性与病毒式传播能力使其成为扩散仇恨言论的重要媒介,而此类内容高度依赖语境、反讽和隐性线索,识别难度较大。研究对 LLaVA-7B、Qwen-VL、GPT-4o mini 和 Claude 3 Haiku 四种视觉语言模型进行了定性评估,并比较了零样本与少样本提示下的表现。作者认为,仅以分类准确率衡量模型不足以反映其能力,因此进一步分析了模型生成的判断理由,以理解其识别仇恨 Meme 时的推理方式与局限。该研究对开发者提示,在将 VLM 用于内容审核时,应重点评估其语境理解、反讽识别及解释质量,而不能只关注准确率指标。
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 6 月 27 日]
标题:超越准确率:视觉语言模型用于 Meme 仇恨言论检测的定性分析
查看 Muhammad Jawad Chowdhury 及其他 4 位作者撰写的论文《超越准确率:视觉语言模型用于 Meme 仇恨言论检测的定性分析》PDF。
摘要:Meme 已成为人们分享其对当代社会与政治问题观点的有力工具。其匿名性以及病毒式传播能力,也使其成为传播仇恨的强大媒介。识别这类复杂且依赖语境的仇恨言论仍然非常困难。尽管视觉语言模型(VLM)在多模态任务中表现优异,但它们往往忽略语境、反讽及其他在识别仇恨 Meme 中发挥关键作用的细微线索。在本研究中,我们对四种最先进的 VLM——LLaVA-7B、Qwen-VL、GPT-4o mini 和 Claude 3 Haiku——进行了定性分析。我们在零样本和少样本提示条件下评估这些模型,以考察语境框定如何影响其输出。我们的分析超越了简单的分类准确率,重点对模型生成的理由进行定性评估,从而更深入地理解模型在处理仇恨 Meme 时的思考过程与约束。
提交历史
来自:Muhammad Jawad Chowdhury [
]
[v1]
2026 年 6 月 27 日,星期六,13:56:19 UTC(8,758 KB)
English / 原文
[Submitted on 27 Jun 2026]
Title:Beyond Accuracy: A Qualitative Analysis of Vision-Language Models for Hate Speech Detection in Memes
View a PDF of the paper titled Beyond Accuracy: A Qualitative Analysis of Vision-Language Models for Hate Speech Detection in Memes, by Muhammad Jawad Chowdhury and 4 other authors
Abstract:Memes have turned out to be a powerful tool through which individuals share their ideas concerning contemporary social and political problems. Their anonymity, as well as their ability to go viral, make them a powerful medium for spreading hate. It remains very difficult to identify such complex and context-dependent hate speech. Although they display excellent performance on multimodal tasks, vision-language models (VLMs) tend to ignore context, irony, and other subtle cues that play a key role in identifying hateful memes. In this work, we present a qualitative analysis of four state-of-the-art VLMs: LLaVA-7B, Qwen-VL, GPT-4o mini, and Claude 3 Haiku. We evaluate these models under zero-shot and few-shot prompting to examine how contextual framing influences their outputs. Our analysis goes beyond simple classification accuracy and focuses on a qualitative evaluation of the models' generated justifications, providing a more in-depth understanding of their thought processes and constraints when dealing with hateful memes.
Submission history
From: Muhammad Jawad Chowdhury [
]
[v1]
Sat, 27 Jun 2026 13:56:19 UTC (8,758 KB)