LG-GER:通过多模态证据蒸馏实现语言引导的群体情绪识别
英文标题:LG-GER: Language-Guided Group Emotion Recognition via Multimodal Evidence Distillation
LG-GER 提出了一种用于群体情绪识别(GER)的语言引导蒸馏框架,可从单张图像中推断人群的整体情绪状态。该方法利用多模态大语言模型(MLLM)为训练图像生成带有情绪信号和置信度的空间定位证据,再通过四类损失将这些知识蒸馏至单一视觉语言模型(VLM)骨干网络。模型在推理时不需要目标检测器、MLLM 或多流融合,因而更适合实时和资源受限的部署场景。在 GroupEmoW 和 GAF~3.0 两个基准数据集上,LG-GER 的表现可与或优于需要检测和多流推理的现有先进方法。
640 B
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 8 月 24 日]
标题:LG-GER:通过多模态证据蒸馏实现语言引导的群体情绪识别
查看 Ahmed Shehab Khan 及另外 2 位作者撰写的论文《LG-GER:通过多模态证据蒸馏实现语言引导的群体情绪识别》PDF
摘要:从单张图像中推断一群人的集体情绪状态,这一任务被称为群体情绪识别(group emotion recognition,GER),需要整合在人群空间范围内分布的线索,例如面部表情、姿态、互动关系和场景上下文。现有方法依赖由检测器驱动的多流处理流水线。这些方法仅使用图像级监督进行训练,缺乏关于哪些区域重要、以及各区域应在多大程度上影响结果的指导。我们提出 LG-GER,这是一种语言引导的蒸馏框架,利用多模态大语言模型(MLLM)为训练图像生成密集且具有空间定位的信息证据,即与情绪信号和置信度分数配对的边界框。该结构化证据通过四种互补损失被蒸馏至单一视觉语言模型(VLM)骨干网络中:分类损失、区域—文本对齐损失、空间情绪损失和空间置信度回归损失。在推理阶段,LG-GER 无需检测器、MLLM 或多流融合,使 GER 能够应用于实时和资源受限的部署环境。LG-GER 已在两个 GER 基准数据集(GroupEmoW 和 GAF~3.0)上进行了评估;与在推理阶段需要检测和多流处理的最先进方法相比,其结果具有竞争力或更优。
提交历史
来自:Ahmed Shehab Khan [
]
[v1]
2026 年 8 月 24 日,周一,22:39:25 UTC(19,529 KB)
English / 原文
[Submitted on 24 Aug 2026]
Title:LG-GER: Language-Guided Group Emotion Recognition via Multimodal Evidence Distillation
View a PDF of the paper titled LG-GER: Language-Guided Group Emotion Recognition via Multimodal Evidence Distillation, by Ahmed Shehab Khan and 2 other authors
Abstract:Inferring the collective emotional state of a group of people from a single image, a task known as group emotion recognition (GER), requires integrating spatially distributed cues such as faces, poses, interactions, and scene context. Current methods rely on detector-driven multi-stream pipelines. These are trained with only image-level supervision that lacks guidance on which regions matter or how strongly each contributes. We propose LG-GER, a language-guided distillation framework that uses a multimodal large language model (MLLM) to generate dense, spatially grounded evidence, i.e., bounding boxes paired with emotion signals and confidence scores, for the training images. This structured evidence is distilled into a single vision-language model (VLM) backbone through four complementary losses: classification, region-text grounding, spatial emotion, and spatial confidence regression. At inference, LG-GER requires no detectors, no MLLM, and no multi-stream fusion, making GER practical for real-time and resource-constrained deployment. LG-GER has been evaluated on two benchmark GER datasets (GroupEmoW and GAF~3.0) and achieves competitive or superior results compared to state-of-the-art methods that require detection and multi-stream processing at inference.
Submission history
From: Ahmed Shehab Khan [
]
[v1]
Mon, 24 Aug 2026 22:39:25 UTC (19,529 KB)