摘要：大型视频语言模型（LVLM）在野外环境中的多模态情感识别（ER）等多模态任务上展现出显著性能。情感识别本质上是一项多模态任务，需要联合理解面部表情、发声、语言、生物信号和手势。然而，真实世界部署仍面临挑战：测试时可能存在模态缺失或噪声。部分观测结果可被视为相对于完整模态分布的一种分布偏移。基于熵最小化或困惑度降低的先进测试时适应（TTA）方法无法迁移至自回归 LVLM，而当观测到的模态较弱时，检索增强生成（RAG）的效果会下降。由于不存在用于验证单次更新的真实标签监督，在这一数据流上进行适应可能持续累积漂移，并在模型偏离可靠解后导致性能下降。因此，有效方案必须能够适应任意的缺失模态模式，并在持续适应过程中保持有效。我们通过测试时自蒸馏（TTSD）同时解决这两个问题。这是一种参数高效框架：一个在完整模态上训练的冻结教师模型，通过自蒸馏指导可适应的低秩学生模型，仅更新数量可忽略不计的参数。Fisher-Anchored Restoration（FAR）则将稳定性构建在同一循环中：它监测 Fisher 信息稳定性，以区分收敛与漂移；当识别出分布偏移时，将学生模型恢复至教师锚点附近。在 MELD、DFEW 和 BAH 数据集上、模态缺失比例为 0% 至 50% 的实验表明，这种统一的适应—恢复设计在较长适应周期内始终优于基于熵的适应方法、RAG 以及基于困惑度的生成方法；缺乏恢复机制的基线方法会逐步退化，而 TTSD-FAR 能保持稳定表现。

Abstract:Large video-language models (LVLMs) have shown remarkable performance on multimodal tasks like multimodal emotion recognition (ER) in the wild. ER is inherently multimodal, requiring a joint understanding of facial expressions, vocalizations, language, biosignals, and gestures. However, real-world deployment remains challenging: modalities may be missing or noisy at test time. Partial observations can be viewed as a distribution shift relative to the complete-modality distribution. SOTA TTA methods based on entropy minimization or perplexity reduction do not transfer to autoregressive LVLMs, while retrieval augmented generation (RAG) degrades when the observed modality is weak. Because no ground-truth supervision exists to verify individual updates, adaptation across this stream risks accumulating drift and degrading once the model departs from a reliable solution. An effective solution must therefore adapt to arbitrary missing-modality patterns and remain effective during continual adaptation. We address both jointly with Test-Time Self-Distillation (TTSD), a parameter-efficient framework in which a frozen teacher, trained on complete modalities, guides an adaptive low-rank student via self-distillation, updating only a negligible number of parameters. Stability is built into this same loop through Fisher-Anchored Restoration (FAR), which monitors Fisher information stability to detect convergence versus drift and restores the student toward the teacher's anchor when distributional shifts are identified. Our experiments on MELD, DFEW, and BAH under 0%-50% missing modalities show that this unified adaptation-restoration design consistently outperforms entropy-based adaptation, RAG, and perplexity-based generation over long adaptation horizons, where baselines without restoration progressively degrade while TTSD-FAR remains consistent.