摘要：二元关系识别旨在根据两个人之间的多模态交互，识别他们之间潜在的社会关系。现有方法通常将异构线索映射到一个共享表示中，并优化扁平的分类目标，忽略了不同关系可能依赖于不同交互模式这一特性。例如，疏远的关系通常可以通过直接的行为对应来推断，而亲密的关系则能更好地通过协调的情感、行为和多模态反应体现出来。为了解决这一局限性，我们提出了 HiRe-DRR——一种分层多模态交互推理框架，该框架从三个互补的视角组织关系证据：特征层级、交互类型和关系层级。HiRe-DRR 将面部、肢体、声学和文本线索分解为低级和高级表示，并构建同质和异质的跨参与者交互。我们进一步引入了交互专家混合 Transformer（Mixture-of-Interaction Transformer, MoI-Former），它将多样化的交互视为专门的专家，并采用关系感知的稀疏路由机制来识别与关系相关的证据，同时保留上下文交互信息。所提出的分层条件损失（HC-Loss）通过从粗到细的监督来利用社会关系的分层特性，从而实现结构化的关系推理。在三个二元关系基准数据集上的大量实验表明，HiRe-DRR 优于近期具有代表性的图像、视频和多模态方法。我们的路由分析进一步揭示，疏远关系倾向于依赖低级同质交互，而更亲密的关系则聚焦于高级异质交互。我们的代码已公开发布在：该 https 链接。

Abstract:Dyadic relationship recognition aims to identify the latent social relationship between two individuals from their multimodal interactions. Existing methods typically project heterogeneous cues into a shared representation and optimise a flat classification objective, overlooking that relationships may rely on different interaction patterns. For instance, distant relationships can often be inferred from direct behavioural correspondence, whereas close relationships are better reflected by coordinated affective, behavioural, and multimodal responses. To address this limitation, we propose HiRe-DRR, a Hierarchical Multimodal Interaction Reasoning framework that organises relationship evidence from three complementary perspectives: feature level, interaction type, and relationship hierarchy. HiRe-DRR decomposes facial, bodily, acoustic, and textual cues into low- and high-level representations and constructs homogeneous and heterogeneous cross-participant interactions. We further introduce the Mixture-of-Interaction Transformer (MoI-Former), which treats diverse interactions as specialised experts and employs relation-aware sparse routing to identify relationship-relevant evidence while preserving contextual interaction information. The proposed Hierarchical Conditional Loss (HC-Loss) exploits the hierarchical nature of social relationships through coarse-to-fine supervision, enabling structured relationship reasoning. Extensive experiments on three dyadic relationship benchmarks demonstrate that HiRe-DRR outperforms recent representative image-, video-, and multimodal methods. Our routing analysis further reveals that distant relationships tend to rely on low-level homogeneous interactions, whereas closer relationships focus on high-level heterogeneous interactions. Our code is publicly available at: this https URL.