摘要：自动驾驶车辆必须以乘客能够理解、监控并信任的方式解释其决策。现有带语言标注的驾驶数据集大多由观察者撰写、在事后生成、基于仿真，或从传感器输入中生成，而不是由实际执行驾驶动作的驾驶员提供。我们推出 NARRATE，这是一个多模态澳大利亚真实道路驾驶数据集，包含来自 35 名经验丰富驾驶员和驾驶教练在公共道路上的 2,050 个标注事件。每个事件都基于同步的视觉、定位、运动和 LiDAR 数据流，并配有车内和/或驾驶后采集的自由文本解释。NARRATE 提供驾驶动作标签、覆盖 6 个高层级类别和 32 个细粒度类别的场景上下文标签，以及针对驾驶员解释的跨度级情境感知（Situational Awareness，SA）标注，涵盖感知（Perception）、理解（Comprehension）和预测（Projection）。四项基准任务——SA、场景上下文、驾驶员动作分类及解释生成——表明，这一结构可以从驾驶员语言中学习；但细粒度上下文识别和解释生成仍然具有挑战性。NARRATE 为自动驾驶领域构建更以人为中心、具有更强领域认知能力的解释模型开辟了路径。

Abstract:Automated vehicles must explain their decisions in ways that passengers can understand, monitor, and trust. Existing language-annotated driving datasets are mostly observer-written, post-hoc, simulation-based, or generated from sensor inputs, rather than elicited from the driver performing the action. We introduce NARRATE, a multimodal real-world Australian driving dataset comprising 2,050 annotated events from 35 experienced drivers and driving instructors on public roads. Each event is grounded in synchronised visual, localisation, motion, and LiDAR streams and paired with in-vehicle and/or post-drive free-text explanations. NARRATE provides action labels, scenario-context labels spanning six high-level and 32 fine-grained categories, and span-level Situational Awareness (SA) annotations over driver explanations for Perception, Comprehension and Projection. Four benchmark tasks (SA, scenario-context, driver-action classification, and explanation generation) show that this structure is learnable from driver language, while fine-grained context recognition and explanation generation remain challenging. NARRATE paves a path towards more human-centred and domain-aware explanation models for automated driving.