摘要：随着自动驾驶系统迈向现实世界部署，可解释的行为级决策对于安全、信任和监管至关重要。我们提出 CommandLM，这是一种多模态大语言模型，能够根据融合后的多传感器数据，为自车生成简洁、可供人类阅读的行为描述。我们的模型通过一个 Q-Former 适配器处理来自 LiDAR 和多摄像头输入的时序融合鸟瞰图表示，该适配器连接至经量化和 LoRA 微调的大语言模型。CommandLM 在我们构建的 CommandLM-nuScenes 数据集上训练，可生成具备意图感知能力、可解释的文本描述，适用于规划器监督和安全审计。实验表明，该模型在语言和行为对齐方面表现强劲，CIDEr 达到 0.67，BERT-F1 达到 0.88，显著优于 BLIP-2 基线模型（CIDEr 为 0.52，BERT-F1 为 0.86）。在人类评估中，58%的生成描述被评为准确、高效且符合规则，证实了其在现实世界中的合理性。尽管其余描述未必总能选择最高效、最具目标导向性的行为，CommandLM 的可解释输出仍可使下游验证系统识别并纠正这些情况，使其成为透明行为审计的有效工具。这些结果表明，将多模态融合与语言推理相结合，能够为自动驾驶提供高效且透明的行为级理解。我们的代码和数据集发布于：此 HTTPS 链接

Abstract:As autonomous driving systems move toward real-world deployment, interpretable, behavior-level decision-making is essential for safety, trust, and regulation. We introduce CommandLM, a multimodal large language model that generates concise, human-readable behavior descriptions for ego vehicles from fused multi-sensor data. Our model processes temporally fused bird's-eye view representations from LiDAR and multi-camera inputs via a Q-Former adapter connected to a quantized, LoRA-fine-tuned large language model. Trained on our CommandLM-nuScenes dataset, CommandLM produces intent-aware, interpretable captions suitable for planner supervision and safety auditing. Experiments demonstrate strong linguistic and behavioral alignment, achieving CIDEr 0.67, and BERT-F1 0.88, substantially outperforming the BLIP-2 baseline (CIDEr 0.52, BERT-F1 0.86). In human evaluation, 58% of the generated descriptions were rated accurate, efficient and rule-compliant, confirming their real-world plausibility. While the remaining descriptions may not always select the most efficient, goal-oriented behavior, CommandLM's interpretable outputs enable downstream validation systems to identify and correct such cases, making it an effective tool for transparent behavior auditing. These results show that integrating multimodal fusion with language reasoning yields efficient and transparent behavior-level understanding for autonomous driving. We release our code and dataset at: this https URL