摘要：马尔可夫决策过程（MDP）被广泛用作决策模型，通常通过状态变量及其取值定义在因子化状态空间上。状态数量的指数级膨胀，使 MDP 中的许多推理任务面临挑战。抽象是一类很有前景的技术，可用于缩减 MDP，从而缓解可扩展性问题。本文为因子化 MDP 引入了一种因果性概念，并提出一种新颖的属性驱动因果抽象技术，该技术能够保留原始 MDP 模型的诸多特征。为此，作者依赖于状态变量谓词之间的因果关系，并识别那些因满足或违反给定抽象属性而具有相同原因的状态。本文从理论和实验两方面比较了多种因果 MDP 抽象方法，涵盖 MDP、区间 MDP 和随机博弈等不同模型类型。评估结果展示了该方法的潜力：在若干标准基准上，作者获得了较小的抽象模型，使得能够为原始 MDP 计算近似最优策略。此外，这些因果抽象通常还可以泛化到相关的大规模 MDP 模型。

Abstract:Markov Decision Processes (MDPs) are widely used as decision-making models, commonly specified over factored state spaces through state variables and their valuations. The exponential blowup in the number of states renders many reasoning tasks in MDPs challenging. Abstractions are promising techniques to reduce MDPs and thus mitigate scalability issues. In this work, we introduce a notion of causality on factored MDPs and a novel property-driven causal abstraction technique that retains many characteristics of the original MDP model. For this, we rely on causal relations over state variable predicates and identify those states that share the same reasons for fulfilling or violating a given abstraction property. We theoretically and empirically compare various causal MDP abstractions using different model types such as MDPs, interval MDPs, or stochastic games. Our evaluation demonstrates the potential of our approach: For several standard benchmarks, we obtain small abstractions that allow us to compute near-optimal policies for the original MDP. Furthermore, our causal abstractions often generalize to related large-scale MDP models.