强化学习能否高效发现价格操纵？
英文标题：Can Reinforcement Learning Efficiently Discover Price Manipulation?
本文研究了无模型强化学习（RL）在价格操纵策略发现上的表现，并与基于模型的方法进行了比较。研究发现，在中等波动率下，RL 智能体能够仅从有限数据中成功发现盈利的操纵策略，无需知晓底层模型，且当参数估计受到采样误差影响时，RL 始终优于基于模型的方法。在高波动率下，两种方法均无法识别操纵机会；在低波动率下，基于模型的方法则表现更好。这些结果凸显了 RL 在复杂金融控制问题中的有效性，也警示了在缺乏适当防护时部署学习算法可能带来的市场风险。
2.6 KB
2.1 KB
640 B
612 B
380 B
9.1 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026年7月7日]
标题：强化学习能否高效发现价格操纵？
查看论文《强化学习能否高效发现价格操纵？》的 PDF，作者：Ioanna-Yvonni Tsaknaki 等 2 位作者
摘要：在本文中，我们探究了无模型强化学习智能体是否能够比传统的基于模型的方法更有效地识别和利用价格操纵机会。传统方法假设正确指定了数据生成过程，但依赖于有噪声的参数估计。我们考虑一个单一资产市场，其中价格按照具有非线性永久冲击和线性临时冲击的 Almgren-Chriss 框架演化。我们首先证明了离散时间中价格操纵策略的存在性，并在完全信息下使用序列最小二乘二次规划计算了最优基准策略。然后我们比较了两种有限样本学习方式：一种基于模型的过程，从模拟执行数据中估计冲击参数；以及一种无模型方法，使用深度确定性策略梯度，直接在相同数据量上训练。在中等波动性下，RL 智能体成功发现了盈利的操纵策略，且无需明确知晓底层模型，即使训练数据非常有限。更重要的是，当参数估计受采样误差影响时，RL 始终优于基于模型的方法，尽管后者得益于正确的模型设定。在较大波动性下，所有方法都无法识别操纵机会；而在较小波动性下，基于模型的方法优于 RL。这些发现既突显了 RL 在复杂控制问题中的有效性，也突显了在没有适当保障措施的情况下在金融市场部署学习算法所涉及的风险。
提交历史
来自：Ioanna-Yvonni Tsaknaki 查看电子邮件
[v1] 2026年7月7日星期二 10:29:20 UTC (2,974 KB)
English / 原文
[Submitted on 7 Jul 2026]
Title:Can Reinforcement Learning Efficiently Discover Price Manipulation?
View a PDF of the paper titled Can Reinforcement Learning Efficiently Discover Price Manipulation?, by Ioanna-Yvonni Tsaknaki and 2 other authors
Abstract:In this paper, we investigate whether a model-free RL agent can identify and exploit price manipulation opportunities more effectively than a traditional model-based approach that assumes correct specification of the data-generating process but relies on noisy parameter estimates. We consider a single-asset market in which prices evolve according to an Almgren-Chriss framework with non-linear permanent impact and linear temporary impact. We first establish the existence of price-manipulative strategies in discrete time and compute the optimal benchmark strategy using Sequential Least Squares Quadratic Programming under full information. We then compare two finite-sample learning approaches: a model-based procedure that estimates impact parameters from simulated execution data and an agnostic RL approach based on Deep Deterministic Policy Gradient, trained directly on the same amount of data. For intermediate volatility, the RL agent successfully discovers profitable manipulative strategies without explicit knowledge of the underlying model, even when training data are quite limited. More importantly, RL consistently outperforms the model-based approach when parameter estimates are affected by sampling error, despite the latter benefiting from the correct model specification. For large volatility, all methods are unable to identify manipulation opportunities, while for small volatility, the model based approach outperforms RL. These findings highlight both the effectiveness of RL in complex control problems and the risks associated with deploying learning algorithms in financial markets without appropriate safeguards.
Submission history
From: Ioanna-Yvonni Tsaknaki [
]
[v1]
Tue, 7 Jul 2026 10:29:20 UTC (2,974 KB)