Abstract
Deep reinforcement learning (DRL) has revolutionized quantitative finance by achieving excellent performance without significant manual effort. Whereas we observe that the DRL models behave unstably in a dynamic stock market due to the low signal-to-noise ratio nature of the financial data. In this paper, we propose a novel logic-guided trading framework, termed as SYENS (Program Synthesis-based Ensemble Strategy). Different from the previous state-of-the-art ensemble reinforcement learning strategy which arbitrarily selects the best-performing agent for testing based on a single measurement, our framework proposes regularizing the model's behavior in a hierarchical manner using the program synthesis by sketching paradigm. First, we propose a high-level, domain-specific language (DSL) that is used for the depiction of the market environment and action. Then based on the DSL, a novel program sketch is introduced, which embeds human expert knowledge in a logical manner. Finally, based on the program sketch, we adopt the program synthesis by sketching a paradigm and synthesizing a logical, hierarchical trading strategy. We evaluate SYENS on the 30 Dow Jones stocks under the cash trading and the margin trading settings. Experimental results demonstrate that our proposed framework can significantly outperform the baselines with much higher cumulative return and lower maximum drawdown under both settings.
Abstract (translated)
深度强化学习（DRL）通过实现无需大量手动努力的优秀性能，极大地推动了量化金融的发展。然而，我们观察到，由于金融数据信号与噪声比低，动态股票市场中的DRL模型表现不稳定。在本文中，我们提出了一个新颖的基于逻辑的指导交易框架，称为SYENS（基于程序合成的主导策略）。与之前的状态级强化学习策略不同，该框架通过绘制范式对模型的行为进行层次化规范。首先，我们提出了一个高级、领域特定的语言（DSL），用于描述市场环境和动作。然后基于DSL，我们引入了一个新颖的程序草图，以直观地表示人类专家知识。最后，基于程序草图，我们采用基于绘图范式进行程序合成，并合成一个逻辑分层交易策略。我们在现金交易和保证金交易设置下对30只道琼斯股票进行了对SYENS的评估。实验结果表明，与基线相比，我们的框架具有更高的累计回报和较低的最大回撤，尤其是在设置下。
URL
https://arxiv.org/abs/2310.05551
