Abstract
Virtual network embedding (VNE) is an essential resource allocation task in network virtualization, aiming to map virtual network requests (VNRs) onto physical infrastructure. Reinforcement learning (RL) has recently emerged as a promising solution to this problem. However, existing RL-based VNE methods are limited by the unidirectional action design and one-size-fits-all training strategy, resulting in restricted searchability and generalizability. In this paper, we propose a FLexible And Generalizable RL framework for VNE, named FlagVNE. Specifically, we design a bidirectional action-based Markov decision process model that enables the joint selection of virtual and physical nodes, thus improving the exploration flexibility of solution space. To tackle the expansive and dynamic action space, we design a hierarchical decoder to generate adaptive action probability distributions and ensure high training efficiency. Furthermore, to overcome the generalization issue for varying VNR sizes, we propose a meta-RL-based training method with a curriculum scheduling strategy, facilitating specialized policy training for each VNR size. Finally, extensive experimental results show the effectiveness of FlagVNE across multiple key metrics. Our code is available at GitHub (this https URL).
Abstract (translated)
虚拟网络嵌入（VNE）是网络虚拟化中一个关键的资源分配任务，旨在将虚拟网络请求（VNRs）映射到物理基础设施。最近，强化学习（RL）已成为解决这个问题的一种有前景的解决方案。然而，现有的基于RL的VNE方法受到单向动作设计和高薪策略的限制，导致搜索性和泛化能力受限。在本文中，我们提出了一个灵活且可扩展的RL框架，名为FlagVNE。具体来说，我们设计了一个双向动作基于马尔可夫决策过程模型，使得虚拟和物理节点能够联合选择，从而提高解决方案空间的探索灵活性。为了应对广泛和动态的动作空间，我们设计了一个分层的解码器，用于生成自适应的动作概率分布，并确保高训练效率。此外，为了克服不同VNR大小下的泛化问题，我们提出了一种基于元RL的培训方法，通过课程计划策略实现针对每个VNR大小的专用策略训练。最后，大量实验结果表明，FlagVNE在多个关键指标上具有有效性。我们的代码可于GitHub上获取（此https链接）。
URL
https://arxiv.org/abs/2404.12633
