Multiagent Deep Reinforcement Learning for Cost- and Delay-Sensitive Virtual Network Function Placement and Routing

Shaoyang Wang, Chau Yuen, Wei Ni, Yong Liang Guan, Tiejun Lv*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

24 Citations (Scopus)

Abstract

This paper proposes an effective and novel multi-agent deep reinforcement learning (MADRL)-based method for solving the joint virtual network function (VNF) placement and routing (P&R), where multiple service requests with differentiated demands are delivered at the same time. The differentiated demands of the service requests are reflected by their delay- and cost-sensitive factors. We first construct a VNF P&R problem to jointly minimize a weighted sum of service delay and resource consumption cost, which is NP-complete. Then, the joint VNF P&R problem is decoupled into two iterative subtasks: placement subtask and routing subtask. Each subtask consists of multiple concurrent parallel sequential decision processes. By invoking the deep deterministic policy gradient method and multi-agent technique, an MADRL-P&R framework is designed to perform the two subtasks. The new joint reward and internal rewards mechanism is proposed to match the goals and constraints of the placement and routing subtasks. We also propose the parameter migration-based model-retraining method to deal with changing network topologies. Corroborated by experiments, the proposed MADRL-P&R framework is superior to its alternatives in terms of service cost and delay, and offers higher flexibility for personalized service demands. The parameter migration-based model-retraining method can efficiently accelerate convergence under moderate network topology changes.

Original languageEnglish
Pages (from-to)5208-5224
Number of pages17
JournalIEEE Transactions on Communications
Volume70
Issue number8
DOIs
Publication statusPublished - Aug 1 2022
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 1972-2012 IEEE.

ASJC Scopus Subject Areas

  • Electrical and Electronic Engineering

Keywords

  • multi-agent deep reinforcement learning
  • Placement and routing
  • virtual network functions

Fingerprint

Dive into the research topics of 'Multiagent Deep Reinforcement Learning for Cost- and Delay-Sensitive Virtual Network Function Placement and Routing'. Together they form a unique fingerprint.

Cite this