Policy Distillation for Real-Time Inference in Fronthaul Congestion Control

Centralized Radio Access Networks (C-RANs) are improving their cost-efficiency through packetized fronthaul networks. Such a vision requires network congestion control algorithms to deal with sub-millisecond delay budgets while optimizing link utilization and fairness. Classic congestion control alg...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Jean P. Martins, Igor Almeida, Ricardo Souza, Silvia Lins
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Reinforcement learning policy distillation congestion control real-time inference fronthaul networks Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/dd37039a1ca4403aaa399607a60fe546
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:dd37039a1ca4403aaa399607a60fe546
record_format	dspace
spelling	oai:doaj.org-article:dd37039a1ca4403aaa399607a60fe5462021-11-25T00:00:25ZPolicy Distillation for Real-Time Inference in Fronthaul Congestion Control2169-353610.1109/ACCESS.2021.3129132https://doaj.org/article/dd37039a1ca4403aaa399607a60fe5462021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9618962/https://doaj.org/toc/2169-3536Centralized Radio Access Networks (C-RANs) are improving their cost-efficiency through packetized fronthaul networks. Such a vision requires network congestion control algorithms to deal with sub-millisecond delay budgets while optimizing link utilization and fairness. Classic congestion control algorithms have struggled to optimize these goals simultaneously in such scenarios. Therefore, many Reinforcement Learning (RL) approaches have recently been proposed to deal with such limitations. However, when considering RL policies’ deployment in the real world, many challenges exist. This paper deals with the real-time inference challenge, where a deployed policy has to output actions in microseconds. The experiments here evaluate the tradeoff of inference time and performance regarding a TD3 (Twin-delayed Deep Deterministic Policy Gradient) policy baseline and simpler Decision Tree (DT) policies extracted from TD3 via a process of policy distillation. The results indicate that DTs with a suitable depth can maintain performances similar to those of the TD3 baseline. Additionally, we show that by converting the distilled DTs to rules in C++, we can make inference-time nearly negligible, i.e., sub-microsecond time scale. The proposed method enables the use of state-of-the-art RL techniques to congestion control scenarios with tight inference-time and computational constraints.Jean P. MartinsIgor AlmeidaRicardo SouzaSilvia LinsIEEEarticleReinforcement learningpolicy distillationcongestion controlreal-time inferencefronthaul networksElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 154471-154483 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Reinforcement learning policy distillation congestion control real-time inference fronthaul networks Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Reinforcement learning policy distillation congestion control real-time inference fronthaul networks Electrical engineering. Electronics. Nuclear engineering TK1-9971 Jean P. Martins Igor Almeida Ricardo Souza Silvia Lins Policy Distillation for Real-Time Inference in Fronthaul Congestion Control
description	Centralized Radio Access Networks (C-RANs) are improving their cost-efficiency through packetized fronthaul networks. Such a vision requires network congestion control algorithms to deal with sub-millisecond delay budgets while optimizing link utilization and fairness. Classic congestion control algorithms have struggled to optimize these goals simultaneously in such scenarios. Therefore, many Reinforcement Learning (RL) approaches have recently been proposed to deal with such limitations. However, when considering RL policies’ deployment in the real world, many challenges exist. This paper deals with the real-time inference challenge, where a deployed policy has to output actions in microseconds. The experiments here evaluate the tradeoff of inference time and performance regarding a TD3 (Twin-delayed Deep Deterministic Policy Gradient) policy baseline and simpler Decision Tree (DT) policies extracted from TD3 via a process of policy distillation. The results indicate that DTs with a suitable depth can maintain performances similar to those of the TD3 baseline. Additionally, we show that by converting the distilled DTs to rules in C++, we can make inference-time nearly negligible, i.e., sub-microsecond time scale. The proposed method enables the use of state-of-the-art RL techniques to congestion control scenarios with tight inference-time and computational constraints.
format	article
author	Jean P. Martins Igor Almeida Ricardo Souza Silvia Lins
author_facet	Jean P. Martins Igor Almeida Ricardo Souza Silvia Lins
author_sort	Jean P. Martins
title	Policy Distillation for Real-Time Inference in Fronthaul Congestion Control
title_short	Policy Distillation for Real-Time Inference in Fronthaul Congestion Control
title_full	Policy Distillation for Real-Time Inference in Fronthaul Congestion Control
title_fullStr	Policy Distillation for Real-Time Inference in Fronthaul Congestion Control
title_full_unstemmed	Policy Distillation for Real-Time Inference in Fronthaul Congestion Control
title_sort	policy distillation for real-time inference in fronthaul congestion control
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/dd37039a1ca4403aaa399607a60fe546
work_keys_str_mv	AT jeanpmartins policydistillationforrealtimeinferenceinfronthaulcongestioncontrol AT igoralmeida policydistillationforrealtimeinferenceinfronthaulcongestioncontrol AT ricardosouza policydistillationforrealtimeinferenceinfronthaulcongestioncontrol AT silvialins policydistillationforrealtimeinferenceinfronthaulcongestioncontrol
_version_	1718414726126370816

Policy Distillation for Real-Time Inference in Fronthaul Congestion Control

Ejemplares similares