Reinforcement Learning-Based Routing Algorithm in Satellite-Terrestrial Integrated Networks

Satellite-terrestrial integrated network (STIN) is an indispensable component of the Next Generation Internet (NGI) due to its wide coverage, high flexibility, and seamless communication services. It uses the part of satellite network to provide communication services to the users who cannot communi...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Yabo Yin, Chuanghe Huang, Dong-Fang Wu, Shidong Huang, M. Wasim Abbas Ashraf, Qianqian Guo
Formato:	article
Lenguaje:	EN
Publicado:	Hindawi-Wiley 2021
Materias:	Technology T Telecommunication TK5101-6720
Acceso en línea:	https://doaj.org/article/9b75be738fa840d692430dd1869c5f76
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:9b75be738fa840d692430dd1869c5f76
record_format	dspace
spelling	oai:doaj.org-article:9b75be738fa840d692430dd1869c5f762021-11-08T02:36:09ZReinforcement Learning-Based Routing Algorithm in Satellite-Terrestrial Integrated Networks1530-867710.1155/2021/3759631https://doaj.org/article/9b75be738fa840d692430dd1869c5f762021-01-01T00:00:00Zhttp://dx.doi.org/10.1155/2021/3759631https://doaj.org/toc/1530-8677Satellite-terrestrial integrated network (STIN) is an indispensable component of the Next Generation Internet (NGI) due to its wide coverage, high flexibility, and seamless communication services. It uses the part of satellite network to provide communication services to the users who cannot communicate directly in terrestrial network. However, existing satellite routing algorithms ignore the users’ request resources and the states of the satellite network. Therefore, these algorithms cannot effectively manage network resources in routing, leading to the congestion of satellite network in advance. To solve this problem, we model the routing problem in satellite network as a finite-state Markov decision process and formulate it as a combinatorial optimization problem. Then, we put forth a Q-learning-based routing algorithm (QLRA). By maximizing users’ utility, our proposed QLRA algorithm is able to select the optimal paths according to the dynamic characteristics of satellite network. Considering that the convergence speed of QLRA is slow due to the routing loop or ping-pong effect in the process of routing, we propose a split-based speed-up convergence strategy and also design a speed-up Q-learning-based routing algorithm, termed SQLRA. In addition, we update the Q value of each node from back to front in the learning process, which further accelerate the convergence speed of SQLRA. Experimental results show that our improved routing algorithm SQLRA greatly enhances the performance of satellite network in terms of throughput, delay, and bit error rate compared with other routing algorithms.Yabo YinChuanghe HuangDong-Fang WuShidong HuangM. Wasim Abbas AshrafQianqian GuoHindawi-WileyarticleTechnologyTTelecommunicationTK5101-6720ENWireless Communications and Mobile Computing, Vol 2021 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Technology T Telecommunication TK5101-6720
spellingShingle	Technology T Telecommunication TK5101-6720 Yabo Yin Chuanghe Huang Dong-Fang Wu Shidong Huang M. Wasim Abbas Ashraf Qianqian Guo Reinforcement Learning-Based Routing Algorithm in Satellite-Terrestrial Integrated Networks
description	Satellite-terrestrial integrated network (STIN) is an indispensable component of the Next Generation Internet (NGI) due to its wide coverage, high flexibility, and seamless communication services. It uses the part of satellite network to provide communication services to the users who cannot communicate directly in terrestrial network. However, existing satellite routing algorithms ignore the users’ request resources and the states of the satellite network. Therefore, these algorithms cannot effectively manage network resources in routing, leading to the congestion of satellite network in advance. To solve this problem, we model the routing problem in satellite network as a finite-state Markov decision process and formulate it as a combinatorial optimization problem. Then, we put forth a Q-learning-based routing algorithm (QLRA). By maximizing users’ utility, our proposed QLRA algorithm is able to select the optimal paths according to the dynamic characteristics of satellite network. Considering that the convergence speed of QLRA is slow due to the routing loop or ping-pong effect in the process of routing, we propose a split-based speed-up convergence strategy and also design a speed-up Q-learning-based routing algorithm, termed SQLRA. In addition, we update the Q value of each node from back to front in the learning process, which further accelerate the convergence speed of SQLRA. Experimental results show that our improved routing algorithm SQLRA greatly enhances the performance of satellite network in terms of throughput, delay, and bit error rate compared with other routing algorithms.
format	article
author	Yabo Yin Chuanghe Huang Dong-Fang Wu Shidong Huang M. Wasim Abbas Ashraf Qianqian Guo
author_facet	Yabo Yin Chuanghe Huang Dong-Fang Wu Shidong Huang M. Wasim Abbas Ashraf Qianqian Guo
author_sort	Yabo Yin
title	Reinforcement Learning-Based Routing Algorithm in Satellite-Terrestrial Integrated Networks
title_short	Reinforcement Learning-Based Routing Algorithm in Satellite-Terrestrial Integrated Networks
title_full	Reinforcement Learning-Based Routing Algorithm in Satellite-Terrestrial Integrated Networks
title_fullStr	Reinforcement Learning-Based Routing Algorithm in Satellite-Terrestrial Integrated Networks
title_full_unstemmed	Reinforcement Learning-Based Routing Algorithm in Satellite-Terrestrial Integrated Networks
title_sort	reinforcement learning-based routing algorithm in satellite-terrestrial integrated networks
publisher	Hindawi-Wiley
publishDate	2021
url	https://doaj.org/article/9b75be738fa840d692430dd1869c5f76
work_keys_str_mv	AT yaboyin reinforcementlearningbasedroutingalgorithminsatelliteterrestrialintegratednetworks AT chuanghehuang reinforcementlearningbasedroutingalgorithminsatelliteterrestrialintegratednetworks AT dongfangwu reinforcementlearningbasedroutingalgorithminsatelliteterrestrialintegratednetworks AT shidonghuang reinforcementlearningbasedroutingalgorithminsatelliteterrestrialintegratednetworks AT mwasimabbasashraf reinforcementlearningbasedroutingalgorithminsatelliteterrestrialintegratednetworks AT qianqianguo reinforcementlearningbasedroutingalgorithminsatelliteterrestrialintegratednetworks
_version_	1718443213309607936

Reinforcement Learning-Based Routing Algorithm in Satellite-Terrestrial Integrated Networks

Ejemplares similares