Intelligent Performance Prediction: The Use Case of a Hadoop Cluster

The optimum utilization of infrastructural resources is a highly desired yet cumbersome task for service providers to achieve. This is because the optimal amount of such resources is a function of various parameters, such as the desired/agreed quality of service (QoS), the service characteristics/pr...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Dimitris Uzunidis, Panagiotis Karkazis, Chara Roussou, Charalampos Patrikakis, Helen C. Leligou
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	heterogeneous monitoring machine learning next generation networking software defined networking hadoop Electronics TK7800-8360
Acceso en línea:	https://doaj.org/article/3ca98c9513364d889f456a5451109212
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:3ca98c9513364d889f456a5451109212
record_format	dspace
spelling	oai:doaj.org-article:3ca98c9513364d889f456a54511092122021-11-11T15:40:55ZIntelligent Performance Prediction: The Use Case of a Hadoop Cluster10.3390/electronics102126902079-9292https://doaj.org/article/3ca98c9513364d889f456a54511092122021-11-01T00:00:00Zhttps://www.mdpi.com/2079-9292/10/21/2690https://doaj.org/toc/2079-9292The optimum utilization of infrastructural resources is a highly desired yet cumbersome task for service providers to achieve. This is because the optimal amount of such resources is a function of various parameters, such as the desired/agreed quality of service (QoS), the service characteristics/profile, workload and service life-cycle. The advent of frameworks that foresee the dynamic establishment and placement of service and network functions further contributes to a decrease in the effectiveness of traditional resource allocation methods. In this work, we address this problem by developing a mechanism which first performs service profiling and then a prediction of the resources that would lead to the desired QoS for each newly deployed service. The main elements of our approach are as follows: (a) the collection of data from all three layers of the deployed infrastructure (hardware, virtual and service), instead of a single layer of the deployed infrastructure, to provide a clearer picture on the potential system break points, (b) the study of well-known container based implementations following that microservice paradigm and (c) the use of a data analysis routine that employs a set of machine learning algorithms and performs accurate predictions of the required resources for any future service requests. We investigate the performance of the proposed framework using our open-source implementation to examine the case of a Hadoop cluster. The results show that running a small number of tests is adequate to assess the main system break points and at the same time to attain accurate resource predictions for any future request.Dimitris UzunidisPanagiotis KarkazisChara RoussouCharalampos PatrikakisHelen C. LeligouMDPI AGarticleheterogeneous monitoringmachine learningnext generation networkingsoftware defined networkinghadoopElectronicsTK7800-8360ENElectronics, Vol 10, Iss 2690, p 2690 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	heterogeneous monitoring machine learning next generation networking software defined networking hadoop Electronics TK7800-8360
spellingShingle	heterogeneous monitoring machine learning next generation networking software defined networking hadoop Electronics TK7800-8360 Dimitris Uzunidis Panagiotis Karkazis Chara Roussou Charalampos Patrikakis Helen C. Leligou Intelligent Performance Prediction: The Use Case of a Hadoop Cluster
description	The optimum utilization of infrastructural resources is a highly desired yet cumbersome task for service providers to achieve. This is because the optimal amount of such resources is a function of various parameters, such as the desired/agreed quality of service (QoS), the service characteristics/profile, workload and service life-cycle. The advent of frameworks that foresee the dynamic establishment and placement of service and network functions further contributes to a decrease in the effectiveness of traditional resource allocation methods. In this work, we address this problem by developing a mechanism which first performs service profiling and then a prediction of the resources that would lead to the desired QoS for each newly deployed service. The main elements of our approach are as follows: (a) the collection of data from all three layers of the deployed infrastructure (hardware, virtual and service), instead of a single layer of the deployed infrastructure, to provide a clearer picture on the potential system break points, (b) the study of well-known container based implementations following that microservice paradigm and (c) the use of a data analysis routine that employs a set of machine learning algorithms and performs accurate predictions of the required resources for any future service requests. We investigate the performance of the proposed framework using our open-source implementation to examine the case of a Hadoop cluster. The results show that running a small number of tests is adequate to assess the main system break points and at the same time to attain accurate resource predictions for any future request.
format	article
author	Dimitris Uzunidis Panagiotis Karkazis Chara Roussou Charalampos Patrikakis Helen C. Leligou
author_facet	Dimitris Uzunidis Panagiotis Karkazis Chara Roussou Charalampos Patrikakis Helen C. Leligou
author_sort	Dimitris Uzunidis
title	Intelligent Performance Prediction: The Use Case of a Hadoop Cluster
title_short	Intelligent Performance Prediction: The Use Case of a Hadoop Cluster
title_full	Intelligent Performance Prediction: The Use Case of a Hadoop Cluster
title_fullStr	Intelligent Performance Prediction: The Use Case of a Hadoop Cluster
title_full_unstemmed	Intelligent Performance Prediction: The Use Case of a Hadoop Cluster
title_sort	intelligent performance prediction: the use case of a hadoop cluster
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/3ca98c9513364d889f456a5451109212
work_keys_str_mv	AT dimitrisuzunidis intelligentperformancepredictiontheusecaseofahadoopcluster AT panagiotiskarkazis intelligentperformancepredictiontheusecaseofahadoopcluster AT chararoussou intelligentperformancepredictiontheusecaseofahadoopcluster AT charalampospatrikakis intelligentperformancepredictiontheusecaseofahadoopcluster AT helencleligou intelligentperformancepredictiontheusecaseofahadoopcluster
_version_	1718434338914172928

Intelligent Performance Prediction: The Use Case of a Hadoop Cluster

Ejemplares similares