KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach

High-Performance Computing (HPC) systems need to be constantly monitored to ensure their stability. The monitoring systems collect a tremendous amount of data about different parameters or Key Performance Indicators (KPIs), such as resource usage, IO waiting time, etc. A proper analysis of this data...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Mohamed Soliman Halawa, Rebeca P. Diaz Redondo, Ana Fernandez Vilas
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Clustering feature extraction high-performance computing time series analysis Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/15158ac5b42c43208be23fbca09f6197
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:15158ac5b42c43208be23fbca09f6197
record_format	dspace
spelling	oai:doaj.org-article:15158ac5b42c43208be23fbca09f61972021-11-19T00:06:31ZKPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach2169-353610.1109/ACCESS.2021.3057427https://doaj.org/article/15158ac5b42c43208be23fbca09f61972021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9348884/https://doaj.org/toc/2169-3536High-Performance Computing (HPC) systems need to be constantly monitored to ensure their stability. The monitoring systems collect a tremendous amount of data about different parameters or Key Performance Indicators (KPIs), such as resource usage, IO waiting time, etc. A proper analysis of this data, usually stored as time series, can provide insight in choosing the right management strategies as well as the early detection of issues. In this paper, we introduce a methodology to cluster HPC jobs according to their KPI indicators. Our approach reduces the inherent high dimensionality of the collected data by applying two techniques to the time series: literature-based and variance-based feature extraction. We also define a procedure to visualize the obtained clusters by combining the two previous approaches and the Principal Component Analysis (PCA). Finally, we have validated our contributions on a real data set to conclude that those KPIs related to CPU usage provide the best cohesion and separation for clustering analysis and the good results of our visualization methodology.Mohamed Soliman HalawaRebeca P. Diaz RedondoAna Fernandez VilasIEEEarticleClusteringfeature extractionhigh-performance computingtime series analysisElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 25522-25543 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Clustering feature extraction high-performance computing time series analysis Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Clustering feature extraction high-performance computing time series analysis Electrical engineering. Electronics. Nuclear engineering TK1-9971 Mohamed Soliman Halawa Rebeca P. Diaz Redondo Ana Fernandez Vilas KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach
description	High-Performance Computing (HPC) systems need to be constantly monitored to ensure their stability. The monitoring systems collect a tremendous amount of data about different parameters or Key Performance Indicators (KPIs), such as resource usage, IO waiting time, etc. A proper analysis of this data, usually stored as time series, can provide insight in choosing the right management strategies as well as the early detection of issues. In this paper, we introduce a methodology to cluster HPC jobs according to their KPI indicators. Our approach reduces the inherent high dimensionality of the collected data by applying two techniques to the time series: literature-based and variance-based feature extraction. We also define a procedure to visualize the obtained clusters by combining the two previous approaches and the Principal Component Analysis (PCA). Finally, we have validated our contributions on a real data set to conclude that those KPIs related to CPU usage provide the best cohesion and separation for clustering analysis and the good results of our visualization methodology.
format	article
author	Mohamed Soliman Halawa Rebeca P. Diaz Redondo Ana Fernandez Vilas
author_facet	Mohamed Soliman Halawa Rebeca P. Diaz Redondo Ana Fernandez Vilas
author_sort	Mohamed Soliman Halawa
title	KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach
title_short	KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach
title_full	KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach
title_fullStr	KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach
title_full_unstemmed	KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach
title_sort	kpis-based clustering and visualization of hpc jobs: a feature reduction approach
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/15158ac5b42c43208be23fbca09f6197
work_keys_str_mv	AT mohamedsolimanhalawa kpisbasedclusteringandvisualizationofhpcjobsafeaturereductionapproach AT rebecapdiazredondo kpisbasedclusteringandvisualizationofhpcjobsafeaturereductionapproach AT anafernandezvilas kpisbasedclusteringandvisualizationofhpcjobsafeaturereductionapproach
_version_	1718420594388631552

KPIs-Based Clustering and Visualization of HPC Jobs: A Feature Reduction Approach

Ejemplares similares