On the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics

Several video quality metrics (VQMs) have been proposed in many publications to predict how humans perceive video quality. It is common to observe significant disagreements amongst the quality predictions of these VQMs for the same video sequence. Following an extensive literature search, we found n...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Lohic Fotio Tiotsop, Florence Agboma, Glenn Van Wallendael, Ahmed Aldahdooh, Sebastian Bosse, Lucjan Janowski, Marcus Barkowsky, Enrico Masala
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Objective measures proprietary metrics subjective test video quality metrics disagreement Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/aeda93748ecd4f338ba445d8a90966bd
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:aeda93748ecd4f338ba445d8a90966bd
record_format	dspace
spelling	oai:doaj.org-article:aeda93748ecd4f338ba445d8a90966bd2021-11-20T00:02:00ZOn the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics2169-353610.1109/ACCESS.2021.3127395https://doaj.org/article/aeda93748ecd4f338ba445d8a90966bd2021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9611244/https://doaj.org/toc/2169-3536Several video quality metrics (VQMs) have been proposed in many publications to predict how humans perceive video quality. It is common to observe significant disagreements amongst the quality predictions of these VQMs for the same video sequence. Following an extensive literature search, we found no publicised work that has investigated if such disagreements convey useful information on the accuracy of VQMs. Herein, a measure for quantifying the disagreement between VQMs is proposed. A small-scale subjective study is carried out to assess the effectiveness of our proposal. In particular, the proposed disagreement measure is shown to be extremely effective in determining whether the quality of any given processed video sequence (PVS) can be accurately predicted by the VQMs. This type of information is particularly useful for identifying video sequences that are likely to degrade the end-user’s quality of experience (QoE). Our proposal is also useful in selecting the most effective PVSs to be employed in a subjective test. We show that the proposed disagreement measure can be effectively predicted from bitstream features. This establishes a link between the capability to accurately assess the quality of a PVS and the way it is encoded. In addition, an analysis is conducted to compare the performances of some well-known and widely used open-source metrics and two proprietary metrics. The two proprietary metrics are used by a large media company for enhancing its delivery pipeline. The outcome of this comparison highlights the suitability of the open-source VQM, Video Multi-method Assessment Fusion (VMAF), as a good benchmark quality measure for both the industrial and academic environments.Lohic Fotio TiotsopFlorence AgbomaGlenn Van WallendaelAhmed AldahdoohSebastian BosseLucjan JanowskiMarcus BarkowskyEnrico MasalaIEEEarticleObjective measuresproprietary metricssubjective testvideo qualitymetrics disagreementElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 152923-152937 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Objective measures proprietary metrics subjective test video quality metrics disagreement Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Objective measures proprietary metrics subjective test video quality metrics disagreement Electrical engineering. Electronics. Nuclear engineering TK1-9971 Lohic Fotio Tiotsop Florence Agboma Glenn Van Wallendael Ahmed Aldahdooh Sebastian Bosse Lucjan Janowski Marcus Barkowsky Enrico Masala On the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics
description	Several video quality metrics (VQMs) have been proposed in many publications to predict how humans perceive video quality. It is common to observe significant disagreements amongst the quality predictions of these VQMs for the same video sequence. Following an extensive literature search, we found no publicised work that has investigated if such disagreements convey useful information on the accuracy of VQMs. Herein, a measure for quantifying the disagreement between VQMs is proposed. A small-scale subjective study is carried out to assess the effectiveness of our proposal. In particular, the proposed disagreement measure is shown to be extremely effective in determining whether the quality of any given processed video sequence (PVS) can be accurately predicted by the VQMs. This type of information is particularly useful for identifying video sequences that are likely to degrade the end-user’s quality of experience (QoE). Our proposal is also useful in selecting the most effective PVSs to be employed in a subjective test. We show that the proposed disagreement measure can be effectively predicted from bitstream features. This establishes a link between the capability to accurately assess the quality of a PVS and the way it is encoded. In addition, an analysis is conducted to compare the performances of some well-known and widely used open-source metrics and two proprietary metrics. The two proprietary metrics are used by a large media company for enhancing its delivery pipeline. The outcome of this comparison highlights the suitability of the open-source VQM, Video Multi-method Assessment Fusion (VMAF), as a good benchmark quality measure for both the industrial and academic environments.
format	article
author	Lohic Fotio Tiotsop Florence Agboma Glenn Van Wallendael Ahmed Aldahdooh Sebastian Bosse Lucjan Janowski Marcus Barkowsky Enrico Masala
author_facet	Lohic Fotio Tiotsop Florence Agboma Glenn Van Wallendael Ahmed Aldahdooh Sebastian Bosse Lucjan Janowski Marcus Barkowsky Enrico Masala
author_sort	Lohic Fotio Tiotsop
title	On the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics
title_short	On the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics
title_full	On the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics
title_fullStr	On the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics
title_full_unstemmed	On the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics
title_sort	on the link between subjective score prediction and disagreement of video quality metrics
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/aeda93748ecd4f338ba445d8a90966bd
work_keys_str_mv	AT lohicfotiotiotsop onthelinkbetweensubjectivescorepredictionanddisagreementofvideoqualitymetrics AT florenceagboma onthelinkbetweensubjectivescorepredictionanddisagreementofvideoqualitymetrics AT glennvanwallendael onthelinkbetweensubjectivescorepredictionanddisagreementofvideoqualitymetrics AT ahmedaldahdooh onthelinkbetweensubjectivescorepredictionanddisagreementofvideoqualitymetrics AT sebastianbosse onthelinkbetweensubjectivescorepredictionanddisagreementofvideoqualitymetrics AT lucjanjanowski onthelinkbetweensubjectivescorepredictionanddisagreementofvideoqualitymetrics AT marcusbarkowsky onthelinkbetweensubjectivescorepredictionanddisagreementofvideoqualitymetrics AT enricomasala onthelinkbetweensubjectivescorepredictionanddisagreementofvideoqualitymetrics
_version_	1718419856970219520

On the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics

Ejemplares similares