A Comparative Study of Objective Video Quality Assessment Metrics

Carlos A.B. Mello (Universidade Federal de Pernambuco, Brazil)

Marília M. Saraiva (Universidade Federal de Pernambuco, Brazil)

Diego P.A. Menor (Universidade Federal de Pernambuco, Brazil)

Ricardo Nishihara (Motorola Company, Brazil)

Abstract: This paper presents a comparison of several video quality metrics, analysing their performance against different types of distortions. Usually, comparisons are made considering a full dataset with few different degradations. We are presenting here a comparison using three very different datasets (VQEG Phase I, LIVE VQA and ReTRIeVED) and a fourth dataset which was generated in a mobile phone network simulator. This was done to check if the video quality metrics can correctly measure the degradations created by variations in the network, very close to real scenarios. The analysis was done with 13 full reference metrics (including Opticom's PEVQ commercial tool) and two no-reference metrics. We have concluded that NTIA's VQM achieved the best results, in most of the cases. It is an open source algorithm that outperformed most of the other techniques, including the licensed PEVQ.

Keywords: PEVQ, SSIM, VQM, video distortions, video quality assessment

Categories: J.0, J.6