Klagenfurt, July 10, 2024
Congratulations to Dr. Babak Taraghi for successfully defending his dissertation on “End-to-end Quality of Experience Evaluation for HTTP Adaptive Streaming” at Universität Klagenfurt in the context of the Christian Doppler Laboratory ATHENA.
Abstract
HTTP Adaptive Streaming (HAS) has risen to prominent acclaim as the prevailing approach for distributing video content across the Internet. The emergence of popular online streaming platforms, which mainly leverage HAS, has led to a surge in the number of users actively generating and consuming high-quality content. Nonetheless, this remarkable surge presents an intricate puzzle for scholars and service providers, who must contend with varying network conditions and limited network resources to meet user expectations for quality.
In response to these challenges, this dissertation explores the end-to-end evaluation of Quality of Experience (QoE) in the context of HAS. This dissertation investigates evaluation methodologies and frameworks designed to measure QoE and end-to-end latency, particularly in live HAS deployments. We identified the gaps and challenges in current QoE evaluation methodologies through extensive literature reviews and analysis of existing approaches. This thesis proposes novel contributions to address these gaps, encompassing the development of evaluation frameworks, enhancing the understanding of QoE, in-depth studies on QoE impacting factors, and curating a comprehensive dataset.
This dissertation’s first category of contributions is the development of two evaluation frameworks, CAdViSE and LLL-CAdViSE. These frameworks provide researchers and developers with powerful tools to assess the performance and QoE of HAS systems. By harnessing the potential of cloud-based architectures and cuttingedge testing functionalities, these frameworks empower the undertaking of expansive evaluations, incorporating various streaming protocols, codecs, and various network
scenarios. As a result, they contribute significantly to the refinement of streaming systems. Notably, both frameworks are available to the public as open-source projects, marking a noteworthy stride in advancing the field.
As a second category of contributions, we present two extensive studies
investigating the metrics and factors influencing QoE. We investigated the impact of the performance of heuristic-based algorithms on QoE by employing subjective assessment methods and analyzing the influence of algorithmic decisions on user perception. We did an in-depth analysis of stall events and quality switches by conducting subjective assessments and Analysis of Variance (ANOVA) to unveil their influence on QoE. We found that the longer stall events led to greater dissatisfaction.
Further investigation focused on stall event duration and rebuffering’s impact on QoE. Our evaluations revealed that stall events under 4ms went unnoticed by users. Shorter stall durations were generally more tolerable, and improved buffering strategies helped mitigate stall effects on QoE.
In the third contribution category, this thesis fulfills the requirement for contemporary datasets that mirror the latest progress in video technology. A thoroughgoing collection named the ”Multi-codec Ultra High Definition 8K MPEG DASH Dataset” has been meticulously curated. It encompasses a wide array of video content, encoded with cutting-edge codecs like VVC and boasting resolutions up to 8K. This comprehensive dataset forms the bedrock for evaluations across diverse streaming scenarios.
This dissertation advances the field of QoE evaluation for HAS through the development of evaluation frameworks, insightful studies, in-depth analysis, and the presentation of a comprehensive dataset. It provides a ground for researchers and developers to assess and enhance the streaming experience, leading to improved algorithms, optimized systems, and enhanced user satisfaction in HAS.
Slides are available here.
