Klagenfurt, June 30, 2023

Congratulations to Dr. Minh Nguyen for successfully defending his dissertation on “Policy-driven Dynamic HTTP Adaptive Streaming Player Environment” at Universität Klagenfurt in the context of the Christian Doppler Laboratory ATHENA.

Abstract

In the last decades, video streaming has been developing significantly. Among cur- rent technologies, HTTP Adaptive Streaming (HAS) is considered the de-facto approach in multimedia transmission over the internet. In HAS, the video is split into temporal segments with the same duration (e.g., 4s), each of which is then encoded into different quality versions and stored at servers. The end user sends requests to the server to retrieve segments with specific quality versions determined by an Adaptive Bitrate (ABR) algorithm for the purpose of adapting the throughput fluctuation. Though the majority of HAS-based media services function well even under throughput restrictions and variations, there are still significant challenges for multimedia systems, especially the tradeoff among the increasing content complexity, various time-related requirements, and Quality of Experience (QoE). Content complexity encompasses the increased demands for data, such as high-resolution videos and high frame rates, as well as novel content formats, such as virtual reality (VR) and augmented reality (AR). Time-related requirements include – but are not limited to – start-up delay and end-to-end latency. QoE can be defined as the level of satisfaction or frustration experienced by the user of an application or service. Optimizing for one aspect usually negatively impacts at least one of the other two aspects. This thesis tackles critical open research questions in the context of HAS that significantly impact the QoE at the client side. The main contributions of this thesis are four-fold:

1. This thesis demonstrates that HTTP/3’s features can be utilized to consider- ably enhance the QoE of the end user by improving the video quality. In a streaming session, the end user would have to download low-quality segments, which impair the QoE, due to the throughput fluctuation. We propose Days of Future Past Plus (DoFP+) approach that leverages HTTP/3’s features to upgrade low-quality segments while downloading others. The experimental results reveal a boost in QoE by as much as 33%. In addition, DoFP+ saves an average of 16% of the downloaded data for all test videos. The findings indicate that the sequential download of segments is more advantageous for re- transmissions compared to concurrent downloads, and upgrading lower-quality segments first will result in a more remarkable improvement in QoE.
2. This thesis proposes a weighted sum model, namely WISH, to provide a high QoE of the video and allow end users to express their preferences among different parameters, including data usage, stall events, and video quality. WISH takes into account three distinct cost elements, namely throughput cost, buffer cost, and quality cost, for each quality version and integrates them into a weighted sum as the overall cost. The results of the experiments indicate that WISH enhances the QoE by up to 17.6% while at the same time reducing data usage by 36.4% in comparison to state-of-the-art approaches. It also offers a dynamic adaptation to meet the demands of end users.
3. To improve segment qualities on high-end mobile devices, this thesis introduces an ABR scheme called WISH-SR that integrates a lightweight Convolutional Neural Network (CNN) to enhance low-resolution/low-quality videos at the client side. WISH-SR extends WISH by utilizing CNN-based Super Resolution (SR) models deployed in mobile devices to improve video quality while remarkably reducing the volume of data transmitted. WISH-SR has the capacity to reduce up to 43% of the total downloaded data and enhance the visual quality compared to WISH.
4. Finally, this thesis presents an approach to determine Common Media Client Data (CMCD) parameters from the client and process them at the server for the purpose of generating a suitable bitrate ladder for each client. The bitrate ladder is created based on clients’ device types and network conditions. Our approach is able to reduce the downloaded data while improving the QoE as much as 2.6 times.

IEEE International Conference on

Visual Communications and Image Processing (VCIP)

03-07 December 2023, Jeju, South Korea

[PDF]

Lingfeng Qu (SWJTU, China), Hongjie He (SWJTU, China), Hadi Amirpour (AAU, Austria), Mohammad Ghanbari (University of Essex, UK) , and Christian Timmerer (AAU, Austria)

Abstract: In this paper, we propose a novel attack model called the Got Plaintext Attack (GPA), where the attacker only requires one plaintext and the ciphertext image set stored in the cloud to attack the content of the ciphertext image. Using this model, we examine the security of the Improved Redundant Space Transfer (IRST) encryption method. To this end, we define an ordered characteristic matrix based on the properties of the three keys used in IRST. By comparing the histogram distance of the ordered characteristic matrix, we are able to obtain a plain-ciphertext pair. Furthermore, by leveraging the invariant properties of the ordered characteristic matrix of image blocks in the plain-ciphertext pair, we estimate the block permutation Π2 and the bit-plane permutation sequence Π1. Our experiments show that the accuracy of estimating Π2 is higher than 70% for block sizes of 3×3 pixels or larger. Despite a 40% accuracy in estimating Π1, the content information of the ciphertext image can still be exposed.

IEEE International Conference on

Visual Communications and Image Processing (VCIP)

03-07 December 2023, Jeju, Korea

[PDF]

Jingwen zhu (Nantes University), Hadi Amirpour (AAU, Austria), Raimund Schatz (AIT, Austria) ,  Christian Timmerer (AAU, Austria), and Patrick Le Callet (Nantes University)

Abstract: In adaptive video streaming, optimizing the selection of representations for the encoding bitrate ladder has a significant impact on the quality and economics of media delivery. An efficient way to select representations for the bitrate ladder of a given clip is to consider the Satisfied User Ratio (SUR) of the perceived quality of consecutive representations. This ensures that only representations with one Just Noticeable Difference (JND) are encoded and streamed by avoiding encoding similar-quality representation. VMAF (Video Multi-method Assessment Fusion) presently stands as the most commonly utilized quality metric for constructing bitrate ladders. Hence, the precise determination of JND-optimal encoding step-sizes for the VMAF proxy holds paramount importance; nevertheless, this task is intricate and can present considerable challenges. In this paper, we evaluate the effectiveness of different Video Quality Metrics (VQM) in predicting SUR for the VMAF proxy to better capture content-specific characteristics. Our experimental results provide evidence that incorporating VQM can improve the precision of the SUR prediction for the VMAF proxy. Compared to a state-of-the-art approach that utilizes video complexity metrics, our proposed approach, which incorporates two quality metrics—specifically, VMAF and SSIM calculated at an optimized quantization parameter (QP)—achieves a substantially reduced Mean Absolute Error (MAE) of 1.67. In contrast, the state-of-the-art approach yields an MAE of 2.01. Hence, we recommend using the above quality metrics to improve the accuracy of SUR prediction for the VMAF proxy.

IEEE International Conference on

Visual Communications and Image Processing (VCIP)

03-07 December 2023, Jeju, Korea

[PDF]

Yiqun Liu (Ateme), Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Mohsen Abdoli (IRT b-com),  Christian Timmerer (Alpen-Adria-Universität Klagenfurt), and Thomas Guionnet (Ateme)

Abstract: The integration of advanced video codecs into the streaming pipeline is growing in response to the increasing demand for high quality video content. However, the significant computational demand for advanced codecs like VVC poses challenges for service providers, including longer encoding time and higher encoding cost. This challenge becomes even more pronounced in streaming, as the same content needs to be encoded at multiple bitrates (also known as representations) to accommodate different network conditions. To accelerate the encoding process of multiple representations of the same content in VVC, we employ the encoding map of a single representation, known as the reference representation, and utilize its partitioning structure to accelerate the encoding of the remaining representations, referred to as dependent representations. To ensure compatibility with parallel processing, we designate the lowest bitrate representation as the reference representation. The experimental results indicate a substantial improvement in the encoding time for the dependent representations, achieving an average reduction of 40%, while maintaining a minimal average quality drop of only 0.43 in VMAF. This improvement is observed when utilizing VVenC, an open and optimized VVC encoder implementation.

CSI Magazine Award for Best Sustainability Project or Initiative to GAIA team

We are proud to announce that our project GAIA  was highly commended at the CSI Magazine Awards for best sustainability initiative!  This award recognizes some of the work taking place in the video streaming process to reduce the environmental impact of technology and to drive the world’s green transition.

GAIA (Greek goddess of Earth, mother of all life, personification of the Earth) is a cooperative project between Bitmovin and Alpen-Adria-Universität Klagenfurt (AAU) that aims to enable development of more climate-friendly video streaming platforms by providing complete energy awareness and accountability throughout the entire video delivery chain. GAIA uses innovative technologies such as  Video Encoding Matching-based Model for Cloud and Edge Computing Instances, Green Per-Title Encoding,  and Designing Energy Efficient Player to minimizing the energy consumption and carbon footprint while maintaining the Quality of Experience for users.

We would like to thank the CSI Magazine Awards for this recognition and our partner, Bitmovin, and collaborators for their contribution and support to GAIA. We hope that GAIA will inspire more initiatives and projects that aim to make a positive difference for our planet.

You can find more information about GAIA on our website

13th International Conference on Computer and Knowledge Engineering (ICCKE)

November 1-2, 2023 | Mashad, Iran

[PDF]

Reza Saeedinia (University of Tehran), S. Omid Fatemi (University of Tehran), Daniele Lorenzi (AAU, Austria), Farzad Tashtarian (AAU, Austria),  Christian Timmerer (AAU, Austria)

Live user-generated content (UGC) has increased significantly in video streaming applications. Improving the quality of experience (QoE) for users is a crucial consideration in UGC live streaming, where a user can be both a subscriber and a streamer. Resource allocation is an NP-complete task in UGC live streaming due to many subscribers and streamers with varying requests, bandwidth limitations, and network constraints. In this paper, to decrease the execution time of the resource allocation algorithm, we first process streamers’ and subscribers’ requests and then aggregate them into a limited number of groups based on their preferences. Second, we
perform resource allocation for these groups that we call communities. We formulate the resource allocation problem for communities into an optimization problem. With an efficient aggregation of subscribers and streamers at the core of the proposed architecture, the computational complexity of the optimization problem is reduced, consequently improving QoE. This improvement occurs because of the prompt reaction to the bandwidth fluctuations and, subsequently, appropriate resource allocation by the proposed model. We conduct experiments in various scenarios. The results show an average of 41% improvement in execution time. To evaluate the impact of bandwidth fluctuations on the proposed algorithm, we employ two network traces: AmazonFCC and NYUBUS. The results show 4%, and 28% QoE improvement in a scenario with 5
streamers over the AmazonFCC and the NYUBUS network traces, respectively