Just Noticeable Difference-aware Per-Scene Bitrate-laddering for Adaptive Video Streaming

IEEE International Conference on Multimedia and Expo (ICME), 2023

10-14 July 2023, Brisbane, Australia

Conference Website


Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Jingwen Zhu (Nantes Universite), Prajit T Rajendran (Universite Paris-Saclay), Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Patrick Le Callet (Nantes Universite),  and Christian Timmerer (Alpen-Adria-Universität Klagenfurt).


In video streaming applications, a fixed set of bitrate-resolution pairs (known as a bitrate ladder) is typically used during the entire streaming session. However, an optimized bitrate ladder per scene may result in (i) decreased storage or delivery costs or/and (ii) increased Quality of Experience. This paper introduces a Just Noticeable Difference (JND)-aware per-scene bitrate ladder prediction scheme (JASLA) for adaptive video-on-demand streaming applications. JASLA predicts jointly optimized resolutions and corresponding constant rate factors (CRFs) using spatial and temporal complexity features for a given set of target bitrates for every scene, which yields an efficient constrained Variable Bitrate encoding. Moreover, bitrate-resolution pairs that yield distortion lower than one JND are eliminated. Experimental results show that, on average, JASLA yields bitrate savings of 34.42% and 42.67% to maintain the same PSNR and VMAF, respectively, compared to the reference HTTP Live Streaming (HLS) bitrate ladder Constant Bitrate encoding using x265 HEVC encoder, where the maximum resolution of streaming is Full HD (1080p). Moreover, a 54.34% average cumulative decrease in storage space is observed.

JASLA architecture.

Posted in News | Comments Off on Just Noticeable Difference-aware Per-Scene Bitrate-laddering for Adaptive Video Streaming

Optimizing Video Streaming for Sustainability and Quality: The Role of Preset Selection in Per-Title Encoding

IEEE International Conference on Multimedia and Expo (ICME)

10-14 July 2023, Brisbane, Australia



Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Samira Afzal (Alpen-Adria-Universität Klagenfurt), Radu Prodan (Alpen-Adria-Universität Klagenfurt),  and Christian Timmerer (Alpen-Adria-Universität Klagenfurt)

Abstract: HTTP Adaptive Streaming (HAS) methods divide a video into smaller segments, encoded at multiple pre-defined bitrates to construct a bitrate ladder. Bitrate ladders are usually optimized per title over several dimensions, such as bitrate, resolution, and framerate. This paper adds a new dimension to the bitrate ladder by considering the energy consumption of the encoding process. Video encoders often have multiple pre-defined presets to balance the trade-off between encoding time, energy consumption, and compression efficiency. Faster presets disable certain coding tools defined by the codec to reduce the encoding time at the cost of reduced compression efficiency. Firstly, this paper evaluates the energy consumption and compression efficiency of different x265 presets for 500 video sequences. Secondly, optimized presets are selected for various representations in a bitrate ladder based on the results to guarantee a minimal drop in video quality while saving energy. Finally, a new per-title model, which optimizes the trade-off between compression efficiency and energy consumption, is proposed. The experimental results show that decreasing the VMAF score by 0.15 and 0.39 while choosing an optimized preset results in encoding energy savings of 70% and 83%, respectively.

Keywords: Bitrate ladder, per-title encoding, green video streaming, energy efficiency.

Posted in GAIA, News | Comments Off on Optimizing Video Streaming for Sustainability and Quality: The Role of Preset Selection in Per-Title Encoding

VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing Instances

GMSys 2023: First International ACM Green Multimedia Systems Workshop

7 – 10 June 2023 | Vancouver, Canada

Conference Website

Samira Afzal (Alpen-Adria-Universität Klagenfurt), Narges Mehran (Alpen-Adria-Universität Klagenfurt), Sandro Linder (Bitmovin), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), and Radu Prodan (Alpen-Adria-Universität Klagenfurt)

Abstract: The considerable surge in energy consumption within data centers can be attributed to the exponential rise in demand for complex computing workflows and storage resources. Video streaming applications are both compute and storage-intensive and account for the majority of today’s internet services. In this work, we designed a video encoding application consisting of codec, bitrate, and resolution set for encoding a video segment. Then, we propose VE-Match, a matching-based method to schedule video encoding applications on both Cloud and Edge resources to optimize costs and energy consumption. Evaluation results on a real computing testbed federated between Amazon Web Services (AWS) EC2 Cloud instances and the Alpen-Adria University (AAU) Edge server reveal that VE-Match achieves lower costs by 17%-78% in the cost-optimized scenarios compared to the energy-optimized and tradeoff between cost and energy. Moreover, VE-Match improves the video encoding energy consumption by 38%-45% and gCO2 emission by up to 80 % in the energy-optimized scenarios compared to the cost-optimized and tradeoff between cost and energy.

Keywords: Video encoding, Cloud and Edge computing, energy consumption, CO2 emission, scheduling.

Posted in GAIA | Comments Off on VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing Instances

HTTP Adaptive Streaming – Quo Vadis? (2023)

IEEE ComSoc MMTC Distinguished Lecture Series

Speaker: Prof. Christian Timmerer, Alpen-Adria-Universität Klagenfurt (AAU), Austria

Date/Time: Thursday, Apr 20, 2023, 10:00 AM Pacific Time (US and Canada), CET Time 7:00 PM Austria
Title: HTTP Adaptive Streaming (HAS) — Quo Vadis? (2023; for the 2021 version, see here)

Abstract: Video traffic on the Internet is constantly growing; networked multimedia applications consume a predominant share of the available Internet bandwidth. A major technical breakthrough and enabler in multimedia systems research and of industrial networked multimedia services certainly was the HTTP Adaptive Streaming (HAS) technique. This resulted in the standardization of MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) which, together with HTTP Live Streaming (HLS), is widely used for multimedia delivery in today’s networks. Existing challenges in multimedia systems research deal with the trade-off between (i) the ever-increasing content complexity, (ii) various requirements with respect to time (most importantly, latency), and (iii) quality of experience (QoE). Optimizing towards one aspect usually negatively impacts at least one of the other two aspects if not both. This situation sets the stage for our research work in the ATHENA Christian Doppler (CD) Laboratory (Adaptive Streaming over HTTP and Emerging Networked Multimedia Services; https://athena.itec.aau.at/), jointly funded by public sources and industry. In this talk, we will present selected novel approaches and research results of the first year of the ATHENA CD Lab’s operation. We will highlight HAS-related research on (i) multimedia content provisioning (machine learning for video encoding); (ii) multimedia content delivery (support of edge processing and virtualized network functions for video networking); (iii) multimedia content consumption and end-to-end aspects (player-triggered segment retransmissions to improve video playout quality); and (iv) novel QoE investigations (adaptive point cloud streaming). We will also put the work into the context of international multimedia systems research.

Biography: Christian Timmerer is a full professor of computer science at Alpen-Adria-Universität Klagenfurt (AAU), Institute of Information Technology (ITEC) and he is the director of the Christian Doppler (CD) Laboratory ATHENA (https://athena.itec.aau.at/). His research interests include multimedia systems, immersive multimedia communication, streaming, adaptation, and quality of experience where he co-authored seven patents and more than 300 articles. He was the general chair of WIAMIS 2008, QoMEX 2013, MMSys 2016, and PV 2018 and has participated in several EC-funded projects, notably DANAE, ENTHRONE, P2P-Next, ALICANTE, SocialSensor, COST IC1003 QUALINET, ICoSOLE, and SPIRIT. He also participated in ISO/MPEG work for several years, notably in the area of MPEG-21, MPEG-M, MPEG-V, and MPEG-DASH where he also served as standard editor. In 2012 he cofounded Bitmovin (http://www.bitmovin.com/) to provide professional services around MPEG-DASH where he holds the position of the Chief Innovation Officer (CIO) –- Head of Research and Standardization. Further information at http://timmerer.com.

Posted in News | Comments Off on HTTP Adaptive Streaming – Quo Vadis? (2023)

MTAP: Performance Analysis of H2BR: HTTP/2-based Segment Upgrading to Improve the QoE in HAS

Multimedia Tools and Applications


Minh Nguyen, Hadi Amirpour, Farzad Tashtarian, Christian Timmerer and Hermann Hellwagner

(Alpen-Adria-Universität Klagenfurt)

Abstract: HTTP Adaptive Streaming (HAS) plays a key role in over-the-top video streaming with the ability to reduce the video stall duration by adapting the quality of transmitted video segments to the network conditions. However, HAS still suffers from two problems. First, it incurs variations in video quality because of throughput fluctuation. Adaptive bitrate (ABR) algorithms at the HAS client usually select a low-quality segment when the throughput drops to avoid stall events, which impairs the Quality of Experience (QoE) of the end-users. Second, many ABR algorithms choose the lowest-quality segments at the beginning of a video streaming session to ramp up the playout buffer early on. Although this strategy decreases the startup time, clients can be annoyed as they have to watch a low-quality video initially.

To address these issues, we introduced the H2BR technique (HTTP/2-Based Retransmission) that utilizes certain features of HTTP/2 (including server push, multiplexing, stream priority, and stream termination) for late transmissions of higher-quality versions of video segments already in the client buffer, in order to improve video quality. Although H2BR was shown to enhance the QoE, limited streaming scenarios were considered resulting in a lack of general conclusions on H2BR’s performance. Thus, this article provides a profound evaluation to answer three open questions: (i) how H2BR’s performance is impacted by parameters at the server side (i.e., various encoding specifications), at the network side (i.e., packet loss rate), and at the client side (i.e., buffer size) on the performance of H2BR; (ii) how H2BR outperforms other state-of-the-art approaches in different configurations of the parameters above; (iii) how to effectively utilize H2BR on top of ABR algorithms in various streaming scenarios.

The experimental results show that H2BR’s performance increases with the buffer size and decreases with increasing packet loss rates and/or video segment duration. The number of quality levels can negatively or positively impact on H2BR’s performance, depending on the ABR algorithm deployed. In general, H2BR is able to enhance the video quality by up to and 14% in scalablevideo streaming and in non-scalable video streaming, respectively. Compared with an existing retransmission technique (i.e., SQUAD), H2BR shows better results with more than 10% in QoE and 9% in the average video quality.

Keywords: HTTP adaptive streaming, DASH, Retransmission, QoE, HTTP/2, H2BR

Posted in News | Comments Off on MTAP: Performance Analysis of H2BR: HTTP/2-based Segment Upgrading to Improve the QoE in HAS

Green video complexity analysis for efficient encoding in Adaptive Video Streaming

GMSys 2023: First International ACM Green Multimedia Systems Workshop

7 – 10 June 2023 | Vancouver, Canada

Conference Website

[PDF] [Slides]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Christian Feldmann (Bitmovin, Klagenfurt), Klaus Schoeffmann (Alpen-Adria-Universität Klagenfurt), Mohammed Ghanbari (University of Essex),  and Christian Timmerer (Alpen-Adria-Universität Klagenfurt).


For adaptive streaming applications, low-complexity and accurate video complexity features are necessary to analyze the video content in real time, which ensures fast and compression-efficient video streaming without disruptions. The popular state-of-the-art video complexity features are Spatial Information (SI) and Temporal Information (TI) features which do not correlate well with the encoding parameters in adaptive streaming applications. To this light, Video Complexity Analyzer (VCA) was introduced, determining the features based on Discrete Cosine Transform (DCT)-energy. This paper presents optimizations on VCA for faster and energy-efficient video complexity analysis. Experimental results show that VCAv2.0, using eight CPU threads, Single Instruction Multiple Data (SIMD), and low-pass DCT optimization determines seven complexity features of Ultra High Definition 8-bit videos with better accuracy at a speed of 292.68 fps and an energy consumption of 97.06% lower than the reference SITI implementation.

Content-adaptive encoding framework using video content complexity analysis.

Posted in GAIA, News | Comments Off on Green video complexity analysis for efficient encoding in Adaptive Video Streaming

IXR’23: Interactive eXtended Reality 2023

IXR’23: Interactive eXtended Reality 2023

colocated with ACM Multimedia 2023

October 2023, Ottawa, Canada

Workshop Chairs:

  • Irene Viola, CWI, Netherlands
  • Hadi Amirpour, Klagenfurt University, Austria
  • Stephanie Arévalo Arboleda, TUIlmenau , Germany
  • Maria Torres Vega, Ghent University, Belgium

Topics of interest include, but are not limited to:

  • Novel low latency encoding techniques for interactive XR applications
  • Novel networking systems and protocols to enable interactive immersive applications. This includes optimizations ranging from hardware (i.e., millimeter-wave networks or optical wireless), physical and MAC layer up to the network, transport and application layers (such as over the top protocols);
  • Significative advances and optimization in 3D modeling pipelines for AR/VR visualization, accessible and inclusive GUI, interactive 3D models;
  • Compression and delivery strategies for immersive media contents, such as omnidirectional video, light fields, point clouds, dynamic and time varying meshes;
  • Quality of Experience management of interactive immersive media applications;
  • Novel rendering techniques to enhance interactivity of XR applications;
  • Application of interactive XR to different areas of society, such as health (i.e., virtual reality exposure therapy), industry (Industry 4.0), XR e-learning (according to new global aims);


  • Submission deadline: 05 July 2023, 23:59 AoE
  • Notifications of acceptance: 30 July 2023
  • Camera ready submission: 06 August 2023
  • Workshop: 29th October to 3rd November
Posted in News | Comments Off on IXR’23: Interactive eXtended Reality 2023