JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming

Posted on June 21, 2023 by

IEEE Transactions on Circuits and Systems for Video Technology (IEEE TCSVT)

Journal Website

[PDF]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Prajit T Rajendran (Universite Paris-Saclay, France), Christian Feldmann (Bitmovin), Klaus Schoeffmann (Alpen-Adria-Universität Klagenfurt), Mohammad Ghanbari (University of Essex, UK), and Christian Timmerer (Alpen-Adria-Universität Klagenfurt)

Abstract: Adaptive live video streaming applications utilize a predefined collection of bitrate-resolution pairs, known as a bitrate ladder, for simplicity and efficiency, eliminating the need for additional run-time to determine the optimal pairs during the live streaming session. These applications do not incorporate two-pass encoding methods due to increased latency. However, an optimized bitrate ladder could result in lower storage and delivery costs and improved Quality of Experience (QoE). This paper presents a Just Noticeable Difference (JND)-aware con-strained Variable Bitrate (cVBR) Two-pass Per-title encoding Scheme (JTPS) designed specifically for live video streaming. JTPS predicts a content- and JND-aware bitrate ladder using low-complexity features based on Discrete Cosine Transform (DCT) energy and optimizes the constant rate factor (CRF) for each representation using random forest-based models. The effectiveness of JTPS is demonstrated using the open source video encoder x265, with an average bitrate reduction of 18.80% and 32.59% for the same PSNR and VMAF, respectively, compared to the standard HTTP Live Streaming (HLS) bitrate ladder using Constant Bitrate (CBR) encoding. The implementation of JTPS also resulted in a 68.96% reduction in storage space and an 18.58% reduction in encoding time for a JND of six VMAF points.

Live HTTP adaptive streaming featuring our JND-aware two-pass per-title encoding scheme (JTPS).

Posted in ATHENA, News | Comments Off on JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming

Report on GMSys 2023: First International ACM Green Multimedia Systems Workshop

Posted on June 19, 2023 by

The first International ACM Green Multimedia Systems Workshop, hosted and organized as part of the 14th ACM Multimedia Systems Conference, took place on Saturday, 10 June 2023 in the beautiful city of Vancouver, Canada. This workshop served as a crucial forum for researchers in multimedia systems to present and share their latest research findings, with a specific focus on energy consumption and greenhouse gas emissions in multimedia systems.

The workshop featured eight high-quality technical presentations, with an impressive acceptance rate of 66.7%. These presentations included both full papers and short papers, showcasing a diverse range of innovative approaches and solutions related to green video streaming. It was a significant opportunity for multimedia researchers to come together and delve into this timely and important topic.

The workshop was the participation of  experts and researchers in the field of green video streaming. Researchers from universities including Fraunhofer FOKUS, Friedrich-Alexander-Universität Erlangen-Nürnberg, and university of Klagenfurt presented their research papers, shared their valuable insights and research findings, further enriching the discussions. Researchers from leading companies such as Synamedia, Amazon, and Ateme also presented their research papers and  actively participated in the workshop.

Presentations in GMSys: 

VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing Instances
Samira Afzal, Narges Mehran (Alpen-Adria-Universität Klagenfurt, Austria); Sandro Linder (Bitmovin, Klagenfurt, Austria); Christian Timmerer, Radu Prodan (Alpen-Adria-Universität Klagenfurt, Austria)

Studying Green Video Distribution as a Whole
Burak Kara, Gwendal Simon (Synamedia); Bruno Tuffin (Inria); Jerome Vieron (Synamedia); Ali C. Begen (Ozyegin University)

End-to-end Optimizations for Green Streaming
Robert Seeliger, Stefan Pham, Stefan Arbanowski (Fraunhofer FOKUS)

Audience Aware Streaming: New Dynamics in OTT distribution
Jan Outters, Mickael Raulet (Ateme S.A.)

Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 
Vignesh V Menon (Alpen-Adria-Universität Klagenfurt); Christian Feldmann (Bitmovin); Klaus Schoeffmann (Alpen-Adria-Universität Klagenfurt); Mohammed Ghanbari (University of Essex); Christian Timmerer (Alpen-Adria Universität Klagenfurt)

Energy Efficiency Improvements in Software-Based Video Encoding
Jan De Cock (Synamedia)

Video Decoding Energy Reduction using Temporal-Domain Filtering
Christian Herglotz, Matthias Kränzler, Robert Ludwig, André Kaup (Friedrich-Alexander-Universität Erlangen-Nürnberg)

The analysis of DASH manifest optimizations
Yongjun Wu (Amazon)

The gathering of researchers, industry professionals, and experts in the field showcased the increasing importance and impact of green video streaming. The workshop served as a catalyst for knowledge dissemination, collaboration, and the development of sustainable solutions in multimedia systems. It significantly contributed to advancing the understanding and development of green video streaming technologies, while addressing the environmental impact of digital media consumption.

Acknowledgments:
We would like to express our heartfelt gratitude to all the workshop participants, speakers, technical program committees, authors, and attendees who played a crucial role in the success of this event. Special thanks go to the ACM organizing committee, particularly Mohamed Hefeeda and Shervin Shirmohammadi, for their invaluable support in providing a platform to host this significant workshop. Additionally, we extend our appreciation to GAIA for their generous technical sponsorship, which greatly contributed to the smooth organization and execution of the workshop.

Posted in GAIA | Comments Off on Report on GMSys 2023: First International ACM Green Multimedia Systems Workshop

QoMEX’23: Are Quality and Sustainability Reconcilable? A Subjective Study on Video QoE, Luminance and Resolution

Posted on June 19, 2023 by

The 15th International Conference on Quality of Multimedia Experience (QoMEX)

June 20-22, 2023 – Ghent, Belgium

https://qomex2023.itec.aau.at/

[PDF][Slides]

*** Diversity and Societal Impact Award ***

Gülnaziye Bingöl (University of Cagliari), Alessandro Floris (University of Cagliari), Simone Porcu (University of Cagliari), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), and Luigi Atzori (University of Cagliari)

Abstract: The increasing use of ICT has raised concerns about its negative impact on energy consumption and CO2 emissions. To address this issue, there is a need to better understand the trade-off between Quality of Experience (QoE) and sustainable video streaming services. In this study, we designed and conducted a subjective assessment to investigate the impact of video resolution, different types of luminance, and different end devices on the QoE and energy consumption of video streaming services. Then, we applied statistical models (Analysis of Variance and t-test) to subjective data to find out what factors influence the QoE the most and consume more energy. The obtained results suggest that under specific conditions (e.g., dark or bright ambient, low device backlight luminance, small-screen device) the users could be encouraged towards a trade-off between acceptable QoE and sustainable (green) choices because spending more energy (e.g., streaming higher-quality video) would not provide noticeable QoE enhancement.

Index Terms—Quality of Experience, Video Streaming, Sustainability, Luminance, Resolution.

 

Posted in ATHENA, GAIA | Comments Off on QoMEX’23: Are Quality and Sustainability Reconcilable? A Subjective Study on Video QoE, Luminance and Resolution

QoMEX’23: Impact of Quality and Distance on the Perception of Point Clouds in Mixed Reality

Posted on June 19, 2023 by

The 15th International Conference on Quality of Multimedia Experience (QoMEX)

June 20-22, 2023 – Ghent, Belgium

https://qomex2023.itec.aau.at/

[PDF][Slides]

Minh Nguyen (Alpen-Adria-Universität Klagenfurt), Shivi Vats (Alpen-Adria-Universität Klagenfurt), Sam Van Damme (Ghent University – imec), Jeroen van der Hooft (Ghent University – imec), Maria Torres Vega (Ghent University – imec), Tim Wauters (Ghent University – imec), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)

Abstract: Point Cloud (PC) streaming has recently attracted research attention as it has the potential to provide six degrees of freedom (6DoF), which is essential for truly immersive media. PCs require high-bandwidth connections, and adaptive streaming is a promising solution to cope with fluctuating bandwidth conditions. Thus, understanding the impact of different factors in adaptive streaming on the Quality of Experience (QoE) becomes fundamental. Mixed Reality (MR) is a novel technology and has recently become popular. However, quality evaluations of PCs in MR environments are still limited to static images. In this paper, we perform a subjective study on four impact factors on the QoE of PC video sequences in MR conditions, including quality switches, viewing distance, and content characteristics. The experimental results show that these factors significantly impact QoE. The QoE decreases if the sequence switches to lower quality and/or is viewed at a shorter distance, and vice versa. Additionally, the end user might not distinguish the quality differences between two quality levels at a specific viewing distance. Regarding content characteristics, objects with lower contrast seem to provide better quality scores.

Index Terms—Point Clouds, Quality of Experience, Subjective Tests, Mixed Reality

Posted in SPIRIT | Comments Off on QoMEX’23: Impact of Quality and Distance on the Perception of Point Clouds in Mixed Reality

QoMEX’23: A Platform for Subjective Quality Assessment in Mixed Reality Environments

Posted on June 19, 2023 by

The 15th International Conference on Quality of Multimedia Experience (QoMEX)

June 20-22, 2023 – Ghent, Belgium

https://qomex2023.itec.aau.at/

[PDF][Slides]

Minh Nguyen (Alpen-Adria-Universität Klagenfurt), Shivi Vats (Alpen-Adria-Universität Klagenfurt), Sam Van Damme (Ghent University – imec), Jeroen van der Hooft (Ghent University – imec), Maria Torres Vega (Ghent University – imec), Tim Wauters (Ghent University – imec), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)

Abstract: 3D objects are important components in Mixed Reality (MR) environments as they allow users to inspect and interact with them in a six degrees of freedom (6DoF) system. Point clouds (PCs) and meshes are two common 3D object representations that can be compressed to reduce the delivered data at the cost of quality degradation. In addition, as the end users can move around in 6DoF applications, the viewing distance can vary. Quality assessment is necessary to evaluate the impact of the compressed representation and viewing distance on the Quality of Experience (QoE) of end users. This paper presents a demonstrator for subjective quality assessment of dynamic PC and mesh objects under different conditions in MR environments. Our platform allows conducting subjective tests to evaluate various QoE influence factors, including encoding parameters, quality switching, viewing distance, and content characteristics, with configurable settings for these factors.

Index Terms—Point Clouds, Subjective Test Platform, Quality of Experience, Mixed Reality

Posted in SPIRIT | Comments Off on QoMEX’23: A Platform for Subjective Quality Assessment in Mixed Reality Environments

QoE- and Energy-aware Content Consumption For HTTP Adaptive Streaming

Posted on June 10, 2023 by

14th ACM Multimedia Systems Conference (MMSys)

7 – 10 June 2023 | Vancouver, BC, Canada

Conference Website

[PDF][Slides][Poster]

Daniele Lorenzi (AAU, Austria)

Abstract: Video streaming services account for the majority of today’s traffic on the Internet, and according to recent studies, this share is expected to continue growing. Given this broad utilization, research in video streaming is recently moving towards energy-aware approaches, which aim at reducing the energy consumption of the devices involved in the streaming process. On the other side, the perception of quality delivered to the user plays an important role, and the advent of HTTP Adaptive Streaming (HAS) changed the way quality is perceived. The focus is not any more exclusively on the Quality of Service (QoS) but rather oriented towards the Quality of Experience (QoE) of the user taking part in the streaming session. Therefore video streaming services need to develop Adaptive BitRate (ABR) techniques to deal with different network conditions on the client side or appropriate end-to-end strategies to provide high QoE to the users. The scope of this doctoral study is within the end-to-end environment with a focus on the end-users domain, referred to as the player environment, including video content consumption and interactivity. This thesis aims to investigate and develop different techniques to increase the delivered QoE to the users and minimize the energy consumption of the end devices in HAS context. We present four main research questions to target the related challenges in the domain of content consumption for HAS systems.

Keywords: Multi-codec, HTTP/3, machine learning, green computing, HAS

Posted in ATHENA | Comments Off on QoE- and Energy-aware Content Consumption For HTTP Adaptive Streaming

Just Noticeable Difference-aware Per-Scene Bitrate-laddering for Adaptive Video Streaming

Posted on June 6, 2023 by

IEEE International Conference on Multimedia and Expo (ICME), 2023

10-14 July 2023, Brisbane, Australia

[PDF] [Slides]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Jingwen Zhu (Nantes Universite), Prajit T Rajendran (Universite Paris-Saclay), Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Patrick Le Callet (Nantes Universite),  and Christian Timmerer (Alpen-Adria-Universität Klagenfurt).

Abstract:

In video streaming applications, a fixed set of bitrate-resolution pairs (known as a bitrate ladder) is typically used during the entire streaming session. However, an optimized bitrate ladder per scene may result in (i) decreased storage or delivery costs or/and (ii) increased Quality of Experience. This paper introduces a Just Noticeable Difference (JND)-aware per-scene bitrate ladder prediction scheme (JASLA) for adaptive video-on-demand streaming applications. JASLA predicts jointly optimized resolutions and corresponding constant rate factors (CRFs) using spatial and temporal complexity features for a given set of target bitrates for every scene, which yields an efficient constrained Variable Bitrate encoding. Moreover, bitrate-resolution pairs that yield distortion lower than one JND are eliminated. Experimental results show that, on average, JASLA yields bitrate savings of 34.42% and 42.67% to maintain the same PSNR and VMAF, respectively, compared to the reference HTTP Live Streaming (HLS) bitrate ladder Constant Bitrate encoding using x265 HEVC encoder, where the maximum resolution of streaming is Full HD (1080p). Moreover, a 54.34% average cumulative decrease in storage space is observed.

JASLA architecture.

Posted in ATHENA | Comments Off on Just Noticeable Difference-aware Per-Scene Bitrate-laddering for Adaptive Video Streaming