Samira Afzal talk at 10th Fraunhofer FOKUS MWS

Exploring the Energy Consumption of Video Streaming: Components, Challenges, and Opportunities

10th FOKUS Media Web Symposium

June 13–14, 2023 – Berlin, Germany

 

Abstract: The rapid growth of video streaming usage is a significant source of energy consumption, driven by improved internet connections and service offerings, the quick development of video entertainment, the deployment of Ultra High-Definition, Virtual and Augmented Reality, as well as an increasing number of video surveillance and IoT applications. However, it is essential to note that these advancements come at the cost of energy consumption. To address this challenge, it is essential to understand the various components involved in energy consumption during video streaming, ranging from video encoding to decoding and displaying the video on the end user’s screen. Then, it is critical to accurately measure energy consumption for each component and conduct an in-depth analysis to develop energy-efficient strategies that optimize video streaming. I categorize these components into three categories: (i) data centers, (ii) networks, and (iii) end-user devices.

Data centers: Data centers are responsible for the significant growth in video data traffic, which is estimated to reach over 1,000 TWh of power consumption by 2025. To effectively manage energy consumption in data centers, it is crucial to understand the various components that contribute to it, including encoding process and parameters, resource provisioning, core network, storage, as well as hardware aspects such as cloud platform features and hardware units (e.g., CPU, GPU). By analyzing and optimizing these energy-intensive components, energy-efficient strategies, such as energy management and task schedulers, and energy-efficient codecs can be developed to improve the sustainability of data centers.

Networks: The next category within video streaming is the transmission of video from data centers to end devices through heterogeneous networks. The network energy-intensive components are CDNs, routers, switches, and network channels. To optimize energy efficiency during video transmission, it is essential to manage and optimize energy consumption in each of these components. This can be achieved by implementing efficient routing algorithms, reducing data redundancy, and utilizing power-saving mechanisms in network devices.

End-user devices: The last category of the video streaming is video usage at the end-user device, which has been shown to account for the majority of energy consumption by the decoding hardware and end-user devices. Energy consumption in this category is due to the components such as decoding, players, browsers, codecs, operating systems, and hardware (e.g., CPU, display). Improving the energy efficiency of end-user devices can significantly reduce energy consumption in video streaming. Some examples of achieving this are through the use of more energy-efficient devices such as laptops, tablets, and smartphones or by improving screen display technologies.

In addition to identifying the primary components of video streaming that affect energy consumption, it is important to conduct a comprehensive analysis of the entire video streaming. It is also important to strike a balance between energy optimization and service quality to ensure that energy-efficient strategies are implemented without sacrificing the quality of video streaming services.

In this talk, my objective is to provide insights into the components of video streaming that contribute to energy consumption and highlight the challenges associated with measuring their energy usage. I will also introduce the tools that can be used for energy measurements for those components and the possible and associated strategies that lie within energy efficiency. By accurately measuring energy consumption, digital media companies can effectively monitor and control their energy usage, ultimately leading to cost savings and improved sustainability.

Posted in GAIA | Comments Off on Samira Afzal talk at 10th Fraunhofer FOKUS MWS

All-intra rate control using low complexity video features for Versatile Video Coding

IEEE International Conference on Image Processing (ICIP), 2023

8-11 October 2023, Kuala Lumpur, Malaysia

Conference Website

[PDF]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Anastasia Henkel (Fraunhofer HHI Berlin), Prajit T Rajendran (Universite Paris-Saclay), Christian R Helmich (Fraunhofer HHI Berlin), Adam Wieckowski (Fraunhofer HHI Berlin), Benjamin Bross (Fraunhofer HHI Berlin), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), and Detlev Marpe (Fraunhofer HHI Berlin).

Abstract:

Versatile Video Coding (VVC) allows for large compression efficiency gains over its predecessor, High Efficiency Video Coding (HEVC). The added efficiency comes at the cost
of increased runtime complexity, especially for encoding. It is thus highly relevant to explore all available runtime reduction options. This paper proposes a novel first pass for
two-pass rate control in all-intra configuration, using low-complexity video analysis and a Random Forest (RF)-based machine learning model to derive the data required for driving the second pass. The proposed method is validated using VVenC, an open and optimized VVC encoder. Compared to the default two-pass rate control algorithm in VVenC, the
proposed method achieves around 32% reduction in encoding time for the preset faster, while on average only causing a 2% BD-rate increase and achieving similar rate control accuracy.

Proposed two-pass rate control encoding architecture.

Posted in ATHENA | Comments Off on All-intra rate control using low complexity video features for Versatile Video Coding

JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming

IEEE Transactions on Circuits and Systems for Video Technology (IEEE TCSVT)

Journal Website

[PDF]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Prajit T Rajendran (Universite Paris-Saclay, France), Christian Feldmann (Bitmovin), Klaus Schoeffmann (Alpen-Adria-Universität Klagenfurt), Mohammad Ghanbari (University of Essex, UK), and Christian Timmerer (Alpen-Adria-Universität Klagenfurt)

Abstract: Adaptive live video streaming applications utilize a predefined collection of bitrate-resolution pairs, known as a bitrate ladder, for simplicity and efficiency, eliminating the need for additional run-time to determine the optimal pairs during the live streaming session. These applications do not incorporate two-pass encoding methods due to increased latency. However, an optimized bitrate ladder could result in lower storage and delivery costs and improved Quality of Experience (QoE). This paper presents a Just Noticeable Difference (JND)-aware con-strained Variable Bitrate (cVBR) Two-pass Per-title encoding Scheme (JTPS) designed specifically for live video streaming. JTPS predicts a content- and JND-aware bitrate ladder using low-complexity features based on Discrete Cosine Transform (DCT) energy and optimizes the constant rate factor (CRF) for each representation using random forest-based models. The effectiveness of JTPS is demonstrated using the open source video encoder x265, with an average bitrate reduction of 18.80% and 32.59% for the same PSNR and VMAF, respectively, compared to the standard HTTP Live Streaming (HLS) bitrate ladder using Constant Bitrate (CBR) encoding. The implementation of JTPS also resulted in a 68.96% reduction in storage space and an 18.58% reduction in encoding time for a JND of six VMAF points.

Live HTTP adaptive streaming featuring our JND-aware two-pass per-title encoding scheme (JTPS).

Posted in ATHENA, News | Comments Off on JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming

Report on GMSys 2023: First International ACM Green Multimedia Systems Workshop

The first International ACM Green Multimedia Systems Workshop, hosted and organized as part of the 14th ACM Multimedia Systems Conference, took place on Saturday, 10 June 2023 in the beautiful city of Vancouver, Canada. This workshop served as a crucial forum for researchers in multimedia systems to present and share their latest research findings, with a specific focus on energy consumption and greenhouse gas emissions in multimedia systems.

The workshop featured eight high-quality technical presentations, with an impressive acceptance rate of 66.7%. These presentations included both full papers and short papers, showcasing a diverse range of innovative approaches and solutions related to green video streaming. It was a significant opportunity for multimedia researchers to come together and delve into this timely and important topic.

The workshop was the participation of  experts and researchers in the field of green video streaming. Researchers from universities including Fraunhofer FOKUS, Friedrich-Alexander-Universität Erlangen-Nürnberg, and university of Klagenfurt presented their research papers, shared their valuable insights and research findings, further enriching the discussions. Researchers from leading companies such as Synamedia, Amazon, and Ateme also presented their research papers and  actively participated in the workshop.

Presentations in GMSys: 

VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing Instances
Samira Afzal, Narges Mehran (Alpen-Adria-Universität Klagenfurt, Austria); Sandro Linder (Bitmovin, Klagenfurt, Austria); Christian Timmerer, Radu Prodan (Alpen-Adria-Universität Klagenfurt, Austria)

Studying Green Video Distribution as a Whole
Burak Kara, Gwendal Simon (Synamedia); Bruno Tuffin (Inria); Jerome Vieron (Synamedia); Ali C. Begen (Ozyegin University)

End-to-end Optimizations for Green Streaming
Robert Seeliger, Stefan Pham, Stefan Arbanowski (Fraunhofer FOKUS)

Audience Aware Streaming: New Dynamics in OTT distribution
Jan Outters, Mickael Raulet (Ateme S.A.)

Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 
Vignesh V Menon (Alpen-Adria-Universität Klagenfurt); Christian Feldmann (Bitmovin); Klaus Schoeffmann (Alpen-Adria-Universität Klagenfurt); Mohammed Ghanbari (University of Essex); Christian Timmerer (Alpen-Adria Universität Klagenfurt)

Energy Efficiency Improvements in Software-Based Video Encoding
Jan De Cock (Synamedia)

Video Decoding Energy Reduction using Temporal-Domain Filtering
Christian Herglotz, Matthias Kränzler, Robert Ludwig, André Kaup (Friedrich-Alexander-Universität Erlangen-Nürnberg)

The analysis of DASH manifest optimizations
Yongjun Wu (Amazon)

The gathering of researchers, industry professionals, and experts in the field showcased the increasing importance and impact of green video streaming. The workshop served as a catalyst for knowledge dissemination, collaboration, and the development of sustainable solutions in multimedia systems. It significantly contributed to advancing the understanding and development of green video streaming technologies, while addressing the environmental impact of digital media consumption.

Acknowledgments:
We would like to express our heartfelt gratitude to all the workshop participants, speakers, technical program committees, authors, and attendees who played a crucial role in the success of this event. Special thanks go to the ACM organizing committee, particularly Mohamed Hefeeda and Shervin Shirmohammadi, for their invaluable support in providing a platform to host this significant workshop. Additionally, we extend our appreciation to GAIA for their generous technical sponsorship, which greatly contributed to the smooth organization and execution of the workshop.

Posted in GAIA | Comments Off on Report on GMSys 2023: First International ACM Green Multimedia Systems Workshop

QoMEX’23: Are Quality and Sustainability Reconcilable? A Subjective Study on Video QoE, Luminance and Resolution

The 15th International Conference on Quality of Multimedia Experience (QoMEX)

June 20-22, 2023 – Ghent, Belgium

https://qomex2023.itec.aau.at/

[PDF][Slides]

*** Diversity and Societal Impact Award ***

Gülnaziye Bingöl (University of Cagliari), Alessandro Floris (University of Cagliari), Simone Porcu (University of Cagliari), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), and Luigi Atzori (University of Cagliari)

Abstract: The increasing use of ICT has raised concerns about its negative impact on energy consumption and CO2 emissions. To address this issue, there is a need to better understand the trade-off between Quality of Experience (QoE) and sustainable video streaming services. In this study, we designed and conducted a subjective assessment to investigate the impact of video resolution, different types of luminance, and different end devices on the QoE and energy consumption of video streaming services. Then, we applied statistical models (Analysis of Variance and t-test) to subjective data to find out what factors influence the QoE the most and consume more energy. The obtained results suggest that under specific conditions (e.g., dark or bright ambient, low device backlight luminance, small-screen device) the users could be encouraged towards a trade-off between acceptable QoE and sustainable (green) choices because spending more energy (e.g., streaming higher-quality video) would not provide noticeable QoE enhancement.

Index Terms—Quality of Experience, Video Streaming, Sustainability, Luminance, Resolution.

 

Posted in ATHENA, GAIA | Comments Off on QoMEX’23: Are Quality and Sustainability Reconcilable? A Subjective Study on Video QoE, Luminance and Resolution

QoMEX’23: Impact of Quality and Distance on the Perception of Point Clouds in Mixed Reality

The 15th International Conference on Quality of Multimedia Experience (QoMEX)

June 20-22, 2023 – Ghent, Belgium

https://qomex2023.itec.aau.at/

[PDF][Slides]

Minh Nguyen (Alpen-Adria-Universität Klagenfurt), Shivi Vats (Alpen-Adria-Universität Klagenfurt), Sam Van Damme (Ghent University – imec), Jeroen van der Hooft (Ghent University – imec), Maria Torres Vega (Ghent University – imec), Tim Wauters (Ghent University – imec), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)

Abstract: Point Cloud (PC) streaming has recently attracted research attention as it has the potential to provide six degrees of freedom (6DoF), which is essential for truly immersive media. PCs require high-bandwidth connections, and adaptive streaming is a promising solution to cope with fluctuating bandwidth conditions. Thus, understanding the impact of different factors in adaptive streaming on the Quality of Experience (QoE) becomes fundamental. Mixed Reality (MR) is a novel technology and has recently become popular. However, quality evaluations of PCs in MR environments are still limited to static images. In this paper, we perform a subjective study on four impact factors on the QoE of PC video sequences in MR conditions, including quality switches, viewing distance, and content characteristics. The experimental results show that these factors significantly impact QoE. The QoE decreases if the sequence switches to lower quality and/or is viewed at a shorter distance, and vice versa. Additionally, the end user might not distinguish the quality differences between two quality levels at a specific viewing distance. Regarding content characteristics, objects with lower contrast seem to provide better quality scores.

Index Terms—Point Clouds, Quality of Experience, Subjective Tests, Mixed Reality

Posted in SPIRIT | Comments Off on QoMEX’23: Impact of Quality and Distance on the Perception of Point Clouds in Mixed Reality

QoMEX’23: A Platform for Subjective Quality Assessment in Mixed Reality Environments

The 15th International Conference on Quality of Multimedia Experience (QoMEX)

June 20-22, 2023 – Ghent, Belgium

https://qomex2023.itec.aau.at/

[PDF][Slides]

Minh Nguyen (Alpen-Adria-Universität Klagenfurt)Shivi Vats (Alpen-Adria-Universität Klagenfurt), Sam Van Damme (Ghent University – imec), Jeroen van der Hooft (Ghent University – imec), Maria Torres Vega (Ghent University – imec), Tim Wauters (Ghent University – imec), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)

Abstract: 3D objects are important components in Mixed Reality (MR) environments as they allow users to inspect and interact with them in a six degrees of freedom (6DoF) system. Point clouds (PCs) and meshes are two common 3D object representations that can be compressed to reduce the delivered data at the cost of quality degradation. In addition, as the end users can move around in 6DoF applications, the viewing distance can vary. Quality assessment is necessary to evaluate the impact of the compressed representation and viewing distance on the Quality of Experience (QoE) of end users. This paper presents a demonstrator for subjective quality assessment of dynamic PC and mesh objects under different conditions in MR environments. Our platform allows conducting subjective tests to evaluate various QoE influence factors, including encoding parameters, quality switching, viewing distance, and content characteristics, with configurable settings for these factors.

Index Terms—Point Clouds, Subjective Test Platform, Quality of Experience, Mixed Reality

Posted in SPIRIT | Comments Off on QoMEX’23: A Platform for Subjective Quality Assessment in Mixed Reality Environments