ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming

18th USENIX Symposium on Networked Systems Design and Implementation

April 16–18, 2024 | Santa Clara, CA, USA

Conference website

[PDF]

Farzad Tashtarian (AAU, Austria),  Abdelhak Bentaleb (Concordia University), Hadi Amirpour (AAU, Austria)Sergey Gorinsky (IMDEA Networks Institute),  Junchen Jiang (University of Chicago), Hermann Hellwagner (AAU, Austria)Christian Timmerer (AAU, Austria)

Live streaming of segmented videos over the Hypertext Transfer Protocol (HTTP) is increasingly popular and serves heterogeneous clients by offering each segment in multiple representations. A bitrate ladder expresses this choice as an ordered list of bitrate-resolution pairs. Whereas existing solutions for HTTP-based live streaming use a static bitrate ladder, the fixed ladders struggle to appropriately accommodate the dynamics in the video content and network-conditioned client capabilities. This paper proposes ARTEMIS as a practical scalable alternative that dynamically configures the bitrate ladder depending on the content complexity, network conditions, and clients’ statistics. ARTEMIS seamlessly integrates with the end-to-end streaming pipeline and operates transparently to video encoders and clients. We develop a cloud-based implementation of ARTEMIS and conduct extensive real-world and trace-driven experiments. The experimental comparison vs. existing prominent bitrate ladders demonstrates that live streaming with ARTEMIS outperforms all baselines, reduces encoding computation by 25%, end-to-end latency by 18%, and increases quality of experience (QoE) by 11%.

Posted in ATHENA | Comments Off on ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming

Grand Challenges in Green Multimedia Signal Processing

Grand Challenges in Green Multimedia Signal Processing at the IEEE 25th International Workshop on Multimedia Signal Processing (MMSP)

https://attend.ieee.org/mmsp-2023/call-for-grand-challenges/

It’s time to take action against the threat of climate change by making significant changes to our global greenhouse gas (GHG) emissions. That includes rethinking how we consume energy for digital technologies. Did you know that video streaming technology alone is responsible for over half of digital technology’s global impact? With the rise of digital and remote work becoming more common, there’s been a rapid increase in video data volume, processing, and streaming. Unfortunately, this also means an increase in energy consumption and GHG emissions. But with thoughtful and positive efforts, we can make a difference and reduce our impact on the environment. Let’s do this!

We are thrilled to invite experts and researchers to join us for the Grand Challenges in Green Multimedia Signal Processing at the IEEE 25th International Workshop on Multimedia Signal Processing (MMSP)! This is an exciting opportunity to explore the latest developments and challenges in reducing energy consumption in multimedia systems. Our session is dedicated to sharing innovative concepts and energy-efficient solutions across the entire spectrum of video generation, processing, delivery, and usage. Let’s work together to make a positive impact on the environment and multimedia industry!

We’re excited to invite you to submit your awesome proposals for the grand challenges in green multimedia signal processing. We’re looking for innovative solutions and results that can make a real difference in this field. To make the process easier, please follow the regular MMSP paper template with a maximum of four pages including acknowledgments, references, or any additional material you’d like to include. You can rest assured that your submission will be reviewed by top experts in the field, and accepted papers will be featured in the MMSP conference proceedings, which will be included in IEEE Xplore. We can’t wait to see what you’ve got!

Submissions should follow the IEEE template given in the Instructions for authors section.

Timeline:

  • Submission deadline: August 16, 2023 [CMT: new submission under “Grand challenges”]
  • Acceptance Notification: August 30, 2023

Chairs:

  • Samira Afzal, Alpen-Adria-Universität, Austria
  • Cagri Ozcinar, MSK AI, UK
  • Christian Timmerer, Alpen-Adria-Universität, Austria

For any questions regarding the challenge, please send an email to mmsp-2023@univ-poitiers.fr

Posted in ATHENA, GAIA | Comments Off on Grand Challenges in Green Multimedia Signal Processing

ICME’23 Tutorial: Advances in HTTP Adaptive Streaming: New Standards, Video Codecs, and Encoding Optimization Techniques

IEEE ICME 2023
Monday, July 10, 2023
https://www.2023.ieeeicme.org/tutorial-prop.php


Lectures:

  • Hadi Amirpour, AAU, AT
  • Yutiy Reznik, Brightcove, USA
  • Nabajeet Barman,  Brightcove
  • Christian Timmerer, AAU, AT

Abstract: The applications of video streaming are the primary drivers of Internet traffic, as over 82% of IP traffic in 2022. The prevailing technology to stream live and video-on-demand (VoD) content over the Internet is HTTP Adaptive Streaming (HAS). In HAS, the content is encoded at multiple representations (bitrate-resolution pairs), and delivered incrementally using segments (or chunks) of encoded representations. This allows for dynamic selection of different representations during playback, and the HAS model supports delivery under different and changing network- and device-specific conditions. As the demand for video streaming applications is on the rise, improved streaming methods, as well as video codecs and video content optimization algorithms, are being developed to meet this demand. This tutorial first presents a detailed overview of the existing file formats, protocols (HLS, DASH, CMAF), video codecs (H.264, HEVC, AV1, VVC), and quality assessment methods (PSNR, SSIM, VMAF, P.1203, CTA2066) used for streaming. Particular emphasis will be given to new video codecs, such as Versatile Video Coding (VVC), and their features applicable to streaming. We will then focus on recent advances in video encoding optimizations for HAS streaming. We will then introduce per-title, content-aware, and context-aware encoding methods, which optimize bitrate ladders based on the characteristics of videos and other contexts. It will also be presented how representations are selected in a way that bitrate ladders are optimized over various dimensions, including spatial resolution, frame rate, energy consumption, device type, and network condition. Additionally, methods for reducing the latency of dynamic ladder generation and fast multi-rates encoding will be covered. Finally, we will discuss various areas of recent progress in the design of streaming systems, including end-to-end modelling, optimization, and analysis, and highlight remaining open problems in the field.

Posted in ATHENA | Comments Off on ICME’23 Tutorial: Advances in HTTP Adaptive Streaming: New Standards, Video Codecs, and Encoding Optimization Techniques

IEEE Network Special Issue on Advancements in Network-Assisted Video Streaming: Optimization and Performance Analysis

IEEE Network Special Issue on

Advancements in Network-Assisted Video Streaming: Optimization and Performance Analysis

Download CfP

Call for Papers

Network-assisted video streaming has become a substantial part of modern multimedia applications, enabling users to access high-quality video content over different networks, including the Internet and wireless networks. Efficiently delivering video content over networks poses numerous challenges, such as limited bandwidth, varying network conditions, heterogeneous end devices, and diverse user preferences. Network-assisted video streaming approaches leverage modern networking technologies, such as Software-Defined Networking (SDN), Network Function Virtualization (NFV), and edge computing, to not only improve the users’ Quality of Experience (QoE) but also enhance network utilization.

This special issue aims to explore the latest advancements in network-assisted video streaming, with a specific focus on optimization techniques and comprehensive performance analysis, by exploring emerging trends and innovations, including novel approaches that leverage Artificial Intelligence (AI) techniques, machine learning algorithms, and data-driven optimization methods to enhance the streaming experience. Furthermore, contributions related to the integration of edge computing, Virtual Reality (VR), Augmented Reality (AR), and volumetric video streaming will be welcomed. Since understanding the performance of network-assisted video streaming systems is important for assessing their effectiveness and identifying areas for improvement, the research articles should cover both experimental and theoretical aspects, utilizing real-world datasets, simulation frameworks, analytical models, and conducting real-world experiments.

The research and advancements of this special issue will have a significant impact on the design, implementation, and operation of video streaming systems. The findings will provide valuable insights to network operators, content providers, researchers, and developers, enabling them to optimize their systems for enhanced user experience. Additionally, the knowledge gained from this special issue will contribute to the development of standards and best practices for network-assisted video streaming, benefiting the broader multimedia community. IEEE Network, the flagship magazine on networking technologies, is a perfect venue for publishing this special issue. We believe that our special issue covers a number of key aspects and emerging topics that are of interest to the readers of the IEEE Network.

This special issue is to publish original research and review articles that should be comprehensive to all readers of the IEEE Network Magazine, regardless of their specialty. This SI aims to bring together researchers and developers working on all aspects of video streaming, in particular network-assisted concepts backed up by experimental evidence. Potential topics include but are not limited to the following:

  • Design, analysis, and evaluation of network-assisted multimedia system architectures
  • Using AI/ML at the network edge and the cloud for supporting video streaming
  • AI/ML-enabled caching of video chunks
  • Network-assisted/AI-based resource allocation for video streaming
  • Experience and lessons learned by deploying large-scale network-assisted video streaming
  • Internet measurement and modeling for enhancing QoE in video streaming applications
  • Network aspects in video streaming: cloud computing, virtualization techniques, network control, and management, including SDN, NFV, and network programmability
  • Topics at the intersection of energy-efficient computing and networking for video streaming
  • Machine learning for improving QoE in video streaming applications
  • Machine learning for traffic engineering and congestion control for video streaming
  • AI/ML-based solutions for supporting streaming applications’ high-speed user mobility
  • Big data analytics at the network edge to assess viewer experience of adaptive video
  • Advanced network-based techniques for point clouds, light fields, and immersive video
  • Using AI/ML techniques for optimizing Interactive Streaming and User-Generated Content
  • The tradeoff between QoE enhancement and network overhead: AI approaches
  • AI/ML-based techniques for live streaming in 5G and 6G networks

Submission Guidelines

Manuscripts should conform to the standard format as indicated in the “Information for Authors” section of the Paper Submission Guidelines. All manuscripts to be considered for publication must be submitted by the deadline through the magazine’s Manuscript Central submission site. Select “May2024/VideoStreaming” from the drop-down menu of Topic titles.

Important Dates

  • Manuscript Submission Deadline: 31 October 2023
  • Initial Decision Notification: 31 December 2023
  • Revised Manuscript Due: 31 January 2024
  • Final Decision Notification: 28 February 2024
  • Final Manuscript Due: 15 March 2024
  • Publication Date: May/June 2024

Guest Editors

  • Farzad Tashtarian, Universität Klagenfurt, Austria (farzad.tashtarian@aau.at)
  • Yao Liu, Rutgers University, USA (yao.liu@rutgers.edu)
  • Müge Sayıt, Ege Üniversitesi, Turkey (muge.sayit@ege.edu.tr)
  • Junchen Jiang, University of Chicago, USA (junchenj@uchicago.edu)
  • Gwendal Simon, Synamedia, UK (gsimon@synamedia.com)
  • Christian Timmerer, Universität Klagenfurt, Austria (christian.timmerer@aau.at)
Posted in ATHENA | Comments Off on IEEE Network Special Issue on Advancements in Network-Assisted Video Streaming: Optimization and Performance Analysis

Samira Afzal talk at ICT4S Workshop: Towards Sustainable and Energy-Aware Video Communications (SEViC)

Energy Consumption in Video Streaming: Components, Measurements, and Strategies

ICT4S Workshop: Towards Sustainable and Energy-Aware Video Communications (SEViC)

June 5-9 2023 – Rennes, France

 

Abstract: The rapid growth of video streaming usage is a significant source of energy consumption, driven by improved internet connections and service offerings, the quick development of video entertainment, the deployment of Ultra High-Definition, Virtual and Augmented Reality, as well as an increasing number of video surveillance and IoT applications. To address this challenge, it is essential to understand the various components involved in energy consumption during video streaming, ranging from video encoding to decoding and displaying the video on the end user’s screen. Then, it is critical to measure energy consumption for each component accurately and conduct an in-depth analysis to develop energy-efficient strategies that optimize video streaming. These components are classified into three categories: (i) data centers, which include encoding, packaging, and storage on cloud data centers; (ii) networks, which include core network and access networks; and (iii) end-user devices which involve decoding, players, hardware, etc.

In addition to identifying the primary components of video streaming that affect energy consumption, it is important to conduct a comprehensive analysis of the entire video streaming. It is also essential to balance energy optimization and service quality to ensure that energyefficient strategies are implemented without sacrificing the quality of video streaming services.

This talk aims to provide insights into the components of video streaming that contribute to energy consumption and highlight the challenges associated with measuring their energy usage. I will also introduce the tools that can be used for energy measurements for those components and the possible and associated strategies that lie within energy efficiency. By accurately measuring energy consumption, digital media companies can effectively monitor and control their energy usage, ultimately leading to cost savings and improved sustainability.

Posted in GAIA | Comments Off on Samira Afzal talk at ICT4S Workshop: Towards Sustainable and Energy-Aware Video Communications (SEViC)

Samira Afzal talk at 10th Fraunhofer FOKUS MWS

Exploring the Energy Consumption of Video Streaming: Components, Challenges, and Opportunities

10th FOKUS Media Web Symposium

June 13–14, 2023 – Berlin, Germany

 

Abstract: The rapid growth of video streaming usage is a significant source of energy consumption, driven by improved internet connections and service offerings, the quick development of video entertainment, the deployment of Ultra High-Definition, Virtual and Augmented Reality, as well as an increasing number of video surveillance and IoT applications. However, it is essential to note that these advancements come at the cost of energy consumption. To address this challenge, it is essential to understand the various components involved in energy consumption during video streaming, ranging from video encoding to decoding and displaying the video on the end user’s screen. Then, it is critical to accurately measure energy consumption for each component and conduct an in-depth analysis to develop energy-efficient strategies that optimize video streaming. I categorize these components into three categories: (i) data centers, (ii) networks, and (iii) end-user devices.

Data centers: Data centers are responsible for the significant growth in video data traffic, which is estimated to reach over 1,000 TWh of power consumption by 2025. To effectively manage energy consumption in data centers, it is crucial to understand the various components that contribute to it, including encoding process and parameters, resource provisioning, core network, storage, as well as hardware aspects such as cloud platform features and hardware units (e.g., CPU, GPU). By analyzing and optimizing these energy-intensive components, energy-efficient strategies, such as energy management and task schedulers, and energy-efficient codecs can be developed to improve the sustainability of data centers.

Networks: The next category within video streaming is the transmission of video from data centers to end devices through heterogeneous networks. The network energy-intensive components are CDNs, routers, switches, and network channels. To optimize energy efficiency during video transmission, it is essential to manage and optimize energy consumption in each of these components. This can be achieved by implementing efficient routing algorithms, reducing data redundancy, and utilizing power-saving mechanisms in network devices.

End-user devices: The last category of the video streaming is video usage at the end-user device, which has been shown to account for the majority of energy consumption by the decoding hardware and end-user devices. Energy consumption in this category is due to the components such as decoding, players, browsers, codecs, operating systems, and hardware (e.g., CPU, display). Improving the energy efficiency of end-user devices can significantly reduce energy consumption in video streaming. Some examples of achieving this are through the use of more energy-efficient devices such as laptops, tablets, and smartphones or by improving screen display technologies.

In addition to identifying the primary components of video streaming that affect energy consumption, it is important to conduct a comprehensive analysis of the entire video streaming. It is also important to strike a balance between energy optimization and service quality to ensure that energy-efficient strategies are implemented without sacrificing the quality of video streaming services.

In this talk, my objective is to provide insights into the components of video streaming that contribute to energy consumption and highlight the challenges associated with measuring their energy usage. I will also introduce the tools that can be used for energy measurements for those components and the possible and associated strategies that lie within energy efficiency. By accurately measuring energy consumption, digital media companies can effectively monitor and control their energy usage, ultimately leading to cost savings and improved sustainability.

Posted in GAIA | Comments Off on Samira Afzal talk at 10th Fraunhofer FOKUS MWS

All-intra rate control using low complexity video features for Versatile Video Coding

IEEE International Conference on Image Processing (ICIP), 2023

8-11 October 2023, Kuala Lumpur, Malaysia

Conference Website

[PDF]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Anastasia Henkel (Fraunhofer HHI Berlin), Prajit T Rajendran (Universite Paris-Saclay), Christian R Helmich (Fraunhofer HHI Berlin), Adam Wieckowski (Fraunhofer HHI Berlin), Benjamin Bross (Fraunhofer HHI Berlin), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), and Detlev Marpe (Fraunhofer HHI Berlin).

Abstract:

Versatile Video Coding (VVC) allows for large compression efficiency gains over its predecessor, High Efficiency Video Coding (HEVC). The added efficiency comes at the cost
of increased runtime complexity, especially for encoding. It is thus highly relevant to explore all available runtime reduction options. This paper proposes a novel first pass for
two-pass rate control in all-intra configuration, using low-complexity video analysis and a Random Forest (RF)-based machine learning model to derive the data required for driving the second pass. The proposed method is validated using VVenC, an open and optimized VVC encoder. Compared to the default two-pass rate control algorithm in VVenC, the
proposed method achieves around 32% reduction in encoding time for the preset faster, while on average only causing a 2% BD-rate increase and achieving similar rate control accuracy.

Proposed two-pass rate control encoding architecture.

Posted in ATHENA | Comments Off on All-intra rate control using low complexity video features for Versatile Video Coding