Grand Challenges in Green Multimedia Signal Processing

Posted on July 17, 2023 by

Grand Challenges in Green Multimedia Signal Processing at the IEEE 25th International Workshop on Multimedia Signal Processing (MMSP)

https://attend.ieee.org/mmsp-2023/call-for-grand-challenges/

It’s time to take action against the threat of climate change by making significant changes to our global greenhouse gas (GHG) emissions. That includes rethinking how we consume energy for digital technologies. Did you know that video streaming technology alone is responsible for over half of digital technology’s global impact? With the rise of digital and remote work becoming more common, there’s been a rapid increase in video data volume, processing, and streaming. Unfortunately, this also means an increase in energy consumption and GHG emissions. But with thoughtful and positive efforts, we can make a difference and reduce our impact on the environment. Let’s do this!

We are thrilled to invite experts and researchers to join us for the Grand Challenges in Green Multimedia Signal Processing at the IEEE 25th International Workshop on Multimedia Signal Processing (MMSP)! This is an exciting opportunity to explore the latest developments and challenges in reducing energy consumption in multimedia systems. Our session is dedicated to sharing innovative concepts and energy-efficient solutions across the entire spectrum of video generation, processing, delivery, and usage. Let’s work together to make a positive impact on the environment and multimedia industry!

We’re excited to invite you to submit your awesome proposals for the grand challenges in green multimedia signal processing. We’re looking for innovative solutions and results that can make a real difference in this field. To make the process easier, please follow the regular MMSP paper template with a maximum of four pages including acknowledgments, references, or any additional material you’d like to include. You can rest assured that your submission will be reviewed by top experts in the field, and accepted papers will be featured in the MMSP conference proceedings, which will be included in IEEE Xplore. We can’t wait to see what you’ve got!

Submissions should follow the IEEE template given in the Instructions for authors section.

Timeline:

  • Submission deadline: August 16, 2023 [CMT: new submission under “Grand challenges”]
  • Acceptance Notification: August 30, 2023

Chairs:

  • Samira Afzal, Alpen-Adria-Universität, Austria
  • Cagri Ozcinar, MSK AI, UK
  • Christian Timmerer, Alpen-Adria-Universität, Austria

For any questions regarding the challenge, please send an email to mmsp-2023@univ-poitiers.fr

Posted in ATHENA, GAIA | Comments Off on Grand Challenges in Green Multimedia Signal Processing

ICME’23 Tutorial: Advances in HTTP Adaptive Streaming: New Standards, Video Codecs, and Encoding Optimization Techniques

Posted on July 10, 2023 by

IEEE ICME 2023
Monday, July 10, 2023
https://www.2023.ieeeicme.org/tutorial-prop.php

Lectures:

  • Hadi Amirpour, AAU, AT
  • Yutiy Reznik, Brightcove, USA
  • Nabajeet Barman,  Brightcove
  • Christian Timmerer, AAU, AT

Abstract: The applications of video streaming are the primary drivers of Internet traffic, as over 82% of IP traffic in 2022. The prevailing technology to stream live and video-on-demand (VoD) content over the Internet is HTTP Adaptive Streaming (HAS). In HAS, the content is encoded at multiple representations (bitrate-resolution pairs), and delivered incrementally using segments (or chunks) of encoded representations. This allows for dynamic selection of different representations during playback, and the HAS model supports delivery under different and changing network- and device-specific conditions. As the demand for video streaming applications is on the rise, improved streaming methods, as well as video codecs and video content optimization algorithms, are being developed to meet this demand. This tutorial first presents a detailed overview of the existing file formats, protocols (HLS, DASH, CMAF), video codecs (H.264, HEVC, AV1, VVC), and quality assessment methods (PSNR, SSIM, VMAF, P.1203, CTA2066) used for streaming. Particular emphasis will be given to new video codecs, such as Versatile Video Coding (VVC), and their features applicable to streaming. We will then focus on recent advances in video encoding optimizations for HAS streaming. We will then introduce per-title, content-aware, and context-aware encoding methods, which optimize bitrate ladders based on the characteristics of videos and other contexts. It will also be presented how representations are selected in a way that bitrate ladders are optimized over various dimensions, including spatial resolution, frame rate, energy consumption, device type, and network condition. Additionally, methods for reducing the latency of dynamic ladder generation and fast multi-rates encoding will be covered. Finally, we will discuss various areas of recent progress in the design of streaming systems, including end-to-end modelling, optimization, and analysis, and highlight remaining open problems in the field.

Posted in ATHENA | Comments Off on ICME’23 Tutorial: Advances in HTTP Adaptive Streaming: New Standards, Video Codecs, and Encoding Optimization Techniques

IEEE Network Special Issue on Advancements in Network-Assisted Video Streaming: Optimization and Performance Analysis

Posted on June 30, 2023 by

IEEE Network Special Issue on

Advancements in Network-Assisted Video Streaming: Optimization and Performance Analysis

Download CfP

Call for Papers

Network-assisted video streaming has become a substantial part of modern multimedia applications, enabling users to access high-quality video content over different networks, including the Internet and wireless networks. Efficiently delivering video content over networks poses numerous challenges, such as limited bandwidth, varying network conditions, heterogeneous end devices, and diverse user preferences. Network-assisted video streaming approaches leverage modern networking technologies, such as Software-Defined Networking (SDN), Network Function Virtualization (NFV), and edge computing, to not only improve the users’ Quality of Experience (QoE) but also enhance network utilization.

This special issue aims to explore the latest advancements in network-assisted video streaming, with a specific focus on optimization techniques and comprehensive performance analysis, by exploring emerging trends and innovations, including novel approaches that leverage Artificial Intelligence (AI) techniques, machine learning algorithms, and data-driven optimization methods to enhance the streaming experience. Furthermore, contributions related to the integration of edge computing, Virtual Reality (VR), Augmented Reality (AR), and volumetric video streaming will be welcomed. Since understanding the performance of network-assisted video streaming systems is important for assessing their effectiveness and identifying areas for improvement, the research articles should cover both experimental and theoretical aspects, utilizing real-world datasets, simulation frameworks, analytical models, and conducting real-world experiments.

The research and advancements of this special issue will have a significant impact on the design, implementation, and operation of video streaming systems. The findings will provide valuable insights to network operators, content providers, researchers, and developers, enabling them to optimize their systems for enhanced user experience. Additionally, the knowledge gained from this special issue will contribute to the development of standards and best practices for network-assisted video streaming, benefiting the broader multimedia community. IEEE Network, the flagship magazine on networking technologies, is a perfect venue for publishing this special issue. We believe that our special issue covers a number of key aspects and emerging topics that are of interest to the readers of the IEEE Network.

This special issue is to publish original research and review articles that should be comprehensive to all readers of the IEEE Network Magazine, regardless of their specialty. This SI aims to bring together researchers and developers working on all aspects of video streaming, in particular network-assisted concepts backed up by experimental evidence. Potential topics include but are not limited to the following:

  • Design, analysis, and evaluation of network-assisted multimedia system architectures
  • Using AI/ML at the network edge and the cloud for supporting video streaming
  • AI/ML-enabled caching of video chunks
  • Network-assisted/AI-based resource allocation for video streaming
  • Experience and lessons learned by deploying large-scale network-assisted video streaming
  • Internet measurement and modeling for enhancing QoE in video streaming applications
  • Network aspects in video streaming: cloud computing, virtualization techniques, network control, and management, including SDN, NFV, and network programmability
  • Topics at the intersection of energy-efficient computing and networking for video streaming
  • Machine learning for improving QoE in video streaming applications
  • Machine learning for traffic engineering and congestion control for video streaming
  • AI/ML-based solutions for supporting streaming applications’ high-speed user mobility
  • Big data analytics at the network edge to assess viewer experience of adaptive video
  • Advanced network-based techniques for point clouds, light fields, and immersive video
  • Using AI/ML techniques for optimizing Interactive Streaming and User-Generated Content
  • The tradeoff between QoE enhancement and network overhead: AI approaches
  • AI/ML-based techniques for live streaming in 5G and 6G networks

Submission Guidelines

Manuscripts should conform to the standard format as indicated in the “Information for Authors” section of the Paper Submission Guidelines. All manuscripts to be considered for publication must be submitted by the deadline through the magazine’s Manuscript Central submission site. Select “May2024/VideoStreaming” from the drop-down menu of Topic titles.

Important Dates

  • Manuscript Submission Deadline: 31 October 2023
  • Initial Decision Notification: 31 December 2023
  • Revised Manuscript Due: 31 January 2024
  • Final Decision Notification: 28 February 2024
  • Final Manuscript Due: 15 March 2024
  • Publication Date: May/June 2024

Guest Editors

  • Farzad Tashtarian, Universität Klagenfurt, Austria (farzad.tashtarian@aau.at)
  • Yao Liu, Rutgers University, USA (yao.liu@rutgers.edu)
  • Müge Sayıt, Ege Üniversitesi, Turkey (muge.sayit@ege.edu.tr)
  • Junchen Jiang, University of Chicago, USA (junchenj@uchicago.edu)
  • Gwendal Simon, Synamedia, UK (gsimon@synamedia.com)
  • Christian Timmerer, Universität Klagenfurt, Austria (christian.timmerer@aau.at)
Posted in ATHENA | Comments Off on IEEE Network Special Issue on Advancements in Network-Assisted Video Streaming: Optimization and Performance Analysis

Samira Afzal talk at ICT4S Workshop: Towards Sustainable and Energy-Aware Video Communications (SEViC)

Posted on June 26, 2023 by

Energy Consumption in Video Streaming: Components, Measurements, and Strategies

ICT4S Workshop: Towards Sustainable and Energy-Aware Video Communications (SEViC)

June 5-9 2023 – Rennes, France

 

Abstract: The rapid growth of video streaming usage is a significant source of energy consumption, driven by improved internet connections and service offerings, the quick development of video entertainment, the deployment of Ultra High-Definition, Virtual and Augmented Reality, as well as an increasing number of video surveillance and IoT applications. To address this challenge, it is essential to understand the various components involved in energy consumption during video streaming, ranging from video encoding to decoding and displaying the video on the end user’s screen. Then, it is critical to measure energy consumption for each component accurately and conduct an in-depth analysis to develop energy-efficient strategies that optimize video streaming. These components are classified into three categories: (i) data centers, which include encoding, packaging, and storage on cloud data centers; (ii) networks, which include core network and access networks; and (iii) end-user devices which involve decoding, players, hardware, etc.

In addition to identifying the primary components of video streaming that affect energy consumption, it is important to conduct a comprehensive analysis of the entire video streaming. It is also essential to balance energy optimization and service quality to ensure that energyefficient strategies are implemented without sacrificing the quality of video streaming services.

This talk aims to provide insights into the components of video streaming that contribute to energy consumption and highlight the challenges associated with measuring their energy usage. I will also introduce the tools that can be used for energy measurements for those components and the possible and associated strategies that lie within energy efficiency. By accurately measuring energy consumption, digital media companies can effectively monitor and control their energy usage, ultimately leading to cost savings and improved sustainability.

https://www.slideshare.net/christian.timmerer/energy-consumption-in-video-streaming-components-measurements-and-strategies

Posted in GAIA | Comments Off on Samira Afzal talk at ICT4S Workshop: Towards Sustainable and Energy-Aware Video Communications (SEViC)

Samira Afzal talk at 10th Fraunhofer FOKUS MWS

Posted on June 26, 2023 by

Exploring the Energy Consumption of Video Streaming: Components, Challenges, and Opportunities

10th FOKUS Media Web Symposium

June 13–14, 2023 – Berlin, Germany

 

Abstract: The rapid growth of video streaming usage is a significant source of energy consumption, driven by improved internet connections and service offerings, the quick development of video entertainment, the deployment of Ultra High-Definition, Virtual and Augmented Reality, as well as an increasing number of video surveillance and IoT applications. However, it is essential to note that these advancements come at the cost of energy consumption. To address this challenge, it is essential to understand the various components involved in energy consumption during video streaming, ranging from video encoding to decoding and displaying the video on the end user’s screen. Then, it is critical to accurately measure energy consumption for each component and conduct an in-depth analysis to develop energy-efficient strategies that optimize video streaming. I categorize these components into three categories: (i) data centers, (ii) networks, and (iii) end-user devices.

Data centers: Data centers are responsible for the significant growth in video data traffic, which is estimated to reach over 1,000 TWh of power consumption by 2025. To effectively manage energy consumption in data centers, it is crucial to understand the various components that contribute to it, including encoding process and parameters, resource provisioning, core network, storage, as well as hardware aspects such as cloud platform features and hardware units (e.g., CPU, GPU). By analyzing and optimizing these energy-intensive components, energy-efficient strategies, such as energy management and task schedulers, and energy-efficient codecs can be developed to improve the sustainability of data centers.

Networks: The next category within video streaming is the transmission of video from data centers to end devices through heterogeneous networks. The network energy-intensive components are CDNs, routers, switches, and network channels. To optimize energy efficiency during video transmission, it is essential to manage and optimize energy consumption in each of these components. This can be achieved by implementing efficient routing algorithms, reducing data redundancy, and utilizing power-saving mechanisms in network devices.

End-user devices: The last category of the video streaming is video usage at the end-user device, which has been shown to account for the majority of energy consumption by the decoding hardware and end-user devices. Energy consumption in this category is due to the components such as decoding, players, browsers, codecs, operating systems, and hardware (e.g., CPU, display). Improving the energy efficiency of end-user devices can significantly reduce energy consumption in video streaming. Some examples of achieving this are through the use of more energy-efficient devices such as laptops, tablets, and smartphones or by improving screen display technologies.

In addition to identifying the primary components of video streaming that affect energy consumption, it is important to conduct a comprehensive analysis of the entire video streaming. It is also important to strike a balance between energy optimization and service quality to ensure that energy-efficient strategies are implemented without sacrificing the quality of video streaming services.

In this talk, my objective is to provide insights into the components of video streaming that contribute to energy consumption and highlight the challenges associated with measuring their energy usage. I will also introduce the tools that can be used for energy measurements for those components and the possible and associated strategies that lie within energy efficiency. By accurately measuring energy consumption, digital media companies can effectively monitor and control their energy usage, ultimately leading to cost savings and improved sustainability.

https://www.slideshare.net/christian.timmerer/exploring-the-energy-consumption-of-video-streaming-components-challenges-and-opportunities
Posted in GAIA | Comments Off on Samira Afzal talk at 10th Fraunhofer FOKUS MWS

All-intra rate control using low complexity video features for Versatile Video Coding

Posted on June 22, 2023 by

IEEE International Conference on Image Processing (ICIP), 2023

8-11 October 2023, Kuala Lumpur, Malaysia

Conference Website

[PDF]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Anastasia Henkel (Fraunhofer HHI Berlin), Prajit T Rajendran (Universite Paris-Saclay), Christian R Helmich (Fraunhofer HHI Berlin), Adam Wieckowski (Fraunhofer HHI Berlin), Benjamin Bross (Fraunhofer HHI Berlin), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), and Detlev Marpe (Fraunhofer HHI Berlin).

Abstract:

Versatile Video Coding (VVC) allows for large compression efficiency gains over its predecessor, High Efficiency Video Coding (HEVC). The added efficiency comes at the cost
of increased runtime complexity, especially for encoding. It is thus highly relevant to explore all available runtime reduction options. This paper proposes a novel first pass for
two-pass rate control in all-intra configuration, using low-complexity video analysis and a Random Forest (RF)-based machine learning model to derive the data required for driving the second pass. The proposed method is validated using VVenC, an open and optimized VVC encoder. Compared to the default two-pass rate control algorithm in VVenC, the
proposed method achieves around 32% reduction in encoding time for the preset faster, while on average only causing a 2% BD-rate increase and achieving similar rate control accuracy.

Proposed two-pass rate control encoding architecture.

Posted in ATHENA | Comments Off on All-intra rate control using low complexity video features for Versatile Video Coding

JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming

Posted on June 21, 2023 by

IEEE Transactions on Circuits and Systems for Video Technology (IEEE TCSVT)

Journal Website

[PDF]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Prajit T Rajendran (Universite Paris-Saclay, France), Christian Feldmann (Bitmovin), Klaus Schoeffmann (Alpen-Adria-Universität Klagenfurt), Mohammad Ghanbari (University of Essex, UK), and Christian Timmerer (Alpen-Adria-Universität Klagenfurt)

Abstract: Adaptive live video streaming applications utilize a predefined collection of bitrate-resolution pairs, known as a bitrate ladder, for simplicity and efficiency, eliminating the need for additional run-time to determine the optimal pairs during the live streaming session. These applications do not incorporate two-pass encoding methods due to increased latency. However, an optimized bitrate ladder could result in lower storage and delivery costs and improved Quality of Experience (QoE). This paper presents a Just Noticeable Difference (JND)-aware con-strained Variable Bitrate (cVBR) Two-pass Per-title encoding Scheme (JTPS) designed specifically for live video streaming. JTPS predicts a content- and JND-aware bitrate ladder using low-complexity features based on Discrete Cosine Transform (DCT) energy and optimizes the constant rate factor (CRF) for each representation using random forest-based models. The effectiveness of JTPS is demonstrated using the open source video encoder x265, with an average bitrate reduction of 18.80% and 32.59% for the same PSNR and VMAF, respectively, compared to the standard HTTP Live Streaming (HLS) bitrate ladder using Constant Bitrate (CBR) encoding. The implementation of JTPS also resulted in a 68.96% reduction in storage space and an 18.58% reduction in encoding time for a JND of six VMAF points.

Live HTTP adaptive streaming featuring our JND-aware two-pass per-title encoding scheme (JTPS).

Posted in ATHENA, News | Comments Off on JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming