Vignesh V Menon and Hadi Amirpour gave a talk on ‘Video Complexity Analyzer for Streaming Applications’ at the Video Quality Experts Group (VQEG) meeting on December 14, 2021. Our research activities on video complexity analysis were presented in the talk.
The link to the presentation can be found here (pdf).
Abstract: Universal access to and provisioning of multimedia content is now a reality. It is easy to generate, distribute, share, and consume any multimedia content, anywhere, anytime, on any device thanks to a plethora of applications and services that are now commodities in our daily life. Interestingly, most of these services adopt a streaming paradigm, are typically deployed over the open, unmanaged Internet, and account for most of today’s Internet traffic. Currently, the global video traffic is greater than 60 percent of all Internet traffic and it is expected that this share will grow to more than 80 percent in the near future (according to Sandvine and Cisco VNI). Additionally, Nielsen’s Law of Internet bandwidth states that the users’ bandwidth grows by 50 percent per year, which roughly fits data from 1983 to 2019. Thus, the users’ bandwidth can be expected to reach approximately 1 Gbps by 2022. At the same time, network applications will grow and utilize the bandwidth provided, just like programs and their data expand to fill the memory available in a computer system. Most of the available bandwidth today is consumed by video applications and the amount of data is further increasing due to already established and emerging applications, e.g., ultra-high definition, high dynamic range, or virtual, augmented, mixed realities, or immersive media applications in general with the aim to increase the Immersive Media Experience (IMEx).
A major technical breakthrough was the adaptive streaming over HTTP resulting in the standardization of MPEG Dynamic Adaptive Streaming over HTTP (DASH), which enables a content-/format-agnostic delivery over-the-top (OTT) of the existing infrastructure. Thus, this tutorial takes DASH as a basis and explains how it is adopted for immersive media delivery such as omnidirectional/360-degree video and any other volumetric video representations (i.e., point clouds, light fields, holography). The focus of this tutorial is related to the principles of Quality of Experience (QoE) for such immersive media applications and services including its assessment and management. Finally, this tutorial concludes with open research issues and industry efforts in this domain.
Posted inNews|Comments Off on IEEE VCIP’21 Tutorial: A Journey towards Fully Immersive Media Access
At Christian Doppler laboratory ATHENA, we offer an internship*) for 2022 for Master Students and we kindly request your applications until 14th of December 2021 with the following data (in German or English):
Record of study/transcript (“Studienerfolgsnachweis”)
*) A 3-month period in 2022 (with an exact time slot to be discussed) with the possibility to spend up to 1-month at the industrial partner; 20h per week “Universitäts-KV, Verwendungsgruppe C1, studentische Hilfskraft”
About ATHENA: The Christian Doppler laboratory ATHENA (AdapTive Streaming over HTTP and Emerging Networked MultimediA Services) is jointly proposed by the Institute of Information Technology (ITEC; http://itec.aau.at) at Alpen-Adria-Universität Klagenfurt (AAU) and Bitmovin GmbH (https://bitmovin.com) to address current and future research and deployment challenges of HAS and emerging streaming methods. AAU (ITEC) has been working on adaptive video streaming for more than a decade, has a proven record of successful research projects and publications in the field, and has been actively contributing to MPEG standardization for many years, including MPEG-DASH; Bitmovin is a video streaming software company founded by ITEC researchers in 2013 and has developed highly successful, global R&D and sales activities and a world-wide customer base since then.
The aim of ATHENA is to research and develop novel paradigms, approaches, (prototype) tools, and evaluation results for the phases
multimedia content provisioning,
content delivery, and
content consumption in the media delivery chain as well as for
end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS).
The new approaches and insights are to enable Bitmovin to build innovative applications and services to account for the steadily increasing and changing multimedia traffic on the Internet.
Posted inNews|Comments Off on Internship 2022 at ATHENA
Title: FSpot: Fast Multi-Objective Heuristic for Efficient Video Encoding Workloads over AWS EC2 Spot Instance Fleet
Authors: Anatoliy Zabrovskiy, Prateek Agrawal, Vladislav Kashansky, Roland Kersche, Christian Timmerer, and Radu Prodan
Abstract: HTTP Adaptive Streaming (HAS) of video content is becoming an undivided part of the Internet and accounts for most of today’s network traffic. Video compression technology plays a vital role in efficiently utilizing network channels, but encoding videos into multiple representations with selected encoding parameters is a significant challenge. However, video encoding is a computationally intensive and time-consuming operation that requires high-performance resources provided by on-premise infrastructures or public clouds. In turn, the public clouds, such as Amazon elastic compute cloud (EC2), provide hundreds of computing instances optimized for different purposes and clients’ budgets. Thus, there is a need for algorithms and methods for optimized computing instance selection for specific tasks such as video encoding and transcoding operations. Additionally, the encoding speed directly depends on the selected encoding parameters and the complexity characteristics of video content. In this paper, we first benchmarked the video encoding performance of Amazon EC2 spot instances using multiple x264 codec encoding parameters and video sequences of varying complexity. Then, we proposed a novel fast approach to optimize Amazon EC2 spot instances and minimize video encoding costs. Furthermore, we evaluated how the optimized selection of EC2 spot instances can affect the encoding cost. The results show that our approach, on average, can reduce the encoding costs by at least 15.8% and up to 47.8% when compared to a random selection of EC2 spot instances.
The Fast Multi-Resolution and Multi-Rate Encoding for HTTP Adaptive Streaming Using Machine Learning paper from ATHENA lab has won the Best New Streaming Innovation Award in the Streaming Media Readers’ Choice Awards 2021.
The journey that led to the publication of the FaRes-ML paper was quite an insightful one.
It all started with the question, “How to efficiently provide multi-rate representations over a wide range of resolutions for HTTP Adaptive Streaming?“. This led to the first publication, Fast Multi-Rate Encoding for Adaptive HTTP Streaming, in which we proposed a double-bound approach to speed up the multi-rate encoding. After analyzing the results, we saw room for improvement in parallel encoding performance, which led to the second publication Towards Optimal Multirate Encoding for HTTP Adaptive Streaming. The results were promising, but we believed we could improve the encoding performance by utilizing machine learning. That was the primary motivation behind our third paper, FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Learning. In FaMe-ML, we have used convolutional neural networks (CNNs) to use the information from the reference representation better to encode other representations, resulting in significant improvement in the multi-rate encoding performance. Finally, we proposed FaRes-ML to extend our FaME-ML approach to include multi-resolution scenarios in Fast Multi-Resolution and Multi-Rate Encoding for HTTP Adaptive Streaming Using Machine Learning paper.
Here is the list of publications that led to FaRes-ML:
Jesús Aguilar Armijo(Alpen-Adria-Universität Klagenfurt), Ekrem Çetinkaya (Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt) and Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)
Abstract: As the video streaming traffic in mobile networks is increasing, improving the content delivery process becomes crucial, e.g., by utilizing edge computing support. At an edge node, we can deploy adaptive bitrate (ABR) algorithms with a better understanding of network behavior and access to radio and player metrics. In this work, we present ECAS-ML, Edge Assisted Adaptation Scheme for HTTP Adaptive Streaming with Machine Learning. ECAS-ML focuses on managing the tradeoff among bitrate, segment switches and stalls to achieve a higher quality of experience (QoE). For that purpose, we use machine learning techniques to analyze radio throughput traces and predict the best parameters of our algorithm to achieve better performance. The results show that ECAS-ML outperforms other client-based and edge-based ABR algorithms.
Keywords: HTTP Adaptive Streaming, Edge Computing, Content
Delivery, Network-assisted Video Streaming, Quality of Experience,
Posted inNews|Comments Off on ECAS-ML: Edge Computing Assisted Adaptation Scheme with Machine Learning for HTTP Adaptive Streaming
Abstract: Deep neural network (DNN) based approaches have been intensively studied to improve video quality thanks to their fast advancement in recent years. These approaches are designed mainly for desktop devices due to their high computational cost. However, with the increasing performance of mobile devices in recent years, it became possible to execute DNN based approaches in mobile devices. Despite having the required computational power, utilizing DNNs to improve the video quality for mobile devices is still an active research area. In this paper, we propose an open-source mobile platform, namely MoViDNN, to evaluate DNN based video quality enhancement methods, such as super-resolution, denoising, and deblocking. Our proposed platform can be used to evaluate the DNN based approaches both objectively and subjectively. For objective evaluation, we report common metrics such as execution time, PSNR, and SSIM. For subjective evaluation, Mean Score Opinion (MOS) is reported. The proposed platform is available publicly at https://github.com/cd-athena/MoViDNN
Keywords: Super resolution, Deblocking, Deep Neural Networks, Mobile Devices
Posted inNews|Comments Off on MoViDNN: A Mobile Platform for Evaluating Video Quality Enhancement with Deep Neural Networks