ACM Multimedia’20: Relevance-Based Compression of Cataract Surgery Videos Using Convolutional Neural Networks

ACM MM’20: Relevance-Based Compression of Cataract Surgery Videos Using Convolutional Neural Networks

ACM International Conference on Multimedia 2020, Seattle, United States.
https://2020.acmmm.org

[PDF][Slides][Video]

Negin Ghamsarian (Alpen-Adria-Universität Klagenfurt), Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Bitmovin), Mario Taschwer (Alpen-Adria-Universität Klagenfurt), and Klaus Schöffmann (Alpen-Adria-Universität Klagenfurt)

Abstract: Recorded cataract surgery videos play a prominent role in training and investigating the surgery, and enhancing the surgical outcomes. Due to storage limitations in hospitals, however, the recorded cataract surgeries are deleted after a short time and this precious source of information cannot be fully utilized. Lowering the quality to reduce the required storage space is not advisable since the degraded visual quality results in the loss of relevant information that limits the usage of these videos. To address this problem, we propose a relevance-based compression technique consisting of two modules: (i) relevance detection, which uses neural networks for semantic segmentation and classification of the videos to detect relevant spatio-temporal information, and (ii) content-adaptive compression, which restricts the amount of distortion applied to the relevant content while allocating less bitrate to irrelevant content. The proposed relevance-based compression framework is implemented considering five scenarios based on the definition of relevant information from the target audience’s perspective. Experimental results demonstrate the capability of the proposed approach in relevance detection. We further show that the proposed approach can achieve high compression efficiency by abstracting substantial redundant information while retaining the high quality of the relevant content.

Keywords: Video Coding, Convolutional Neural Networks, HEVC, ROI Detection, Medical Multimedia.

Posted in News | Comments Off on ACM Multimedia’20: Relevance-Based Compression of Cataract Surgery Videos Using Convolutional Neural Networks

QUALINET White Paper on Definitions of Immersive Media Experience (IMEx)

QUALINET announces its recent White Paper on Definitions of Immersive Media Experience (IMEx).

It is online available here https://arxiv.org/abs/2007.07032 for free.

With the coming of age of virtual/augmented reality and interactive media, numerous definitions, frameworks, and models of immersion have emerged across different fields ranging from computer graphics to literary works. Immersion is oftentimes used interchangeably with presence as both concepts are closely related. However, there are noticeable interdisciplinary differences regarding definitions, scope, and constituents that are required to be addressed so that a coherent understanding of the concepts can be achieved. Such consensus is vital for paving the directionality of the future of immersive media experiences (IMEx) and all related matters.

The aim of this white paper is to provide a survey of definitions of immersion and presence which leads to a definition of immersive media experience (IMEx). The Quality of Experience (QoE) for immersive media is described by establishing a relationship between the concepts of QoE and IMEx followed by application areas of immersive media experience. Influencing factors on immersive media experience are elaborated as well as the assessment of immersive media experience. Finally, standardization activities related to IMEx are highlighted and the white paper is concluded with an outlook related to future developments.

This White Paper is a contribution by QUALINET, the European Network on Quality of Experience in Multimedia Systems and Services (http://www.qualinet.eu/) to the discussions related to Immersive Media Experience (IMEx). It is motivated by the need for definitions around this term to foster a deeper understanding of ideas and concepts originating from multidisciplinary groups but with a joint interest in multimedia experiences. Thus, this white paper has been created mainly with such multimedia experiences in mind but may be also used beyond.

The QUALINET community aims at extending the notion of network-centric Quality of Service (QoS) in multimedia systems, by relying on the concept of Quality of Experience (QoE). The main scientific objective is the development of methodologies for subjective and objective quality metrics taking into account current and new trends in multimedia communication systems as witnessed by the appearance of new types of content and interactions. QUALINET (2010-2014 as COST Action IC1003) meets once a year collocated with QoMEX (http://qomex.org/) to coordinate its activities around 4 Working Groups (WGs): (i) research, (ii) standardization, (iii) training, and (iv) innovation.

List of Authors and Contributors
Andrew Perkis (andrew.perkis@ntnu.no, editor), Christian Timmerer (christian.timmerer@itec.uni-klu.ac.at, editor), Sabina Baraković, Jasmina Baraković Husić, Søren Bech, Sebastian Bosse, Jean Botev, Kjell Brunnström, Luis Cruz, Katrien De Moor, Andrea de Polo Saibanti, Wouter Durnez, Sebastian Egger-Lampl, Ulrich Engelke, Tiago H. Falk, Asim Hameed, Andrew Hines, Tanja Kojic, Dragan Kukolj, Eirini Liotou, Dragorad Milovanovic, Sebastian Möller, Niall Murray, Babak Naderi, Manuela Pereira, Stuart Perry, Antonio Pinheiro, Andres Pinilla, Alexander Raake, Sarvesh Rajesh Agrawal, Ulrich Reiter, Rafael Rodrigues, Raimund Schatz, Peter Schelkens, Steven Schmidt, Saeed Shafiee Sabet, Ashutosh Singla, Lea Skorin-Kapov, Mirko Suznjevic, Stefan Uhrig, Sara Vlahović, Jan-Niklas Voigt- Antons, Saman Zadtootaghaj.

How to reference this white paper
Perkis, A., Timmerer, C., et al., “QUALINET White Paper
on Definitions of Immersive Media Experience (IMEx)”, European Network on Quality of Experience in Multimedia Systems and Services, 14th QUALINET meeting (online), May 25, 2020.

Alternatively, you may export the citation from arXiv: https://arxiv.org/abs/2007.07032.

Posted in News | Comments Off on QUALINET White Paper on Definitions of Immersive Media Experience (IMEx)

BigMM’20: ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Sequences using Artificial Neural Network

ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Sequences Using Artificial Neural Network

The Sixth IEEE International Conference on Multimedia Big Data (BigMM 2020)

September 24-26, 2020 New Delhi. http://bigmm2020.org/

Anatoliy Zabrovskiy (Alpen-Adria-Universität Klagenfurt), Prateek Agrawal (Alpen-Adria-Universität Klagenfurt, Lovely Professional  University), Roland Mathá (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Bitmovin) and Radu Prodan (Alpen-Adria-Universität Klagenfurt).

Abstract: HTTP Adaptive Streaming of video content is becoming an integral part of the Internet and accounts for the majority of today’s traffic. Although Internet bandwidth is constantly increasing, video compression technology plays an important role and the major  challenge is to select and set up multiple video codecs, each with hundreds of transcoding parameters. Additionally, the transcoding speed depends directly on the selected transcoding parameters and the infrastructure used. Predicting transcoding time for multiple transcoding parameters with different codecs and processing units is a challenging task, as it depends on many factors. This paper provides a novel and considerably fast method for transcoding time prediction using video content classification and neural network prediction. Our artificial neural network (ANN) model predicts the transcoding times of video segments for state-of-the-art video codecs based on transcoding parameters and content complexity. We evaluated our method for two video codecs/implementations (AVC/x264 and HEVC/x265) as part of large-scale HTTP Adaptive Streaming services. The ANN model of our method is able to predict the transcoding time by minimizing the mean absolute error (MAE) to 1.37 and 2.67 for x264 and x265 codecs, respectively. For x264, this is an improvement of 22% compared to the state of the art.

Keywords: Transcoding time prediction, adaptive streaming, video transcoding, neural networks, video encoding, video complexity class, MPEG-DASH

Acknowledgment: This work has been supported in part by the Austrian Research Promotion Agency (FFG) under the APOLLO project and by the European Union Horizon 2020 Research and Innovation Programme under the ASPIDE Project with grant agreement number 801091.

Posted in APOLLO, News | Comments Off on BigMM’20: ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Sequences using Artificial Neural Network

Hadi Amirpour awarded the Best Engagement Award at ACM MMSys 2020

Hadi Amirpour has been awarded the Best Engagement Award at ACM MMSys 2020.

This year’s ACM MMSys was held as a fully virtual/online event and Slido was used for asking questions about keynotes and presentations including offline discussions with presenters. The interaction report provides some interesting key insights including the word cloud below which provides an overview of this year’s discussion items.

Although ACM MMSys 2020 is over, everyone is welcome joining the MMSys Slack workspace where the discussion will continue until ACM MMSys 2021 (available soon!) and hopefully beyond.

Posted in News | Comments Off on Hadi Amirpour awarded the Best Engagement Award at ACM MMSys 2020

EPIQ’20: Scalable High Efficiency Video Coding based HTTP Adaptive Streaming over QUIC Using Retransmission

Scalable High Efficiency Video Coding based HTTP Adaptive Streaming over QUIC Using Retransmission

ACM SIGCOMM 2020 Workshop on Evolution, Performance, and Interoperability of QUIC (EPIQ 2020)
August 10–14, 2020
https://conferences.sigcomm.org/sigcomm/2020/workshop-epiq.html

[PDF][Slides][Video]

Minh Nguyen (Alpen-Adria-Universität Klagenfurt), Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt / Bitmovin Inc.), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)

Abstract: HTTP/2 has been explored widely for video streaming, but still suffers from Head-of-Line blocking, and three-way hand-shake delay due to TCP. Meanwhile, QUIC running on top of UDP can tackle these issues. In addition, although many adaptive bitrate (ABR) algorithms have been proposed for scalable and non-scalable video streaming, the literature lacks an algorithm designed for both types of video streaming approaches. In this paper, we investigate the impact of quick and HTTP/2 on the performance of adaptive bitrate(ABR) algorithms in terms of different metrics. Moreover, we propose an efficient approach for utilizing scalable video coding formats for adaptive video streaming that combines a traditional video streaming approach (based on non-scalable video coding formats) and a retransmission technique. The experimental results show that QUIC benefits significantly from our proposed method in the context of packet loss and retransmission. Compared to HTTP/2, it improves the average video quality and also provides a smoother adaptation behavior. Finally, we demonstrate that our proposed method originally designed for non-scalable video codecs also works efficiently for scalable videos such as Scalable High EfficiencyVideo Coding(SHVC).

Keywords: QUIC, H2BR, HTTP adaptive streaming, Retransmission, SHVC

Posted in News | Comments Off on EPIQ’20: Scalable High Efficiency Video Coding based HTTP Adaptive Streaming over QUIC Using Retransmission

PV’20: H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive Video Streaming

H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive Video Streaming

Packet Video Workshop 2020 (PV)
June 10-11, 2020, Istanbul, Turkey (co-located with ACM MMSys’20)

https://2020.packet.video/

[PDF][Slides][Video]

Minh Nguyen (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt / Bitmovin Inc.), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)

Abstract: HTTP-based Adaptive Streaming (HAS) plays a key role in over-the-top video streaming. It contributes towards reducing the rebuffering duration of video playout by adapting the video quality to the current network conditions. However, it incurs variations of video quality in a streaming session because of the throughput fluctuation, which impacts the user’s Quality of Experience (QoE). Besides, many adaptive bitrate (ABR) algorithms choose the lowest-quality segments at the beginning of the streaming session to ramp up the playout buffer as soon as possible. Although this strategy decreases the startup time, the users can be annoyed as they have to watch a low-quality video initially. In this paper, we propose an efficient retransmission technique, namely H2BR, to replace low-quality segments being stored in the playout buffer with higher-quality versions by using features of HTTP/2 including (i) stream priority, (ii) server push, and (iii) stream termination. The experimental results show that H2BR helps users avoid watching low video quality during video playback and improves the user’s QoE. H2BR can decrease by up to more than 70% the time when the users suffer the lowest-quality video as well as benefits the QoE by up to 13%.

Keywords: HTTP adaptive streaming, DASH, ABR algorithms, QoE, HTTP/2

Posted in News | Comments Off on PV’20: H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive Video Streaming

ICME’20: Towards View-aware Adaptive Streaming of Holographic content

ICME’20: Towards View-aware Adaptive Streaming of Holographic content

IEEE International Conference on Multimedia & Expo (ICME) 2020, London, UK.

Workshop on Hyper-Realistic Multimedia for Enhanced Quality of Experience

[PDF][Slides][Video]

Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Bitmovin), and Mohammad Ghanbari (University of Essex)

Abstract: Holography is able to reconstruct a three-dimensional structure of an object by recording full wave fields of light emitted from the object. This requires a huge amount of data to be encoded, stored, transmitted, and decoded for holographic content, making its practical usage challenging especially for bandwidth-constrained networks and memory-limited devices. In the delivery of holographic content via the internet, bandwidth wastage should be avoided to tackle high bandwidth demands of holography streaming. For real-time applications, encoding time-complexity is also a major problem. In this paper, the concept of dynamic adaptive streaming over HTTP (DASH) is extended to holography image streaming and view-aware adaptation techniques are studied. As each area of a hologram contains information of a specific view, instead of encoding and decoding the entire hologram, just the part required to render the selected view is encoded and transmitted via the network based on the users’ interactivity. Four different strategies, namely, monolithic, single view, adaptive view, and non-real time streaming strategies are explained and compared in terms of bandwidth requirements, encoding time-complexity, and bitrate overhead. Experimental results show that the view-aware methods reduce the required bandwidth for holography streaming at the cost of a bitrate increase.

Keywords: Holography, compression, bitrate adaptation, dynamic adaptive streaming over HTTP, DASH.

Posted in News | Comments Off on ICME’20: Towards View-aware Adaptive Streaming of Holographic content

MMSys’20: Cloud-based Adaptive Video Streaming Evaluation Framework for the Automated Testing of Media Players (CAdViSE)

CAdViSE: Cloud-based Adaptive Video Streaming Evaluation Framework for the Automated Testing of Media Players
ACM Multimedia Systems Conference 2020 (MMSys 2020)
https://2020.acmmmsys.org/
Babak Taraghi (Alpen-Adria-Universität Klagenfurt), Anatoliy Zabrovskiy (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Bitmovin) and Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)

Abstract: Attempting to cope with fluctuations of network conditions in terms of available bandwidth, latency and packet loss, and to deliver the highest quality of video (and audio) content to users, research on adaptive video streaming has attracted intense efforts from the research community and huge investments from technology giants. How successful these efforts and investments are, is a question that needs precise measurements of the results of those technological advancements. HTTP-based Adaptive Streaming (HAS) algorithms, which seek to improve video streaming over the Internet, introduce video bitrate adaptivity in a way that is scalable and efficient. However, how each HAS implementation takes into account the wide spectrum of variables and configuration options, brings a high complexity to the task of measuring the results and visualizing the statistics of the performance and quality of experience. In this paper, we introduce CAdViSE, our Cloud-based Adaptive Video Streaming Evaluation framework for the automated testing of adaptive media players. The paper aims to demonstrate a test environment which can be instantiated in a cloud infrastructure, examines multiple media players with different network attributes at defined points of the experiment time, and finally concludes the evaluation with visualized statistics and insights into the results.

Keywords: HTTP Adaptive Streaming, Media Players, MPEG-DASH, Network Emulation, Automated Testing, Quality of Experience

ACM Reference Format:
Babak Taraghi, Anatoliy Zabrovskiy, Christian Timmerer, and Hermann Hellwagner. 2020. CAdViSE: Cloud-based Adaptive Video Streaming Evaluation Framework for the Automated Testing of Media Players. In 11th ACM Multimedia Systems Conference (MMSys’20), June 8–11, 2020, Istanbul, Turkey. , 4 pages. https://doi.org/10.1145/3339825.3393581

Posted in News | Comments Off on MMSys’20: Cloud-based Adaptive Video Streaming Evaluation Framework for the Automated Testing of Media Players (CAdViSE)

QoMEX’20: Objective and Subjective QoE Evaluation for Adaptive Point Cloud Streaming

Objective and Subjective QoE Evaluation for Adaptive Point Cloud Streaming

*** Best Paper Award ***

International Conference on Quality of Multimedia Experience (QoMEX)
May 26-28, 2020, Athlone, Ireland
http://qomex2020.ie/

[PDF][Slides][Video]

Jeroen van der Hooft (Ghent University), Maria Torres Vega (Ghent University), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Bitmovin), Ali C. Begen (Ozyegin University, Networked Media), Filip De Turck (Ghent University), Raimund Schatz (Alpen-Adria Universität Klagenfurt & AIT Austrian Institute of Technology, Austria)

Abstract: Volumetric media has the potential to provide the six degrees of freedom (6DoF) required by truly immersive media. However, achieving 6DoF requires ultra-high bandwidth transmissions, which real-world wide area networks cannot provide economically. Therefore, recent efforts have started to target efficient delivery of volumetric media, using a combination of compression and adaptive streaming techniques. It remains, however, unclear how the effects of such techniques on the user perceived quality can be accurately evaluated. In this paper, we present the results of an extensive objective and subjective quality of experience (QoE) evaluation of volumetric 6DoF streaming. We use PCC-DASH, a standards-compliant means for HTTP adaptive streaming of scenes comprising multiple dynamic point cloud objects. By means of a thorough analysis we investigate the perceived quality impact of the available bandwidth, rate adaptation algorithm, viewport prediction strategy and user’s motion within the scene. We determine which of these aspects has more impact on the user’s QoE, and to what extent subjective and objective assessments are aligned.

Keywords: Volumetric Media; HTTP Adaptive Streaming; 6DoF; MPEG V-PCC; QoE Assessment; Objective Metrics

Posted in News | Comments Off on QoMEX’20: Objective and Subjective QoE Evaluation for Adaptive Point Cloud Streaming

ACM TOMM: Performance Analysis of ACTE: a Bandwidth Prediction Method for Low-Latency Chunked Streaming

Performance Analysis of ACTE: a Bandwidth Prediction Method for Low-Latency Chunked Streaming

ACM Transactions on Multimedia Computing, Communications, and Applications

[PDF]

Abdelhak Bentaleb (National University of Singapore), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Bitmovin), Ali C. Begen (Ozyegin University, Networked Media), Roger Zimmermann (National University of Singapore)

Abstract: HTTP adaptive streaming with chunked transfer encoding can offer low-latency streaming without sacrificing the coding efficiency. This allows media segments to be delivered while still being packaged. However, conventional schemes often make widely inaccurate bandwidth measurements due to the presence of idle periods between the chunks and hence this is causing sub-optimal adaptation decisions. To address this issue, we earlier proposed ACTE (ABR for Chunked Transfer Encoding), a bandwidth prediction scheme for low-latency chunked streaming. While ACTE was a significant step forward, in this study we focus on two still remaining open areas, namely (i) quantifying the impact of encoding parameters, including chunk and segment durations, bitrate levels, minimum interval between IDR-frames and frame rate onACTE, and (ii) exploring the impact of video content complexity on ACTE. We thoroughly investigate these questions and report on our findings. We also discuss some additional issues that arise in the context of pursuing very low latency HTTP video streaming.

Keywords: HAS; ABR; DASH; CMAF; low-latency; HTTP chunked transfer encoding; bandwidth measurement and prediction; RLS; encoding parameters; FFmpeg

Posted in News | Comments Off on ACM TOMM: Performance Analysis of ACTE: a Bandwidth Prediction Method for Low-Latency Chunked Streaming