IXR’23: Interactive eXtended Reality 2023

colocated with ACM Multimedia 2023

October 2023, Ottawa, Canada

[Link]

Workshop Chairs:

• Irene Viola, CWI, Netherlands
• Hadi Amirpour, Klagenfurt University, Austria
• Stephanie Arévalo Arboleda, TUIlmenau , Germany
• Maria Torres Vega, Ghent University, Belgium

Topics of interest include, but are not limited to:

• Novel low latency encoding techniques for interactive XR applications
• Novel networking systems and protocols to enable interactive immersive applications. This includes optimizations ranging from hardware (i.e., millimeter-wave networks or optical wireless), physical and MAC layer up to the network, transport and application layers (such as over the top protocols);
• Significative advances and optimization in 3D modeling pipelines for AR/VR visualization, accessible and inclusive GUI, interactive 3D models;
• Compression and delivery strategies for immersive media contents, such as omnidirectional video, light fields, point clouds, dynamic and time varying meshes;
• Quality of Experience management of interactive immersive media applications;
• Novel rendering techniques to enhance interactivity of XR applications;
• Application of interactive XR to different areas of society, such as health (i.e., virtual reality exposure therapy), industry (Industry 4.0), XR e-learning (according to new global aims);

Dates:

• Submission deadline: 05 July 2023, 23:59 AoE
• Notifications of acceptance: 30 July 2023
• Camera ready submission: 06 August 2023
• Workshop: 29th October to 3rd November

 International Conference on Visual Communications and Image Processing (VCIP)

1-4 December 2025

Klagenfurt, Austria

VCIP has a long tradition in showcasing pioneering technologies in visual communication and processing, and many landmark papers first appeared in VCIP. We will carry on this tradition of VCIP in disseminating the state of art of visual communication technology, brainstorming and envisioning the future of visual communication technology and applications.

General Chairs:

• Lu Yu (ZJU, CN)
• Shan Liu (Tencent, USA)
• Christian Timmerer (AAU, AT)

Technical Program Committee Chairs:

• Fernando Pereira (IST-IT, PT)
• Carla Pagliari (IME, BR)
• Hadi Amirpour (AAU, AT)

Plenary Session Chairs:

• Christine Guillemot (INRIA, FR)
• Ali Begen (OZU, TR)

Special Session Chairs:

• Jörn Ostermann (LUH, DE)
• Frederic Dufaux (CNRS, FR)

Tutorial Chairs:

• Eckehard Steinbach (TUM, DE)
• Roger Zimmermann (NUS, SG)

Publicity Chairs:

• Carl James Debono (UM, MT)
• Bruno Zatt (ViTech, BR)
• Wen-Huang Cheng (NYCU, TW)

Publication Chairs:

• Abdelhak Bentaleb (Concordia Univ., CA)
• Christian Herglotz (FAU, DE)

Industry Liaison:

• Iraj Sodagar (Tencent, USA)
• Michael Raulet (ATEME, FR)
• Christian Feldmann (Bitmovin, DE)
• Rufael Mekuria (Unified Streaming, NL)
• Debargha Mukherjee (Google, USA)

Demo, Open Source, Dataset Chairs:

• Daniel Silhavy (Fraunhofer FOKUS, DE)
• Farzad Tashtarian (AAU, AT)
• Werner Bailer (JOANNEUM RESEARCH, AT)

Doctoral Symposium Chairs:

• Angeliki Katsenou (TCD, IE)
• Mathias Wien (RWTH, DE)

Diversity and Inclusion Chairs:

• TBD
• Samira Afzal (AAU, AT)

Local Organization Team:

• Martina Steinbacher
• Margit Letter
• Mario Taschwer
• Rudi Messner

More Information to be announced.

IEEE Communications Surveys and Tutorials

[PDF]

Jeroen van der Hooft (Ghent University, Belgium), Hadi Amirpour (AAU, Austria), Maria Torres Vega (KU Leuven, Belgium), Yago Sanchez (Fraunhofer/HHI), Raimund Schatz (AIT, Austria), Thomas Schierl (Fraunhofer/HHI, Germany), and Christian Timmerer (AAU, Austria)

Abstract: Video services are evolving from traditional two-dimensional video to virtual reality and holograms, which offer six degrees of freedom to users, enabling them to freely move around in a scene and change focus as desired. However, this increase in freedom translates into stringent requirements in terms of ultra-high bandwidth (in the order of Gigabits per second) and minimal latency (in the order of milliseconds). To realize such immersive services, the network transport, as well as the video representation and encoding, have to be fundamentally enhanced. The purpose of this tutorial article is to provide an elaborate introduction to the creation, streaming, and evaluation of immersive video. Moreover, it aims to provide lessons learned and to point at promising research paths to enable truly interactive immersive video applications toward holography.

Keywords—Immersive video delivery, 3DoF, 6DoF, omnidirectional video, volumetric video, point clouds, meshes, light fields, holography, end-to-end systems

2023 ACM Mile High Video (MHV) 

May 7-10, 2023 | Denver, US

[PDF] [Slides]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Reza Farahani (Alpen-Adria-Universität Klagenfurt), Prajit T Rajendran (Universite Paris-Saclay), Mohammed Ghanbari (University of Essex), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt),  and Christian Timmerer (Alpen-Adria-Universität Klagenfurt).

Abstract:

In recent years, video streaming applications have proliferated the demand for Video Quality Assessment (VQA). Reduced reference video quality assessment (RR-VQA) is a category of VQA where certain features (e.g., texture, edges) of the original video are provided for quality assessment. It is a popular research area for various applications such as social media, online games, and video streaming. This paper introduces a reduced reference Transcoding Quality Prediction Model (TQPM) to determine the visual quality score of the video possibly transcoded in multiple stages. The quality is predicted using Discrete Cosine Transform (DCT)-energy-based features of the video (i.e., the video’s brightness, spatial texture information, and temporal activity) and the target bitrate representation of each transcoding stage. To do that, the problem is formulated, and a Long Short-Term Memory (LSTM)-based quality prediction model is presented. Experimental results illustrate that, on average, TQPM yields PSNR, SSIM, and VMAF predictions with an ?2 score of 0.83, 0.85, and 0.87, respectively, and Mean Absolute Error (MAE) of 1.31 dB, 1.19 dB, and 3.01, respectively, for single-stage transcoding.
Furthermore, an ?2 score of 0.84, 0.86, and 0.91, respectively, and MAE of 1.32 dB, 1.33 dB, and 3.25, respectively, are observed for a two-stage transcoding scenario. Moreover, the average processing time of TQPM for 4s segments is 0.328s, making it a practical VQA method in online streaming applications.

An example scenario of VQA in adaptive streaming applications.

IEEE Access, A Multidisciplinary, Open-access Journal of the IEEE

[PDF, GitHub, Slides, Video]

Babak Taraghi , Hermann Hellwagner and Christian Timmerer
(AAU, Austria)

Low-latency live streaming by HTTP Chunked Transfer EncodingAbstract: Live media streaming is a challenging task by itself, and when it comes to use cases that define low-latency as a must, the complexity will rise multiple times. In a typical media streaming session, the main goal can be declared as providing the highest possible Quality of Experience (QoE), which has proved to be measurable using quality models and various metrics. In a low-latency media streaming session, the requirements are to provide the lowest possible delay between the moment a frame of video is captured and the moment that the captured frame is rendered on the client screen, also known as end-to-end (E2E) latency and maintain the QoE. This paper proposes a sophisticated cloud-based and open-source testbed that facilitates evaluating a low-latency live streaming session as the primary contribution. Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation (LLL-CAdViSE) framework is enabled to asses the live streaming systems running on two major HTTP Adaptive Streaming (HAS) formats, Dynamic Adaptive Streaming over HTTP (MPEG-DASH) and HTTP Live Streaming (HLS). We use Chunked Transfer Encoding (CTE) to deliver Common Media Application Format (CMAF) chunks to the media players. Our testbed generates the test content (audiovisual streams). Therefore, no test sequence is required, and the encoding parameters (e.g., encoder, bitrate, resolution, latency) are defined separately for each experiment. We have integrated the ITU-T P.1203 quality model inside our testbed. To demonstrate the flexibility and power of LLL-CAdViSE, we have presented a secondary contribution in this paper; we have conducted a set of experiments with different network traces, media players, ABR algorithms, and with various requirements (e.g., E2E latency (typical/reduced/low/ultra-low), diverse bitrate ladders, and catch-up logic) and presented the essential findings and the experimental results.

Keywords: Live Streaming; Low-latency; HTTP Adaptive Streaming; Quality of Experience; Objective Evaluation, Open-source Testbed.

IEEE Access

[PDF]

Jesús Aguilar Armijo (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt) and Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)

Abstract: Multi-access Edge Computing (MEC) is a new paradigm that brings storage and computing close to the clients. MEC enables the deployment of complex network-assisted mechanisms for video streaming that improve clients’ Quality of Experience (QoE). One of these mechanisms is segment prefetching, which transmits the future video segments in advance closer to the client to serve content with lower latency. In this work, for HAS-based (HTTP Adaptive Streaming) video streaming and specifically considering a cellular (e.g., 5G) network edge, we present our approach Segment Prefetching and Caching at the Edge for Adaptive Video Streaming (SPACE). We propose and analyze different segment prefetching policies that differ in resource utilization, player and radio metrics needed, and deployment complexity. This variety of policies can dynamically adapt to the network’s current conditions and the service provider’s needs. We present segment prefetching policies based on diverse approaches and techniques: past segment requests, segment transrating (i.e., reducing segment bitrate/quality), Markov prediction model, machine learning to predict future segment requests, and super-resolution.We study their performance and feasibility using metrics such as QoE characteristics, computing times, prefetching hits, and link bitrate consumption. We analyze and discuss which segment prefetching policy is better under which circumstances, as well as the influence of the client-side Adaptive Bit Rate (ABR) algorithm and the set of available representations (“bitrate ladder”) in segment prefetching. Moreover, we examine the impact on segment prefetching of different caching policies for (pre-)fetched segments, including Least Recently Used (LRU), Least Frequently Used (LFU), and our proposed popularity-based caching policy Least Popular Used (LPU).

Keywords: Adaptive video streaming, content delivery, HAS, edge computing, cellular network edge, MEC, segment prefetching, segment caching.

How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions

Abstract: Empowered by today’s rich tools for media generation and collaborative production and convenient network access to the Internet, video streaming has become very popular. Dynamic adaptive video streaming is a technique used to deliver video content to users over the Internet, where the quality of the video adapts in real time based on the network conditions and the capabilities of the user’s device. HTTP Adaptive Streaming (HAS) has become the de-facto standard to provide a smooth and uninterrupted viewing experience, especially when network conditions frequently change. Improving the QoE of users concerning various applications‘ requirements presents several challenges, such as network variability, limited resources, and device heterogeneity. For example, the available network bandwidth can vary over time, leading to frequent changes in the video quality. In addition, different users have different preferences and viewing habits, which can further complicate live streaming optimization. Researchers and engineers have developed various approaches to optimize dynamic adaptive streaming, such as QoE-driven adaptation, machine learning-based approaches, and multi-objective optimization, to address these challenges. In this talk, we will give an introduction to the topic of video streaming and point out the significant challenges in the field. We will present a layered architecture for video streaming and then discuss a selection of approaches from our research addressing these challenges. For instance, we will present approaches to improve the  QoE of clients in User-generated content applications in centralized and distributed fashions. Moreover, we will present a novel architecture for low-latency live streaming that is agnostic to the protocol and codecs that can work equally with existing HAS-based approaches.