MMSys ’24: ComPEQ–MR: Compressed Point Cloud Dataset with Eye Tracking and Quality Assessment in Mixed Reality

15th ACM Multimedia Systems Conference

April 15-18, 2024 – Bari, Italy

https://2024.acmmmsys.org/

[PDF],[Dataset]

Minh Nguyen (Fraunhofer Fokus, Germany), Shivi Vats (Alpen-Adria-Universität Klagenfurt, Austria), Xuemei Zhou (Centrum Wiskunde & Informatica and TU Delft, Netherlands), Irene Viola (Centrum Wiskunde & Informatica, Netherlands), Pablo Cesar (Centrum Wiskunde & Informatica, Netherlands), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Austria) Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt, Austria)

Abstract: Point clouds (PCs) have attracted researchers and developers due to their ability to provide immersive experiences with six degrees of freedom (6DoF). However, there are still several open issues in understanding the Quality of Experience (QoE) and visual attention of end users while experiencing 6DoF volumetric videos. First, encoding and decoding point clouds require a significant amount of both time and computational resources. Second, QoE prediction models for dynamic point clouds in 6DoF have not yet been developed due to the lack of visual quality databases. Third, visual attention in 6DoF is hardly explored, which impedes research into more sophisticated approaches for adaptive streaming of dynamic point clouds. In this work, we provide an open-source Compressed Point cloud dataset with Eye-tracking and Quality assessment in Mixed Reality (ComPEQ–MR). The dataset comprises four compressed dynamic point clouds processed by Moving Picture Experts Group (MPEG) reference tools (i.e., VPCC and GPCC), each with 12 distortion levels. We also conducted subjective tests to assess the quality of the compressed point clouds with different levels of distortion. The rating scores are attached to ComPEQ–MR so that they can be used to develop QoE prediction models in the context of MR environments. Additionally, eye-tracking data for visual saliency is included in this dataset, which is necessary to predict where people look when watching 3D videos in MR experiences. We collected opinion scores and eye-tracking data from 41 participants, resulting in 2132 responses and 164 visual attention maps in total. The dataset is available at https://ftp.itec.aau.at/datasets/ComPEQ-MR/.

Index Terms: Point Clouds, Quality of Experience, Subjective Tests, Augmented Reality

Posted in SPIRIT | Comments Off on MMSys ’24: ComPEQ–MR: Compressed Point Cloud Dataset with Eye Tracking and Quality Assessment in Mixed Reality

MHV ’24: No-Reference Quality of Experience Model for Dynamic Point Clouds in Augmented Reality

ACM Mile High Video (MHV) 2024

February 11-14, 2024 – Denver, USA

https://www.mile-high.video/

[PDF],[GitHub]

Minh Nguyen (Alpen-Adria-Universität Klagenfurt, Austria), Shivi Vats (Alpen-Adria-Universität Klagenfurt, Austria), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt, Austria)

Abstract: Point cloud streaming is becoming increasingly popular due to its ability to provide six degrees of freedom (6DOF) for immersive media. Measuring the quality of experience (QoE) is essential to evaluate the performance of point cloud applications. However, most existing QoE models for point cloud streaming are complicated and/or not open source. Therefore, it is desirable to provide an opensource QoE model for point cloud streaming.

(…)

In this work, we provide a fine-tuned ITU-T P.1203 model for dynamic point clouds in Augmented Reality (AR) environments. We re-train the P.1203 model with our dataset to get the optimal coefficients in this model that achieves the lowest root mean square error (RMSE). The dataset was collected in a subjective test in which the participants watched dynamic point clouds from the 8i lab database with Microsoft’s HoloLens 2 AR glasses. The dynamic point clouds have static qualities or a quality switch in the/ middle of the sequence. We split this dataset into a training set and a validation set. We train the coefficients of the P.1203 model with the former set and validate its performance with the latter one.

The trained model is available on Github: https://github.com/minhkstn/itu-p1203-point-clouds.

Index Terms: Point Clouds, Quality of Experience, Subjective Tests, Augmented Reality

Posted in SPIRIT | Comments Off on MHV ’24: No-Reference Quality of Experience Model for Dynamic Point Clouds in Augmented Reality

EVCA: Enhanced Video Complexity Analyzer

The 15th ACM Multimedia Systems Conference (Technical Demos)

15-18 April, 2024 in Bari, Italy

[PDF],[Github]

Hadi Amirpour (AAU, Austria), Mohammad Ghasempour (AAU, Austria), Lingfen Qu (Guangzhou University, China), Wassim Hamidouche (TII, UAE), and Christian Timmerer (AAU, Austria)

The optimization of video compression and streaming workflows critically relies on understanding the video complexity, including both spatial and temporal features. These features play a vital role in guiding rate control, predicting video encoding parameters (such as resolution and frame rate), and selecting test videos for subjective analysis. Traditional methods primarily utilize SI and TI to measure these spatial and temporal complexity features, respectively. Moreover, VCA has been introduced as a tool employing DCT-based functions to evaluate these features, specifically E and h for spatial and temporal complexity features, respectively. In this paper, we introduce Enhanced Video Complexity Analyzer(EVCA), an advanced tool that integrates the functionalities of both VCA and the SITI approach. Developed in Python to ensure compatibility with GPU processing, EVCA enhances the definition of temporal complexity originally used in VCA. This refinement significantly improves the detection of temporal complexity features in VCA (i.e., h), raising its Peasrson Correlation Coefficient (PCC) from 0.6 to 0.77. Furthermore, EVCA demonstrates exceptional performance on GPU devices, achieving feature extraction speeds exceeding 1200 fps for 1080p resolution videos.

Posted in ATHENA | Comments Off on EVCA: Enhanced Video Complexity Analyzer

GREEM : An Open-Source Energy Measurement Tool for Video Processing

GREEM: An Open-Source Benchmark Tool Measuring the Environmental Footprint of Video Streaming

The 15th ACM Multimedia Systems Conference (Open-source Software and Datasets)

15-18 April, 2024 in Bari, Italy

[PDF], [Github]

Christian Bauer  (AAU, Austria),  Samira Afzal (AAU, Austria)Sandro Linder (AAU, Austria), Radu Prodan (AAU,Austria)and Christian Timmerer (AAU, Austria)

Addressing climate change requires a global decrease in greenhouse gas (GHG) emissions. In today’s digital landscape, video streaming significantly influences internet traffic, driven by the widespread use of mobile devices and the rising popularity of streaming plat-
forms. This trend emphasizes the importance of evaluating energy consumption and the development of sustainable and eco-friendly video streaming solutions with a low Carbon Dioxide (CO2) footprint. We developed a specialized tool, released as an open-source library called GREEM , addressing this pressing concern. This tool measures video encoding and decoding energy consumption and facilitates benchmark tests. It monitors the computational impact on hardware resources and offers various analysis cases. GREEM is helpful for developers, researchers, service providers, and policy makers interested in minimizing the energy consumption of video encoding and streaming.

Posted in GAIA | Comments Off on GREEM : An Open-Source Energy Measurement Tool for Video Processing

VEEP: Video Encoding Energy and CO2 Emission Prediction

VEEP: Video Encoding Energy and CO2 Emission Prediction

ACM MMsys, GMSys workshop (2024)

15-18 April, 2024 in Bari, Italy

[PDF], [Slides]

Manuel Hoi* (AAU,  Austria), Armin Lachini* (AAU, Austria), Samira Afzal (AAU, Austria), Sandro Linder (AAU, Austria), Farzad Tashtarian (AAU, Austria), Radu Prodan (AAU, Austria), and Christian Timmerer (AAU, Austria)

*These authors contributed equally to this work

In the context of rising environmental concerns, this paper introduces VEEP, an architecture designed to predict energy consumption and CO2 emissions in cloud-based video encoding. VEEP combines video analysis with machine learning (ML)-based energy prediction and real-time carbon intensity, enabling precise estimations of CPU energy usage and CO2 emissions during the encoding process. It is trained on the Video Complexity Dataset (VCD) and encoding results from various AWS EC2 instances. VEEP achieves high accuracy, indicated by an 𝑅2-score of 0.96, a mean absolute error (MAE) of 2.41 × 105, and a mean squared error (MSE) of 1.67 × 109. An important finding is the potential to reduce emissions by up to 375 times when comparing cloud instances and their locations. These results highlight the importance of considering environmental factors in cloud computing.

 

Posted in GAIA | Comments Off on VEEP: Video Encoding Energy and CO2 Emission Prediction

VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances

VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances

The 15th ACM Multimedia Systems Conference (Open-source Software and Datasets)

15-18 April, 2024 in Bari, Italy

[PDF],[Github]

Sandro Linder (AAU, Austria), Samira Afzal (AAU, Austria), Christian Bauer  (AAU, Austria), Hadi Amirpour (AAU, Austria), Radu Prodan (AAU,Austria)and Christian Timmerer (AAU, Austria)

Video streaming constitutes 65 % of global internet traffic, prompting an investigation into its energy consumption and CO2 emissions. Video encoding, a computationally intensive part of streaming, has moved to cloud computing for its scalability and flexibility. However, cloud data centers’ energy consumption, especially video encoding, poses environmental challenges. This paper presents VEED, a FAIR Video Encoding Energy and CO2 Emissions Dataset for Amazon Web Services (AWS) EC2 instances. Additionally, the dataset also contains the duration, CPU utilization, and cost of the encoding. To prepare this dataset, we introduce a model and conduct a benchmark to estimate the energy and CO2 emissions of different Amazon EC2 instances during the encoding of 500 video segments with various complexities and resolutions using Advanced Video Coding (AVC)
and High-Efficiency Video Coding (HEVC). VEED and its analysis can provide valuable insights for video researchers and engineers to model energy consumption, manage energy resources, and distribute workloads, contributing to the sustainability of cloud-based video encoding and making them cost-effective. VEED is available at Github.

Posted in GAIA | Comments Off on VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances

PyStream: Enhancing Video Streaming Evaluation

The 15th ACM Multimedia Systems Conference (Technical Demos)

15-18 April, 2024 | Bari, Italy

Conference website

[PDF] [Github]

Samuel Radler* (AAU, Austria) , Leon Prüller* (AAU, Austria), Emanuele Artioli (AAU, Austria), Farzad Tashtarian (AAU, Austria), and Christian Timmerer (AAU, Austria)

*These authors contributed equally to this work

As streaming services become more commonplace, analyzing their behavior effectively under different network conditions is crucial. This is normally quite expensive, requiring multiple players with different bandwidth configurations to be emulated by a powerful local machine or a cloud environment. Furthermore, emulating a realistic network behavior or guaranteeing adherence to a real network trace is challenging. This paper presents PyStream, a simple yet powerful way to emulate a video streaming network, allowing multiple simultaneous tests to run locally. By leveraging a network of Docker containers, many of the implementation challenges are abstracted away, keeping the resulting system easily manageable and upgradeable. We demonstrate how PyStream not only reduces the requirements for testing a video streaming system but also improves the accuracy of the emulations with respect to the current state-of-the-art. On average, PyStream reduces the error between the original network trace and the bandwidth emulated by video players by a factor of 2-3 compared to Wondershaper, a common network traffic shaper in many video streaming evaluation environments. Moreover, PyStream decreases the cost of running experiments compared to existing cloud-based video streaming evaluation environments such as CAdViSE.

 

Posted in ATHENA | Comments Off on PyStream: Enhancing Video Streaming Evaluation