ATHENA Christian Doppler (CD) Laboratory

LMM-10K: Large-Scale 4K Multimodal Dataset for Perceptual, Semantic, and Content-Aware Video Processing

ACM Multimedia 2026

November 10 – November 14, 2026

Rio de Janeiro, Brazil

[PDF]

Mohammad Ghasempour (AAU, Austria), Yiying Wei (AAU, Austria), Hadi Amirpour (AAU, Austria), Christian Timmerer (AAU, Austria)

Abstract: The growing integration of vision and language models is driving a fundamental shift in video understanding and processing. This evolution calls for datasets that jointly capture visual content and its semantic representations at scale. To address this need, we introduce LMM-10K, a large-scale, curated multimodal dataset comprising 10,000 high-fidelity 4K video sequences at 60 fps with rich semantic and perceptual annotations. We developed an automated acquisition pipeline to curate videos from the Pexels repository, using targeted search queries and strict filtering criteria to capture a wide range of real-world scenes. Beyond the video sequences, LMM-10K is enriched with comprehensive multimodal annotations that integrate low-level visual features with high-level semantic information. These include LLM-generated semantic descriptors, no-reference quality metrics, spatial-temporal complexity metrics, and visual diversity attributes. By combining structured annotations with high-quality video data, LMM-10K provides a versatile resource for a wide range of applications, including video enhancement, content-aware compression and streaming, neural video coding, multimodal learning, generative video modeling, and perceptual quality modeling. Dataset URL: Link

Selective Multi-Pass Encoding for Cost-Efficient Video Streaming

International Broadcasting Convention (IBC)

[PDF]

Mohammad Ghasempour (AAU, Austria), Hadi Amirpour (AAU, Austria), and Christian Timmerer (AAU, Austria)

Abstract: As video streaming scales across platforms, resolutions, and devices, encoding efficiency has become critical to maintaining quality while controlling computational cost and energy consumption. Multi-pass encoding is widely used in streaming workflows to improve compression efficiency, rate-control accuracy, and quality consistency. However, its computational overhead is applied uniformly across all content, even when additional passes deliver minimal benefit. At scale, this results in unnecessary processing, higher computational cost, and increased energy consumption. This paper presents a content-adaptive strategy, CASE, that predicts whether additional encoding passes would provide meaningful gains using a lightweight mechanism that derives spatial and temporal features from each video segment. We evaluated the approach in two production-oriented scenarios using local and cloud-based video encoders. Results show that the method reduces computational time and encoding cost, with minimal impact on compression efficiency and visual quality. Experimental results show that CASE reduces encoding time by 25.3% on average with only a 2.23% bitrate increase, while its preprocessing and decision overhead is about 1155 times lower than multi-pass encoding time.

The 4th Workshop on Emerging Multimedia Systems (EMS) 2026

Cross-Layer Dynamics in Live Low-Latency: A Dataset of ABR, CC, and AQM Interactions

18th International Conference on Quality of Multimedia Experience

[PDF]

Md Tariqul Islam (UNICAMP, Brazil),  Farzad Tashtarian (AAU, Austria),  Christian Esteve Rothenberg (UNICAMP, Brazil), Christian Timmerer (AAU, Austria)

Low-latency video streaming, such as Low-Latency DASH (LL-DASH), requires maintaining high Quality of Experience (QoE) under varying network conditions. In LL-DASH, QoE is jointly influenced not only by Adaptive Bitrate (ABR) decisions, but also by transport-layer Congestion Control (CC) and network-layer Active Queue Management (AQM), whose interactions remain insufficiently characterized due to limited cross-layer experimentation. Therefore, we present a large-scale LL-DASH dataset comprising approximately 2,000 controlled sessions across three dash.js ABR algorithms (L2A, Dynamic, LoLP), three CC schemes (CUBIC, BBRv1, Prague) across both TCP and QUIC transport protocols, four AQM configurations (FIFO, FQ-CoDel, CAKE, DualPI2), and multiple congestion scenarios. The dataset supports QoE-aware cross-layer analysis and ABR benchmarking under diverse network configurations and is available at: https://github.com/cd-athena/ ll-dash-crosslayer-dataset

EVLM: Intent-Driven Edge Vision Language Model for UAV-Based Power Line Inspection

2026 IEEE International Conference on Edge Computing and Communications (IEEE EDGE 2026)

Reza Farahani (TU Wien, Austria), Zoha Azimi (AAU, Austria), Ilir Murturi (University of Prishtina, Kosovo), Arda Goknil (SINTEF, Norway), Sagar Sen (SINTEF, Norway), Christian Timmerer (AAU, Austria), Schahram Dustdar (TU Wien, Austria)

Abstract: Inspection of critical infrastructure, such as power lines, is increasingly conducted using unmanned aerial vehicles (UAVs) that capture aerial video for subsequent human review. Although recent edge-based approaches deploy onboard object detectors to identify predefined defect classes, these pipelines remain closed-set, task-specific, and largely decoupled from operator intent and edge resource constraints. This paper introduces EVLM, an intent-driven vision-language framework for onboard UAV-based power line inspection. Given a high-level operator intent, EVLM (i) leverages lightweight histogram-based frame filtering to extract salient key frames under bounded compute budgets, (ii) executes a domain-adapted vision language model (VLM) directly on the UAV for intent-conditioned multimodal reasoning, and (iii) synthesizes structured inspection reports together with a minimal set of evidence frames, replacing continuous raw video transmission with compact semantic outputs. To align the VLM with infrastructure inspection semantics while preserving edge efficiency, we perform parameter-efficient fine-tuning using Low-Rank Adaptation (LoRA), enabling domain specialization without updating the full model parameters. We implement and fully deploy EVLM on an NVIDIA Jetson device representative of UAV-class onboard hardware and evaluate it using 20 publicly released power line inspection video sequences spanning 8 heterogeneous environments and 5 operational intent categories. Experimental results show a data reduction of 94.8 %, with transmitted data decreasing from 485 kB to 25 kB per 4 s segment, corresponding to 72.75 MB versus 3.75 MB over a 10 min inspection mission. EVLM operates feasibly on embedded hardware, maintaining moderate CPU/GPU utilization and bounded power consumption (5.6 W), while producing interpretable, intent-aligned inspection outputs with richer semantic insights than detection-centric baselines.

Assistant Prof. Dr. Hadi Amirpour has been elevated to IEEE Senior Member in recognition of his contributions to multimedia streaming systems.

IEEE Senior Member is the highest professional grade for which an IEEE member can apply. This distinction requires extensive professional experience and demonstrated accomplishments that reflect technical expertise, leadership, and professional maturity. Fewer than 10% of IEEE’s nearly half a million members worldwide have achieved this honor.