Selective Multi-Pass Encoding for Cost-Efficient Video Streaming

International Broadcasting Convention (IBC)

[PDF]

Mohammad Ghasempour (AAU, Austria), Hadi Amirpour (AAU, Austria), and Christian Timmerer (AAU, Austria)

Abstract: As video streaming scales across platforms, resolutions, and devices, encoding efficiency has become critical to maintaining quality while controlling computational cost and energy consumption. Multi-pass encoding is widely used in streaming workflows to improve compression efficiency, rate-control accuracy, and quality consistency. However, its computational overhead is applied uniformly across all content, even when additional passes deliver minimal benefit. At scale, this results in unnecessary processing, higher computational cost, and increased energy consumption. This paper presents a content-adaptive strategy, CASE, that predicts whether additional encoding passes would provide meaningful gains using a lightweight mechanism that derives spatial and temporal features from each video segment. We evaluated the approach in two production-oriented scenarios using local and cloud-based video encoders. Results show that the method reduces computational time and encoding cost, with minimal impact on compression efficiency and visual quality. Experimental results show that CASE reduces encoding time by 25.3% on average with only a 2.23% bitrate increase, while its preprocessing and decision overhead is about 1155 times lower than multi-pass encoding time.

Beyond One-Size-Fits-All: Adaptive and Computationally Efficient Video Streaming

Hadi Amirpour

Habilitation

On 25.06.2026, Dr. Hadi Amirpour defended his habilitation “Beyond One-Size-Fits-All: Adaptive and Computationally Efficient Video Streaming” at the University of Klagenfurt, Austria.

Abstract:

Video streaming has become an integral and ever-growing component of daily life, driving increasing demands for efficient, scalable, and high-quality delivery across heterogeneous networks and devices. This habilitation thesis investigates novel methods to improve the compression, computation, and perceptual performance of video streaming systems across the entire delivery pipeline. A central focus is placed on bitrate ladder construction, where the same video content is encoded into multiple representations, typically at different bitrates and resolutions, to enable adaptive streaming. Although a one-size-fits-all approach is widely adopted due to its simplicity and ease of deployment, it is inherently suboptimal, as it neglects content characteristics and network variability, leading to inefficient resource utilization and reduced quality of experience (QoE).

This habilitation thesis first presents advanced bitrate ladder optimization strategies encompassing both content-aware and network-aware approaches to determine the appropriate number of representations and their associated bitrates for video-on-demand and live streaming scenarios. The optimization is further extended to a multi-dimensional representation space, including spatial and temporal resolution, bit depth, encoder presets, and device capabilities, with explicit consideration of compression efficiency, computational cost, and energy consumption. These contributions establish the foundation for deciding what should be encoded and delivered in adaptive streaming systems.

However, determining optimal representations through exhaustive encoding is computationally prohibitive, particularly in live streaming scenarios. To address this limitation, efficient feature extraction and encoding-aware as well as semantic-aware video complexity analysis methods are introduced. These predictive models enable fast and reliable content characterization without requiring brute-force encoding, thereby  supporting scalable and real-time bitrate ladder construction and encoder configuration.

Once the bitrate ladder is determined, this habilitation thesis further investigates fast multi-rate encoding approaches, which typically exploit encoding decisions from one representation to accelerate the encoding of other representations, thereby reducing computational redundancy while maintaining rate–distortion performance. Additionally, lightweight transcoding techniques are developed for cloud and edge environments, where metadata generated in the cloud is leveraged to accelerate subsequent transcoding processes. Finally, quality evaluation is addressed as a key cross-cutting aspect, focusing on the measurement and modeling of QoE to ensure that system-level optimizations translate into perceptually meaningful improvements for end users.

This habilitation thesis also includes technical advances in multimedia security and privacy, such as reversible data hiding, selective encryption, and cryptographic analysis, ensuring that efficiency and scalability do not compromise confidentiality, integrity, and trustworthiness in modern streaming ecosystems. It is complemented by survey and tutorial contributions that position the work within the broader research landscape and consolidate the presented methods within a unified scientific context.

Taken together, the research contributions presented in this habilitation thesis establish a coherent progression for adaptive video streaming systems, addressing what to encode and deliver through adaptive and multi-dimensional bitrate ladder construction; how to predict optimal encoding decisions efficiently using encoding-aware and semantic visual complexity modeling; how to produce optimized representations at scale via accelerated multi-rate encoding and lightweight edge transcoding; how to evaluate system decisions perceptually through QoE modeling and real-time quality metrics; and how to protect multimedia content in distributed cloud and edge infrastructures through format-compliant encryption and security mechanisms. This integrated perspective advances video streaming as a cross-layer, end-to-end optimization framework that jointly considers compression efficiency, computational scalability, perceptual quality, and security within modern multimedia delivery ecosystems.

Keywords: Video streaming; bitrate ladder construction; encoding optimization; video complexity analysis; Quality of Experience (QoE); accelerated encoding; multimedia security.

The 4th Workshop on Emerging Multimedia Systems (EMS) 2026

Cross-Layer Dynamics in Live Low-Latency: A Dataset of ABR, CC, and AQM Interactions

18th International Conference on Quality of Multimedia Experience

[PDF]

Md Tariqul Islam (UNICAMP, Brazil),  Farzad Tashtarian (AAU, Austria),  Christian Esteve Rothenberg (UNICAMP, Brazil), Christian Timmerer (AAU, Austria)

Low-latency video streaming, such as Low-Latency DASH (LL-DASH), requires maintaining high Quality of Experience (QoE) under varying network conditions. In LL-DASH, QoE is jointly influenced not only by Adaptive Bitrate (ABR) decisions, but also by transport-layer Congestion Control (CC) and network-layer Active Queue Management (AQM), whose interactions remain insufficiently characterized due to limited cross-layer experimentation. Therefore, we present a large-scale LL-DASH dataset comprising approximately 2,000 controlled sessions across three dash.js ABR algorithms (L2A, Dynamic, LoLP), three CC schemes (CUBIC, BBRv1, Prague) across both TCP and QUIC transport protocols, four AQM configurations (FIFO, FQ-CoDel, CAKE, DualPI2), and multiple congestion scenarios. The dataset supports QoE-aware cross-layer analysis and ABR benchmarking under diverse network configurations and is available at: https://github.com/cd-athena/ ll-dash-crosslayer-dataset

EVLM: Intent-Driven Edge Vision Language Model for UAV-Based Power Line Inspection

2026 IEEE International Conference on Edge Computing and Communications (IEEE EDGE 2026)

Reza Farahani (TU Wien, Austria), Zoha Azimi (AAU, Austria), Ilir Murturi (University of Prishtina, Kosovo), Arda Goknil (SINTEF, Norway), Sagar Sen (SINTEF, Norway), Christian Timmerer (AAU, Austria), Schahram Dustdar (TU Wien, Austria)

Abstract: Inspection of critical infrastructure, such as power lines, is increasingly conducted using unmanned aerial vehicles (UAVs) that capture aerial video for subsequent human review. Although recent edge-based approaches deploy onboard object detectors to identify predefined defect classes, these pipelines remain closed-set, task-specific, and largely decoupled from operator intent and edge resource constraints. This paper introduces EVLM, an intent-driven vision-language framework for onboard UAV-based power line inspection. Given a high-level operator intent, EVLM (i) leverages lightweight histogram-based frame filtering to extract salient key frames under bounded compute budgets, (ii) executes a domain-adapted vision language model (VLM) directly on the UAV for intent-conditioned multimodal reasoning, and (iii) synthesizes structured inspection reports together with a minimal set of evidence frames, replacing continuous raw video transmission with compact semantic outputs. To align the VLM with infrastructure inspection semantics while preserving edge efficiency, we perform parameter-efficient fine-tuning using Low-Rank Adaptation (LoRA), enabling domain specialization without updating the full model parameters. We implement and fully deploy EVLM on an NVIDIA Jetson device representative of UAV-class onboard hardware and evaluate it using 20 publicly released power line inspection video sequences spanning 8 heterogeneous environments and 5 operational intent categories. Experimental results show a data reduction of 94.8 %, with transmitted data decreasing from 485 kB to 25 kB per 4 s segment, corresponding to 72.75 MB versus 3.75 MB over a 10 min inspection mission. EVLM operates feasibly on embedded hardware, maintaining moderate CPU/GPU utilization and bounded power consumption (5.6 W), while producing interpretable, intent-aligned inspection outputs with richer semantic insights than detection-centric baselines.

Assistant Prof. Dr. Hadi Amirpour has been elevated to IEEE Senior Member in recognition of his contributions to multimedia streaming systems.

IEEE Senior Member is the highest professional grade for which an IEEE member can apply. This distinction requires extensive professional experience and demonstrated accomplishments that reflect technical expertise, leadership, and professional maturity. Fewer than 10% of IEEE’s nearly half a million members worldwide have achieved this honor.