Efficient Content-Adaptive Feature-based Shot Detection for HTTP Adaptive Streaming
IEEE International Conference on Image Processing (ICIP)
September 19-22, 2021, Alaska, USA.
Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Mohammad Ghanbari (School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK), and Christian Timmerer (Alpen-Adria-Universität Klagenfurt).
Video delivery over the Internet has been becoming a commodity in recent years, owing to the widespread use of DASH. The DASH specification defines a hierarchical data model for Media Presentation Descriptions (MPDs) in terms of segments. This paper focuses on segmenting video into multiple shots for encoding in VoD HAS applications.
This paper proposes a novel DCT feature-based shot detection and successive elimination algorithm for shot detection algorithm and benchmark the algorithm against the default shot detection algorithm of the x265 implementation of the HEVC standard. Our experimental results demonstrate that the proposed feature-based pre-processor has a recall rate of 25% and an F-measure of 20% greater than the benchmark algorithm for shot detection.
Keywords: HTTP Adaptive Streaming, Video-on-Demand, Shot detection, multi-shot encoding.
Fast Multi-Resolution and Multi-Rate Encoding for HTTP Adaptive Streaming Using Machine Learning
IEEE Open Journal of Signal Processing
Ekrem Çetinkaya (Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität Klagenfurt), Hadi Amirpour, (Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität Klagenfurt), and Mohammad Ghanbari (Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität Klagenfurt, University of Essex)
Abstract: Video streaming applications keep getting more attention over the years, and HTTP Adaptive Streaming (HAS) became the de-facto solution for video delivery over the Internet. In HAS, each video is encoded at multiple quality levels and resolutions (i.e., representations) to enable adaptation of the streaming session to viewing and network conditions of the client. This requirement brings encoding challenges along with it, e.g., a video source should be encoded efficiently at multiple bitrates and resolutions. Fast multi-rate encoding approaches aim to address this challenge of encoding multiple representations from a single video by re-using information from already encoded representations. In this paper, a convolutional neural network is used to speed up both multi-rate and multi-resolution encoding for HAS. For multi-rate encoding, the lowest bitrate representation is chosen as the reference. For multi-resolution encoding, the highest bitrate from the lowest resolution representation is chosen as the reference. Pixel values from the target resolution and encoding information from the reference representation are used to predict Coding Tree Unit (CTU) split decisions in High-Efficiency Video Coding (HEVC) for dependent representations. Experimental results show that the proposed method for multi-rate encoding can reduce the overall encoding time by 15.08% and parallel encoding time by 41.26%, with a 0.89% bitrate increase compared to the HEVC reference software. Simultaneously, the proposed method for multi-resolution encoding can reduce the encoding time by 46.27% for the overall encoding and 27.71% for the parallel encoding on average with a 2.05% bitrate
Keywords: HTTP Adaptive Streaming, HEVC, Multirate Encoding, Machine Learning
Students in MobiSys (SMS)
In Conjugation with MobiSys 2021
June 26, 2021, Mars, Solar System, Milky Way
- Hannaneh Barahouei Pasandi, Virginia Commonwealth University
- Mallesham Dasari, Stony Brook University
- Hadi Amirpour, University of Klagenfurt
Students Workshop in MobiSys (SMS): The SMS workshop provides a unique venue for graduate students around the world to discuss research ideas for mobile and wireless systems. The workshop is organized by a student-run TPC. SMS workshop aims to foster early-career development among students, expose them to the workings of academic life, and encourage student leadership and participation in the research community. SMS 2021 will be held in conjunction with MobiSys 2021 .
SMS Workshop provides a unique venue for graduate students around the world to present, discuss, and exchange ideas on cross-cutting research on mobile wireless networks. As its name suggests, the workshop is organized by students, and the technical sessions are given by student presenters. Submissions must be first-authored by a student. The workshop aims at fostering early-career development among students and exposing them to the workings of academic life. It provides a venue for students to learn about each others’ work and discover opportunities for collaboration.
Topics of interest include, but are not limited to:
- Experience with mobile applications, networks, and systems
- Innovative mobile, mobile sensing, and mobile crowdsourcing applications
- Tools for building and measuring mobile systems
- Innovative wearable or mobile devices
- Novel software architectures for mobile devices and mobile computing
- Data management for mobile applications
- Infrastructure support for mobile computing
- System-level energy management for mobile devices
- Operating systems for mobile devices
- Support for mobile social networking and the mobile web
- Security and privacy in mobile systems
- Resource-efficient machine learning and AI for mobile devices
- Systems for location and context sensing and awareness
- Mobile computing support for pervasive computing Vehicular and mobile robotic systems
- Systems and networking support for virtual or augmented reality
- Applications of mobile systems in health, sustainability, and smart cities
- Satellites Communication and networks
- Architectures, protocols, and algorithms in mobile network
- Measurements of mobile and network ecosystems
- Mobile data science & analysis
- Sensing using mobile phones, wearables, robots, quad-copters, crowd-sourcing etc
- Operating system and middle-ware support for mobile computing and networking
- Modeling, measurement and simulation of mobile networks
- Applications of machine learning to mobile/wireless research
- Mobile web, video, AR/VR/Immersive reality, and other applications
- User interfaces, experience, and usability for mobile applications and systems
- Data management for mobile and wireless systems
The workshop invites students to submit papers, posters, and demos through the following hotcrp link: https://sms2021.hotcrp.com/ . All submissions will be peer-reviewed by the student TPC. We encourage students with a paper, poster, or demo at ACM MobiSys main conference to present their work at SMS as well. Please contact the TPC co-chairs at firstname.lastname@example.org for any queries.
- Submission Deadline: June 04, 2021
- Acceptance Notifications: June 08, 2021
- Camera Ready Deadline: June 11, 2021
Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming
Picture Coding Symposium (PCS)
29 June-2 July 2021, Bristol, UK
Vignesh V Menon (Alpen-Adria-Universität Klagenfurt), Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), and Mohammad Ghanbari (School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK)
Since video accounts for the majority of today’s internet traffic, the popularity of HTTP Adaptive Streaming (HAS) is increasing steadily. In HAS, each video is encoded at multiple bitrates and spatial resolutions (i.e., representations) to adapt to a heterogeneity of network conditions, device characteristics, and end-user preferences. Most of the streaming services utilize cloud-based encoding techniques which enable a fully parallel encoding process to speed up the encoding and consequently to reduce the overall time complexity. State-of-the-art approaches further improve the encoding process by utilizing encoder analysis information from already encoded representation(s) to improve the encoding time complexity of the remaining representations. In this paper, we investigate various multi-encoding algorithms (i.e., multi-rate and multi-resolution) and propose novel multi- encoding algorithms for large-scale HTTP Adaptive Streaming deployments. Experimental results demonstrate that the proposed multi-encoding algorithm optimized for the highest compression efficiency reduces the overall encoding time by 39% with a 1.5% bitrate increase compared to stand-alone encodings. Its optimized version for the highest time savings reduces the overall encoding time by 50% with a 2.6% bitrate increase compared to stand-alone encodings.
Keywords: HTTP Adaptive Streaming, HEVC, Multi-rate Encoding, Multi-encoding.
NOSSDAV’21: The 31st edition of the Workshop on Network and Operating System Support for Digital Audio and Video
Sept. 28-Oct. 1, 2021, Istanbul, Turkey
Reza Farahani (Alpen-Adria-Universität Klagenfurt), Farzad Tashtarian (Alpen-Adria-Universität Klagenfurt), Alireza Erfanian (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Mohammad Ghanbari (School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK) and Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)
Abstract: Recently, HTTP Adaptive Streaming (HAS) has become the dominant video delivery technology over the Internet. In HAS, clients have full control over the media streaming and adaptation processes. Lack of coordination among the clients and lack of awareness of the network conditions may lead to sub-optimal user experience, and resource utilization in a pure client-based HAS adaptation scheme. Software-Defined Networking (SDN) has recently been considered to enhance the video streaming process. In this paper, we leverage the capability of SDN and Network Function Virtualization (NFV) to introduce an edge- and SDN-assisted video streaming framework called ES-HAS. We employ virtualized edge components to collect HAS clients’ requests and retrieve networking information in a time-slotted manner. These components then perform an optimization model in a time-slotted manner to efficiently serve clients’ requests by selecting an optimal cache server (with the shortest fetch time). In case of a cache miss, a client’s request is served (i) by an optimal replacement quality (only better quality levels with minimum deviation) from a cache server, or (ii) by the original requested quality level from the origin server. This approach is validated through experiments on a large-scale testbed, and the performance of our framework is compared to pure client-based strategies and the SABR system . Although SABR and ES-HAS show (almost) identical performance in the number of quality switches, ES-HAS outperforms SABR in terms of playback bitrate and the number of stalls by at least 70% and 40%, respectively.
Keywords: Dynamic Adaptive Streaming over HTTP (DASH), Edge Computing, Network-Assisted Video Streaming, Quality of Experience (QoE), Software Defined Networking (SDN), Network Function Virtualization (NFV)
PSTR: Per-title encoding using Spatio-Temporal Resolutions
IEEE International Conference on Multimedia and Expo (ICME)
5-9 July 2021, Shenzhen, China
Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), and Mohammad Ghanbari (School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK)
Current per-title encoding schemes encode the same video content (or snippets/subsets thereof) at various bitrates and spatial resolutions to find an optimal bitrate ladder for each video content. Compared to traditional approaches, in which a predefined, content-agnostic (“fit-to-all”) encoding ladder is applied to all video contents, per-title encoding can result in (i) a significant decrease of storage and delivery costs and (ii) an increase in the Quality of Experience. In the current per-title encoding schemes, the bitrate ladder is optimized using only spatial resolutions, while we argue that with the emergence of high framerate videos, this principle can be extended to temporal resolutions as well. In this paper, we improve the per-title encoding for each content using spatio-temporal resolutions. Experimental results show that our proposed approach doubles the performance of bitrate saving by considering both temporal and spatial resolutions compared to considering only spatial resolutions.
Keywords: Bitrate ladder, per-title encoding, framerate, spatial resolution.
NOSSDAV’21: The 31st edition of the Workshop on Network and Operating System Support for Digital Audio and Video
Sept. 28-Oct. 1, 2021, Istanbul, Turkey
Babak Taraghi (Alpen-Adria-Universität Klagenfurt), Abdelhak Bentaleb (National University of Singapore), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Roger Zimmermann (National University of Singapore) and Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)
Abstract: Adaptive BitRate (ABR) algorithms play a crucial role in delivering the highest possible viewer’s Quality of Experience (QoE) in HTTP Adaptive Streaming (HAS). Online video streaming service providers use HAS – the dominant video streaming technique on the Internet – to deliver the best QoE for their users. Viewer’s delightfulness relies heavily on how the ABR of a media player can adapt the stream’s quality to the current network conditions. QoE for end-to-end video streaming sessions has been evaluated in many research projects to give better insight into the quality metrics. Objective evaluation models such as ITU Telecommunication Standardization Sector (ITU-T) P.1203 allow for the calculation of Mean Opinion Score (MOS) by considering various QoE metrics, and subjective evaluation is the best assessment approach in investigating the end-user opinion over a video streaming session’s experienced quality. We have conducted subjective evaluations with crowdsourced participants and evaluated the MOS of the sessions using the ITU-T P.1203 quality model. This paper’s main contribution is subjective evaluation analogy with objective evaluation for well-known heuristic-based ABRs.
Keywords: HTTP Adaptive Streaming, ABR Algorithms, Quality of Experience, Crowdsourcing, Subjective Evaluation, Objective Evaluation, MOS, (ITU-T) P.1203
Christian Timmerer, Associate Professor at the Institute of Information Technology (ITEC) and Director of the ATHENA Christian Doppler Laboratory, has been appointed IEEE Communications Society Distinguished Lecturer for the term 2021-2022.
“The Distinguished Lecturer Program (DLP) connects Senior IEEE ComSoc members, who are renowned communications technology experts, with ComSoc chapters so they can share their knowledge, expertise, and insights into the future of communications technology.”
In the context of the Distinguished Lecturer Program (DLP), Christian Timmerer will offer the following (virtual) lecture topics:
- HTTP Adaptive Streaming (HAS) — Quo Vadis?
- Quality of Experience (QoE) for Traditional and Immersive Media Services
- Immersive Media Services: from Encoding to Consumption
- 20 Years of Streaming in 20 Minutes
- Multimedia Communication, Networking, Protocols, Delivery
- Multimedia Standards (MPEG, IETF, W3C)
The details of how to request a Distinguished Lecturer can be found here.
We are happy and proud to see Bitmovin among the 72nd Annual Technology & Engineering Emmy® Awards Recipients. The award is received for “Development of Massive Processing Optimized Compression Technologies” which acknowledges Bitmovin’s Encoding product including its reportedly best per-title encoding feature.
Bitmovins press release can be found here and approximately one year we had the official opening ceremony of the ATHENA project that will continue to fed the innovation pipeline with respect to HTTP Adaptive Streaming (HAS) and beyond. Please see our latest publications in this field and in case of questions please do not hesitate to contact us.
OSCAR: On Optimizing Resource Utilization in Live Video Streaming
IEEE Transactions on Network and Service Management (TNSM)
Alireza Erfanian (Alpen-Adria-Universität Klagenfurt), Farzad Tashtarian (Alpen-Adria-Universität Klagenfurt), Anatoliy Zabrovskiy (Alpen-Adria-Universität Klagenfurt), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)
Abstract: Live video streaming traffic and related applications have experienced significant growth in recent years. However, this has been accompanied by some challenging issues, especially in terms of resource utilization. Although IP multicasting can be recognized as an efficient mechanism to cope with these challenges, it suffers from many problems. Applying software-defined networking (SDN) and network function virtualization (NFV) technologies enable researchers to cope with IP multicasting issues in novel ways. In this paper, by leveraging the SDN concept, we introduce OSCAR (Optimizing reSourCe utilizAtion in live video stReaming) as a new cost-aware video streaming approach to provide advanced video coding (AVC)-based live streaming services in the network. In this paper, we use two types of virtualized network functions (VNFs): virtual reverse proxy (VRP) and virtual transcoder function (VTF). At the edge of the network, VRPs are responsible for collecting clients’ requests and sending them to an SDN controller. Then, by executing a mixed-integer linear program (MILP), the SDN controller determines a group of optimal multicast trees for streaming the requested videos from an appropriate origin server to the VRPs. Moreover, to elevate the efficiency of resource allocation and meet the given end-to-end latency threshold, OSCAR delivers only the highest requested quality from the origin server to an optimal group of VTFs over a multicast tree. The selected VTFs then transcode the received video segments and transmit them to the requesting VRPs in a multicast fashion. To mitigate the time complexity of the proposed MILP model, we present a simple and efficient heuristic algorithm that determines a near-optimal solution in polynomial time. Using the MiniNet emulator, we evaluate the performance of OSCAR in various scenarios. The results show that OSCAR surpasses other SVC- and AVC-based multicast and unicast approaches in terms of cost and resource utilization.
Keywords: Dynamic Adaptive Streaming over HTTP (DASH), Live Video Streaming, Software Defined Networking (SDN), Video Transcoding, Network Function Virtualization (NFV).