Scalable Platform for Innovations on Real-time Immersive Telepresence
SPIRIT (Scalable Platform for Innovations on Real-time Immersive Telepresence) is a project funded by the European Union (SPIRIT, 101070672).
Note: The first SPIRIT Open Calls (SPIRIT-OC1) are live! Apply for up to 200 thousand € in funding for a 9-month long project. For more information, refer to the SPIRIT OC1 website.
Motivation
Telepresence represents the next generation of communication applications that will significantly improve immersive experiences in both human-to-human and human-to-machine interactions by blurring the physical and virtual worlds. While this is already influencing how we work, it will have an even greater impact in the future, contributing to increased resilience to environmental disruptions, industry productivity, and energy efficiency.
Due to their complexity, cost, data compression, and bandwidth requirements, these solutions have not been scaled yet. SPIRIT, which is funded by the European Commission’s Horizon Europe programme, will research, develop, and demonstrate low-latency and scalable solutions that will ultimately bring real-time immersive telepresence into practice.
Overview of SPIRIT work methodology
Objectives
- Support a wide range of networking bandwidth with adequate compression transmission of information.
- Overcome the limitations of the current technologies handling a large number of simultaneous users.
- Support different input and output modalities as the bandwidth capacities extend.
- Through open standards, support the integration of additional services.
- Ensure security and protect user privacy as well as implement innovative identity management solutions.
- Develop, document and promote open standards-based interfaces (such as APIs).
- Provide systems to facilitate human-to-human and human-to-machine interaction.
- Demonstrate clear and efficient integration paths for take-up by European industries.
Publications
- Minh Nguyen (Fraunhofer Fokus, Germany), Shivi Vats (Alpen-Adria-Universität Klagenfurt, Austria), Xuemei Zhou (Centrum Wiskunde & Informatica and TU Delft, Netherlands), Irene Viola (Centrum Wiskunde & Informatica, Netherlands), Pablo Cesar (Centrum Wiskunde & Informatica, Netherlands), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Austria) Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt, Austria), “ComPEQ–MR: Compressed Point Cloud Dataset with Eye Tracking and Quality Assessment in Mixed Reality“, 15th ACM Multimedia Systems Conference (MMSys ’24)
- Minh Nguyen (Alpen-Adria-Universität Klagenfurt, Austria), Shivi Vats (Alpen-Adria-Universität Klagenfurt, Austria), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt, Austria), “No-Reference Quality of Experience Model for Dynamic Point Clouds in Augmented Reality“, ACM Mile High Video (MHV) 2024 [PDF]
- Minh Nguyen (Alpen-Adria-Universität Klagenfurt), Shivi Vats (Alpen-Adria-Universität Klagenfurt), Sam Van Damme (Ghent University – imec), Jeroen van der Hooft (Ghent University – imec), Maria Torres Vega (Ghent University – imec), Tim Wauters (Ghent University – imec), Filip De Turck (Ghent University – imec), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt), “Characterization of the Quality of Experience and Immersion of Point Cloud Videos in Augmented Reality through a Subjective Study“, IEEE Access, 2023 (impact factor: 3.48; quality factor: I) [PDF]
- Shivi Vats (Alpen-Adria-Universität Klagenfurt), Minh Nguyen (Alpen-Adria-Universität Klagenfurt), Sam Van Damme (Ghent University – imec), Jeroen van der Hooft (Ghent University – imec), Maria Torres Vega (Ghent University – imec), Tim Wauters (Ghent University – imec), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt), “A Platform for Subjective Quality Assessment in Mixed Reality Environments“, The 15th International Conference on Quality of Multimedia Experience (QoMEX’23)[PDF]
- Minh Nguyen (Alpen-Adria-Universität Klagenfurt), Shivi Vats (Alpen-Adria-Universität Klagenfurt), Sam Van Damme (Ghent University – imec), Jeroen van der Hooft (Ghent University – imec), Maria Torres Vega (Ghent University – imec), Tim Wauters (Ghent University – imec), Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt), “Impact of Quality and Distance on the Perception of Point Clouds in Mixed Reality“, The 15th International Conference on Quality of Multimedia Experience (QoMEX’23) (acceptance rate: 49%)[PDF]
Team members
Hermann Hellwagner is a full professor of computer science at Klagenfurt University where he leads the research group “Multimedia Communication (MMC)” in the Institute of Information Technology (ITEC). His current research areas of interest are distributed multimedia systems, multimedia communications and adaptation, QoS/QoE, Information-Centric Networking (ICN), and communication multi-UAV networks. Further information is available here.
Christian Timmerer (head of ATHENA) is a full professor of computer science at Klagenfurt University, Institute of Information Technology (ITEC). He is a Co-Founder of Bitmovin Inc. where he is the Chief Innovation Officer (CIO) and the Head of Research and Standardization. His research interests include immersive multimedia communication, streaming, adaptation, and quality of experience. Further information is available here.
Google Scholar | Twitter | LinkedIn
Minh Nguyen received his Degree of Engineer in Electronics and Telecommunications in 2018 from Hanoi University of Science and Technology in Vietnam. He received his Ph.D. from Klagenfurt University in Informatics Technology in 2023. He is currently a Postdoctoral Research in the SPIRIT project and his research interests are adaptive video streaming, immersive multimedia communication, computer vision, and QoS/QoE evaluation. Further information is available here.
Google Scholar | Twitter | LinkedIn
Shivi Vats received his Bachelors of Computer Science from IIIT Una, India in 2018 and his Masters in Game Studies and Engineering from Klagenfurt University in 2022. He is currently working as a Project Assistant with the SPIRIT project and has previously worked on the 5G Playground research project on behalf of Klagenfurt University. His reserach interests include volumetric video, virtual, augmented, and mixed reality, and low latency video streaming. Further information is available here.
