Allowing someone equipped with a 5G smartphone to record himself with a camera and appear to his interlocutors in high-definition 3D hologram is how Matsuko wants to take advantage of 5G networks and Edge Computing servers deployed by operators. This unique solution would allow each individual to go beyond the current limits of 2D video communication and move towards greater proximity and immersion in 3D while limiting mobility. Yet, to make this innovation a smooth and sustainable reality, the startup must be able to offer a quality of service equal to the 3D dream it aims to achieve!
Matsuko Has Tested 5G!
Line of Business
Case Study
Startup’s need
Benefits of 5G
High bandwidth
Low latency
Matsuko had been experimenting with its technology in the Orange 5G Lab and the Edge Computing Lab in Châtillon on several occasions in 2022.
The hologram: a future nearer than ever.
Communicating through holograms is hardly a new idea. All the movies that have been announcing this technology for several decades as a new means of immersive communication announced in a not-so-distant future finally testify. Thanks to Matsuko, we have never been that close to the finish line. To define the main lines of its innovation, the startup began with the following statement: the holographic communication services offered until now are too heavy to set up – especially to realize the 3D capture of the interlocutor. It requires a studio equipped with several cameras, a green background, etc. According to Matsuko, « generalizing the use of holograms by the general public requires breaking this constraining model by offering a simplified filming made only with a camera of a 5G smartphone, » explains Fabien Le Clech, Marketing Manager of the Orange 5G Lab. The startup’s idea is to combine the real and virtual worlds through a mobile connection « by using a smartphone’s camera to capture a 2D video, then transforming these images into 3D holograms in an Edge Computing server, and finally broadcasting these holograms in an AR/VR/MR (augmented, virtual and mixed realities) environment, through augmented reality glasses, a virtual reality headset, a smartphone or a tablet supporting augmented reality. » It’s a viable solution today thanks to the advantages of 5G (high connection speed, low latency, high bandwidth) that can guarantee the broadcasting of ultra-realistic 3D images.
From technical support to testing with the Orange 5G Lab
Aware of the potential that such a communication tool represents for private discussions, professional meetings, or live events in search of ever more realism and immersion, the Orange 5G Lab and the Edge Computing Lab in Chatillon organized a series of experiments with the startup. Our private 5G infrastructure, connected to a Google Anthos Edge Computing server, was ideally suited to testing this type of innovation, » says Fabien Le Clech. Embedding the Matsuko application on our Edge Computing server was the most complex phase. However, this was a crucial step since the application hosted and executed on the Edge server reconstructs the 3D hologram from a 2D video stream sent through 5G and broadcasts it to one or more interlocutors equipped with AR glasses. Fortunately, we could count on the expertise of Orange teams on the 5G and Edge network part! » For now, the concept requires an iPhone because of the requirements related to the depth of field, which are less developed on Android phones. « So we used a 5G iPhone on which Matsuko could install its application to ensure the capture. To test the service, a person places himself in front of the iPhone front camera to generate his avatar. However, we chose to stabilize the iPhone 5G on a tripod with good lighting to ensure good capture conditions. We then performed reception tests to visualize and converse with the 3D avatar of their interlocutor with another « recipient » person equipped with Nreal Augmented Reality glasses connected in 5G. They are very light – like a pair of sunglasses – and developed by a Chinese supplier who is very advanced in augmented reality. They are part of the 5G Lab equipment that we make available to the startups we support, working on AR ».
5G is a guarantee for real-time – even for a hologram.
Currently, the challenge for Matsuko is to offer real-time holographic communication. This is why the Orange 5G Lab team has performed multiple tests to capture and broadcast the 3D avatar with different resolutions. The 5G Lab performed tests with holograms of lower quality – requiring less throughput – and others of better quality, requiring streaming the 3D avatar with a more consequent flow. Numerous scenarios and tests have allowed Matsuko to measure the benefits of 5G on its holographic communication system. « The tests in the Orange 5G Lab put the capabilities and latency of 5G to the test, explains the 5G Lab Marketing Manager. And the results were very conclusive on the 5G transmission part! The network-related issues that once hindered the smoothness and immersion of holographic experiences have been resolved. But there is still a need to improve the quality of the 3D hologram when broadcast in real-time. »
A comprehensive experiment to move from innovation to a soon-to-be-available service on the market
« During the tests, we visualized and recorded the holograms with the augmented reality glasses. We provided these recordings to Matsuko so they could analyze the hologram’s rendering, texture, and fluidity. In this way, they were able to see that in real-time broadcasting, the quality of the hologram was still very perfectible – not well defined enough to launch a commercial service. » Indeed, while the latency and bandwidth offered by 5G are sufficient to ensure real-time communication, the experiment highlighted the fact « that some progress is still needed to improve the quality and performance of Matsuko’s hologram reconstruction algorithm (which is ran on an Edge Computing server). » New work is already underway on the startup’s side as well as the declination of the app for Android to ensure capture.
“This technology has great potential for virtual meeting apps such as Teams or Zoom! During a meeting, participants at a distance from each other could be projected into the room with their 3D holograms. It would greatly improve the immersion of virtual exchanges between participants thanks to more realistic interactions and proximity between people. However, to launch such a service, Matsuko still needs to improve its technology to reach a sufficient hologram quality.”
To go further
Industry: 5G boosts the performance of the industrial sector