Before going further into this topic, it would be logical to know what is volumetric video (also known as VV). It is a video technique that records object, spaces the way it is – in 3D volume. It gives and opportunity to captures a three-dimensional space, such as location or performance in real-time. Basically, everything that you see around you: objects, environments, living creatures, can be filmed with their exact dimensions and transferred into web, mobile and virtual worlds. There is an easy example to understand volumetric capture even better.
Photo source: Dimension Studio
Imagine yourself filmed with at least two cameras, or more for extra realistic result. You don’t have to stay still while filming, you are performing and cameras captures it. After this process you’ll get a digital 3D model of yourself in motion, with your actual dimensions. That’s it. You’ve used volumetric capture and got your volumetric video. Then immerse this created “hologram” into any digital environment. As a user you can watch these movements from any angle by using position tracking. Seems easy, right? Well, maybe it is for professionals with high-tech devices. Actually, one of the objectives of every player in this field is to make it available for as many people and platforms as possible.
For decades everyone considered that holograms or capturing 3D real-world objects is science-fiction. We’ve seen them featured prominently in Star Wars, Blade Runner and many others.
Gif source: giphy.com
But day by day, minute by minute, these things, considered impossible, slowly becoming reality. It’s stunning how fast Volumetric Capture evolves.
What is the process of Volumetric Capture?
Professionals make volumetric video in studios with multiple synchronised cameras, green screens and incredible amount of computing power. However, now there are companies developing small and easy to use volumetric video setups so everyone with not much technical skill can use. The process don e by large studio like sets starts by capturing performances with 30 or more than 100 synchronised cameras on a calibrated green screen stage. Than they subtract the background to compute silhouettes then schedule the data for processing. The reconstruction process uses computer vision algorithms to produce a set of 3D models that are arranged as a sequence. The last thing to do is to unwrap the meshes and generate the texture atlas then compress and encode the data into single streaming ready MP4 video file or game engine like Unity.
With smaller set ups of 2 or more sensors it is easier process. It gives real time capture and real time render. However, the quality of 2 cameras is for sure lower than what you can capture in a large studio.
Volumetric capture process by Microsoft.
One of the professional fields that will benefit from volumetric videos is film-making industry. You might ask why. Volumetric movies will allow viewers not only watch through 2D screen, but be inside them, walk around zoom in/out and truly experience the action that’s going on. So it’s legitimate to call them Walkable movies. Fun fact, first movies with sound were called ‘talkies’ and now there’s a place for new term – ‘walkies’.
Gif source: giphy.com
Now think about documentaries or action movies. Wouldn’t be amazing to explore each corner of the planet from every angle by navigating in three-dimensional viewpoint? Or examine every muscle of the man during an intensive fight? Now there are couple of companies such as EF EVE™ and Depthkit offering VFX functionality for volumetric video.
Volumetric VR film ‘Vestige’, by Aaron Bradbury touched everyone’s hearts by bringing viewer into someone else’s memories.
Photo source: twitter.com
In ‘Vestige’ blackness surrounds you and as narrator Lisa guides you through her memories and emotions, you learn about the tragic death of her husband. “A lot of this was produced with volumetric capture. And to do that, I used a piece of technology called Depthkit, which basically brings together an infrared camera on a Kinect, which captures depth and a normal color camera, a Canon DSLR. Then it calibrates the two together, so it can remap the color data back onto the volumetric data.“ A. Bradbury explained the technology, that helped him to achieve this stunning result.
There’s no more limit for your curiosity. Volumetric video simply adds another dimension to storytelling and visualization.
Volumetric Capture is for everyone
After all this information you probably think that Volumetric Video is just for professionals. You’re actually a little bit wrong. As I mentioned before, the goal is to make VV consumer-friendly. To be able to use it even in your living room with the help of off-the-shelf devices.
The biggest impact to the start of opening volumetric capture for everyone was the Xbox Kinect and now the newest Azure Kinect as well as Intel Realsense. With these devices EF EVE ™ makes volumetric capture easily accessible for every user, whether he’s experienced in tech or not.
It’s not only consumer-friendly, but also they are bringing the cost of entry down for those who are looking to build their own volumetric capture studio. User just needs the app, two Azure Kinect, Kinect v2 or Intel Realsense cameras, one computer and he is ready to make volumetric videos in seconds with no studio setup. They have also introduced 4 sensor set up on 2 PC with automatic calibration giving a very easy high quality set up.
Communication is everything
So, now that you can see how capturing volumetric video is simple, it might still be tough for you to understand what for. Nowadays, when most of the conversations are via emails and texts, there is so much miscommunication. In the absence of facial expression, tone of voice, gesture we have very little to help us discern what the other person is trying to tell us. Without these clarifying cues, we frequently “fill in the blanks” with our customary worries and assumptions. This leads to arguments, hard feelings or even breakups. But there is possibility to have intimate conversations just as as real. That’s where volumetric video comes. It will take communication and interactive collaboration to a whole new level, allowing people to emotionally and intellectually connect on a deeper level with the content they’re consuming.
Photo source: vrfocus.com
Record memories with volumetric capture
If you’re still a fan of the good old face to face communication, maybe “holograms” seem deterrent. But the information below might change your mind for good.
Imagine yourself looking at the photo taken few decades ago. The woman in it is wearing such a beautiful dress. You wish to have one for yourself, but the back of the dress is unseeable, so it’s hard to recreate it. Volumetric capture makes problems like that to disappear. While it is still a controversial idea to capture “holographic” memories of your loved ones or yourself for the future generations, 8i studios, California is making a huge progress in this field. 8i studios is a company that makes “holograms”: volumetric videos that can be viewed in a VR headset or as part of an AR app.
Ashley Scott is an actress, an interior designer and a mom which is worth to remember. Ashley is one of not many people in the world, who has turned her kids into “holograms”.
Photo source: squarespace.com
Being recorded isn’t any different than being filmed (I know that for sure, because I had a volumetric video of myself made just today). What is different is what you get to experience afterwards. At the end of this process Ashley said “I think when you’re dealing with a newborn, it feels like that stage is going to last forever […]. But it really goes by so fast,” Ashley said. “So to be able to jump into that again is overwhelming.”
See full conversation with Ashley below.
Hopefully, Not in a very distant future those technologies are becoming open commercially, then digital photos and videos are in a past.
Photo source: hypevr.com
What the future holds for volumetric video?
Everything in this article is just a tip of an iceberg what volumetric capture is capable of. The experts say, that in near future AI will be inseparable from Volumetric Videos. The volumetric capture has the potential to be used in many ways. VV along with AI gives even more power to VR, AR and MR. What if holographic AI isn’t just recognising questions but is formulating responses from AI databases?
This will undeniably impact the way that we experience memories and conversations. Being able to interact with a person that maybe doesn’t exist anymore would be emotional. So far these technologies are closest to our hearts and minds.
For now incredible amount of data and the computing power that’s needed is holding volumetric video from spreading around the world and becoming fully consumer-friendly. But very soon we’ll communicate, shop, entertain, make memories, study and work with a help of volumetric videos. Volumetric capture is a new black. No need to avoid it. Let this whole new world of technologies make our lives less complicated and smooth. Also, don’t forget 5G is coming!
Want to know more about volumetric video subscribe to our Newsletter.
Gif source: giphy.com