Ahh Screw it, Let’s Use Depth Sensors for Volumetric Capture


VRLA 2018

Ahh Screw it, Let’s Use Depth Sensors and VR/AR Equipment in Production” – this is the title of the speech that Jasper Brekelmans gave in VRLA 2018. Making his talk more relatable, Jasper focused his speech on the real world examples. That included using low-cost hardware in 3D animation, visual effects, Virtual Reality, game production, and Volumetric Capture. Not only Jasper is talented in giving fascinating speeches, but he also has an amazing experience in Motion capture. More to add – Jasper leads Volumetric Capture company by himself. Intrigued? Read more!

From Motion Capture to Volumetric Video

After 15+ years of having expertise in being in high-end motion capture and working as a 3D character rigger, mocap operator, supervisor, and tools developer, in 2012 Jasper Brekelmans founded a Volumetric Capture company, that goes behind the name ‘Brekel’. In late 2010, Kinect was launched and afterward – it was hacked. Jasper got interested and wanted to see how Kinect could be compared to high-end tools. That involved him in coding just out of pure interest Even though the skeleton tracking quality was lower than Jasper’s high-end systems, he did not stop investigating this sphere. Moreover, results were amazing –  coding Kinect opened up other doors due to its live 3D point-cloud capabilities and tracking without the need for special suits or calibration. The realization of free experimental tools influenced Jasper to focus on developing fully fledged tools. Certainly, these ultimately became the Brekel’s toolset.

Brekel’s products range from apps for motion capture to Volumetric Video recording tools. All of them use consumer depth sensors like Kinect, Orbbec Astra, Intel RealSense and Leap Motion in single as well as multi-sensor configurations. And if Jasper’s clients do not know what is a Volumetric Capture, Brekelmans believes that the best way of explaining things is to simply show and let customers experience it by themselves. Jasper emphasizes, that, preferably, observation of Volumetric Video should be done with a proper AR headset like HoloLens, Meta2, Magic Leap or a high-end VR headset with proper positional tracking.

Brekel's PointCloud Beta with Intel RealSenseSDK D415/D435 support for Volumetric Capture
First Brekel’s PointCloud v2 beta with Intel RealSenseSDK D415/D435 support.

Volumetric Capture – exciting technology

Jasper explains that the fact, that both playback and recording is becoming more and more accessible for consumer level devices usage, makes him really excited about Volumetric Video technology. In a more general sense, Volumetric data can help computers to understand environments and actions in an easier way. This is one of the fundamentals which is important for the upcoming Spatial Computing era to happen.

“[…] once we live in a future where we all have eye-wear that seamlessly and continuously allows us to mix virtual photons with real ones, we can use volumetric video to be present in different places and times and still communicate as if we were actually there,” that is how Jasper Brekelmans explains what makes him excited about this new technology.

Brekel's Body v3 Beta testers with UE4 live streaming for Volumetric Capture
Brekel’s Body v3 Beta testers with the long-awaited UE4 live streaming.

Use cases of Volumetric Video?

As there are many possible ways how to apply Volumetric Capture to daily life, Brekel highlights these 5 examples:

  1. Telepresence. It will allow for loved ones to sit next to each other on the couch. Despite the fact, that they are on the other side of the world!
  2. Instructional and educational, which could include pre-recorded virtual Volumetric humans, while explaining subjects;
  3. Sports. Watch your favorite sports game with Volumetric replays and zoom-ins on your coffee table – how awesome is that?
  4. Gaming development – holographic characters in VR/AR worlds;
  5. Tool for media production. Volumetric Video gives the possibility to see 3D models as 3D volumes instead of using a 2D screen.

Photo resource: Brekel’s personal Twitter account.


Back to top button