As the demand for volumetric capture is rapidly growing, naturally there has been a huge growth of interest in Volumetric Filmmaking too. New ways to create engaging content with more realistic outcome and unexpected narrators – Volumetric filmmaking, as a hybrid of Virtual Reality (VR) gaming and cinema, has to offer a lot: it could be used as a tool in a documentary film, immersive theater, cinematic 360 virtual reality, interactive installations, choose-your-own-adventure books, walking simulator games, and in generative art. And yet – this list could continue as Volumetric Video Capture is taking first but huge steps into immersive media and could be used as an appliance for various content creation.
Volumetric Filmmakers NYC (VFNYC) describes Volumetric Filmmaking as a growing movement ‘in Immersive (AR/VR/MR) content characterized by interactive experiences created using predominantly 3D scanned imagery, such as volumetric video and photogrammetry, in conjunction with game engines to enable viewer agency within reconstructed lifelike environments.’ In order to spread awareness about this developing movement, VFNYC hosted a meet up at Vimeo Brooklyn for various developers, makers artists and creators that are passionate about art, craft, and tech of volumetric capture for VR, AR & Immersive Media. The forum was intended to be a place where everyone, who is familiar with or curious about this new and emerging technology, could join and learn about volumetric filmmaking and broaden their creative horizons.
VFNYC meet up co-hosts Kyle Kukshtel, Bill Platt, James George, Alexander Porter, Josh Apter, Scatter, Manhattan Edit Workshop and presenters Rob Ruffler, Rosalie Yu, Az Balabanian, Ben Nunez, Sebastian Marino, Evercoast, Casey Pugh.
Creating the Aeronaut VR; Photogrammetry Deep Dive; Creative Potential of 3D scanning; Scalable, Portable, and Affordable Volumetric Capture, for Everyone and Accessible Volumetric Distribution – these are the topics that were covered in this meetup. Seems a little bit complicated but yet intriguing, right?
Before the start of presentations, Kyle Kukshtel, Integration Engineer at DepthKit, Freelance Director, and Game Developer presented a brief summary of what is new in the field of volumetric filmmaking. Kyle talked about Unity Book of Dead Assets, Queerskins updates, The Volume and their ‘3D Reconstruction of History’, Magic Leap monthly live-streams, Facebook and RED announcement about new development of the new camera, Lightform Sensor Launch, Adobe AR app and ‘Soccer on you Tabletop’ by the University of Washington. All of these are really compelling topics, so just press the links and check these articles in order to stay updated.
Bill Corgan in behind scenes of his music video clip Aeronaut.
After Kyle has finished a presentation on the news, the first was one to speak was Rob Ruffler, former VP of Viacom Next, a Digital Media executive which is a Cannes Lions Grand Prix award-winning director for his work on Aeronaut and an Emmy award-winner for the Nick app. Rob explained how Bill Corgan music video clip was filmed – they used 106 cameras and it took 6 hours to get the results they wanted. Rob also expressed his hope that Volumetric Video and example of Bill Corgan would change the music video industry and that applies to old good MTV, that is focused on music video display. He also noted that they are thinking of ways how to make Volumetric Video content engaging to those who do not have VR headsets.
Az Balabian, founder of Azadux
‘Photogrammetry Deep Dive’ was the topic covered by Az Balabian, cinematographer, director of Photography, the founder of Azadux, which also directs and shoots commercials for VR companies and is the host of Research VR podcast. Az Balabian started his presentation while showing a short video with music instruments played in VR and explaining that virtual reality is an amazing place for creativity as you can build your own instruments and create music with them as the possibilities of creation in VR are limitless. ‘One-man-army’ – that is Az’s goal, and that means that one person could do the job worth 100 VFX. As for Az – one backpack is enough for him to capture amazing views with his drone and convert them into VR and we think it is a spectacular what he is doing.
‘Embrace in progress’ – a project by Rosalie Yu, that involves usage of a 3D scanning, motion capture and 3D printing.
The third presenter was Roselie Yu and she talked about ‘Creative potential of 3D scanning’. As Roselie Yu mentioned, her interest on volumetric capture has started when she heard about Scatter and then she started pushing 3D scanning and photogrammetry outside of its roots as a preservationist’s tools by showing what it looks like to approach the medium creatively. Roselie showed her projects such as ‘Skin Deep’, ‘A ritual of Habits’ and ‘Embrace in progress’. Last one project involved a 3D scanning, motion capture and 3D printing to express the intimate relationship in areas where physical contact is limited.
Roselie also showed how the body could be used as a canvas in her project ‘Skin Deep’ – after people were captured by volumetric video, the texture was changed. Used holograms as pencils’ drawings, creating watercolor animations and even letting spectators to color volumetric bodies and see it live in Indiebo festival. The new and mesmerizing way to emerge Volumetric filmmaking and 3D capture into spectacular art!
Evercoast and their live-stream volumetric capture.
‘Volumetric for everyone’ – that is the statement of Evercoast. Two presenters, Ben Nunez, EO of Evercoast, a computer vision and 3D sensing company providing a software platform for the creation and distribution of volumetric video and holographic content, and Sebastian Marino, a 15 year veteran of the film industry, having worked on some of the largest franchises in history from Star Wars to Avatar, talked about Evercoast software, which ensures complete camera control with automated calibration. Evercoast uses RealSense cameras, usually 10 cameras set up, but it can vary from 2 to 20+. Presenters stated, that working with low-cost sensors is a hard task, whatsoever, as they wanted to make their product portable and accessible, they had to deal with this challenge and we can notice that they are doing a really great job, as we got chance to see how they filmed in a bad studio lighting, with a black background and even the the outfit was all black. Even though Evercoast had to deal with all of this ordeals, Evercoast managed to show the capabilities of their software and we have to say – it looks amazing!
Casey Pugh showing Vimeo live volumetric test.
The meet up was closed by a presentation of Casey Pugh, Head of Creator Labs at Vimeo, and he talked about ‘Volumetric Accessibility’ – he described Vimeo’s efforts, exploration and creator tools for Volumetric Filmmaking. One of main Casey’s goal is to make Volumetric Video Filmmaking more accessible, and that means mobile first. Casey presented a Vimeo live volumetric test that could be viewed by anyone – and a speaker reminded, that Vimeo is a developer-friendly platform, which is focusing on making tools as an open source.
It is hard to expect, what breathtaking development’s news of Volumetric capture will be presented as this is a rapidly improving area, so do not miss out an opportunity to stay updated and inspired.