On-set VFX and virtual production system unveiled

Real-time virtual production specialist On Set Facilities (OSF) has launched an on-set VFX and virtual production system called Realtime Pipeline, which combines hardware, software and ‘proprietary code’.

The end-to-end real-time virtual production system was developed in partnership with Brainstorm, Octane Render, Black Magic, Mo-Sys and Unreal Engine. It’s available as a product that can be installed at facilities, production companies and studios, with OSF providing system design, installation and a training service.

The company has built a Realtime Pipeline studio in North Wales to demonstrate the technology and is installing the setup at a number of studios worldwide.

OSF has created a short film called Wake Up 2 (watch it below) to showcase the capabilities of Realtime Pipeline in providing real-time compositing in Unreal Engine. It was directed by the company’s Asa Bailey.

The project was produced by MR Factory on a 500m green screen stage in Madrid. The film was live-composited on-set, giving the director, actors and crew the ability to work together with immediate feedback.

There were big screen projectors on-set, giving everyone a large-scale view of the real-time production. Both the cameras and the virtual production system were recorded in sync, locked by timecode.

Some of the 3D animation elements in the film were composited in later in post-production, but animations and particle effects will also soon be added to the Realtime Pipeline on-set system too.

Bailey said: “The key is to give your teams enough time to pre-produce all the assets you’ll need on set, but if you need to change anything post shooting, it’s ok as you can recreate all the Unreal backgrounds in post using the systems saved camera FBX data.”

“The ability to shoot VFX and composite in real-time is a game changer for me as a director,” he adds. “I’m really looking forward to sharing these techniques with other directors and producers. Better on-set storytelling and better creative decisions are just two immediate benefits. Shooting in realtime also gives crew and talent a more tangible world to perform in.”