Post-production uncovered: DI4D on Double (short)

Double is a CG-animated short produced in-house by the facial capture company DI4D to demonstrate its Pure4D 2.0 technology, creating its own CG-animation featuring Neil Newbon without using a facial rig. 

Performance-driven digital doubles

Double is entirely performance-driven, meaning the digital double of Neil Newbon is an exact reproduction of his performance. This is celebrated in the film which highlights the significant contribution made by actors in the video game, film and animation industries.

Double was made possible with DI4D’s Pure4D 2.0 pipeline, an innovative approach to facial animation that reproduces an actor’s performance by combining performance data from a HMC (head-mounted camera) with high-fidelity data from a seated 4D capture system, delivering quality far beyond the capabilities of traditional pipelines.  

3D scan of the actor

First, DI4D scanned Neil Newbon at Ten24’s 3D photogrammetry capture stage. The capture stage is comprised of 170 cameras and an integrated 40 camera head capture system. Using this stage, DI4D obtained a scan of Newbon’s 3D shape, providing the team with accurate maps of his appearance, such as the details of his skin. 

DI4D then wrapped the scan with a mesh topology for animation purposes. 

Capturing the actor in the PRO system

Next, Newbon’s face was captured inside the DI4D PRO system, which is comprised of nine synchronised 12-megapixel machine vision cameras. It uses standard video lighting to eliminate the need for markers and can capture the highest level of fidelity possible.  

Inside the DI4D PRO, Newbon performed phonetic pangrams and dynamic facial expressions to provide data on the full range of his facial movements. 

Performance Capture in Action

Next came the final performance. Using the DI4D HMC at PitStop Productions’ studio in Croydon, the DI4D team captured Newbon’s facial performance simultaneously with body mocap and audio - a technique called performance capture. 

Like the PRO system, DI4D’s HMC technology is markerless, helping to reduce time on set while improving accuracy. The markerless technology is designed to be non-intrusive and provides the actor and director with greater freedom.  

All of the video captured was then reviewed to select the best possible performance for the final animation.

Tracking the 4D data

After the capture process was completed, DI4D applied the mesh topology from the 3D scan to the PRO and HMC Data. The maps are used to ensure correspondence between the 3D scan and the 4D data from both the HMC and PRO system.

Multiple shapes and pangrams are then tracked and analysed to form an accurate understanding of Newbon’s entire facial movement. Similarly, for the HMC footage, DI4D then tracked the facial animation for all of the selected performances.

DI4D’s machine learning technology speeds up this process considerably, continually learning Newbon’s facial expressions over time and reducing any subjectivity in the manual clean-up.  

Finally, DI4D combined the HMC performance with the 4D data to provide a faithful reproduction of Newbon’s performance. You can watch the full film below.