Immersive technology: Who’s that girl?

Madonna’s eye-catching performance at the 2019 Billboard Music Awards was, even by the Queen of Pop’s lofty standards, an attention-grabbing showpiece, with the star dancing alongside four different versions of herself.

It would have been difficult to tell which one was the real Madonna, if not for a sprinkling of VFX fairy dust that welcomed each digital Madonna onto the stage.

The photo-real illusion was created using digital avatars made by immersive studio Dimension. The company creates highly realistic virtual human avatars inside a circular volumetric stage with 106 cameras, split between 4K video and infrared.

The subject is surrounded by the cameras, which are mounted on a circular grid forming the framework of the stage.

The combination of video and infrared cameras makes it possible to capture depth information. A wireframe of the subject is first created, then a volumetric mesh is added to give the model a degree of realism. Finally, a photo-real texture is applied to the mesh to create the lifelike 3D avatar.

You can move a virtual or physical camera 360 degrees around this virtual human to show whatever angle you like, but the avatar will only perform the moves captured and can’t be reanimated.

The companies behind Dimension are immersive media firm Hammerhead, digital technology innovation centre Digital Catapult and Microsoft.

Dimension joint managing director Simon Windsor says: “The Madonna shoot all came together at the eleventh hour. She came to our volumetric studio in Wimbledon over two evenings, staying from 7pm to 2am each time, and we created four different personas for her.

“There was a lot of pre-production time working with Madonna’s choreography team and dancers to ensure what we were filming would work flawlessly with the physical camera moves on the night.

“We made sure they understood the way the tech works, the dos and don’ts of interacting with the avatars. We also offered advice on Madonna’s wardrobe and what items of clothing would resolve better than others. To make sure the holograms would look convincing and slot in seamlessly alongside the real Madonna and her dancers, we planned the routine with pre-viz and storyboards.

“Madonna was fantastic to work with,” he continues. “She’s a big advocate of using tech in live performances. It’s a world first having holograms perform on stage in this way and introduces lots of different creative possibilities around how the physical stage translates into virtual worlds. It’s a watershed moment for believable holograms.”

Jeremy Silver, chief executive of Digital Catapult, says: “Madonna’s performance at the Billboard Music Awards provided just a taste of the unprecedented levels of visualisation and storytelling made possible by immersive technologies.

“Such performances are critical moments for the UK immersive sector, demonstrating the vast potential of these technologies and the UK-based studios championing them. It is only a matter of time before we see their wider adoption within the music industry and the broader creative sector.”