Boxing clever

Sky’s VR team spent a morning with professional boxer Anthony Joshua in London’s mixed reality capture studio Dimension, recreating 35 of his signature moves under the gaze of the facility’s 106- camera rig set-up. Every move he made was recorded from every angle.

The broadcaster is now busy turning the mass of images and data from the shoot into highly detailed, rendered virtual versions of the famed boxer. Using game engines, the holograms it creates can be inserted into VR films where the viewer feels completely immersed in the action.

The type of filming enabled by Dimension is known as volumetric capture, which is a fancy way of saying it has the appearance of existing in three dimensions.

Prior to working with Joshua in the volumetric studio, Sky had already captured every one of his fights since he became World Champion in 360. The contrast between those relatively basic 360 films and the quality of result from the mixed reality volumetric shoot is truly striking, says Neil Graham, head of Sky VR Studios.

“We believe volumetric capture will be a really big thing for VR,” he said. “Compared with standard 360 capture, it’s a much more compelling experience. You feel like you’re there with the boxer in the ring. It’s so lifelike and doesn’t feel like CG.”

“It was a Eureka moment when we saw the quality you can get out of the camera rig,” continues Graham. “The detail captured makes it look so much more believable. It’s also more flexible and affordable than shooting for real, and enables greater creativity – you can create camera paths and dynamic scenes that would be impossible to capture conventionally.

“By doing volumetric capture with him, viewers get that visceral boxing experience of being face-to-face in the ring with Anthony Joshua,” he adds.

“It allows them to get a sense of the skill and technique involved, and what it feels like to be him. You really benefit from that close proximity and intimacy.”

KEEPING THE OPTIONS OPEN

Sky’s plans with the content is to create a library of assets that can be delivered as VR, AR or even integrated into traditional TV.

“The potential deliverables are very varied, but, to begin with, we’re thinking of both longer-form and shorter VR clips,” says Graham. “Some of it could potentially be used as a ‘how to’ guide to his most important boxing moves, where you learn and copy his actions. We’re looking at capturing the photo-geometry of a boxing ring so we can place him into the ring.

“We’re also planning on integrating clips into fight footage that we’ve already captured for a forthcoming VR doc following Joshua’s career over the past 12 months.” Another use for the material is in AR, where you could have Joshua look as though he’s with you boxing in your living room.

“It’s all about enhancing the sports experience,” says Graham. He says what Sky has created so far is “shaping up well”.

The key driver for spending time with Joshua in a volumetric studio is to create assets for immersive platforms, but Graham says Sky is also keen to investigate its options in using some of the photo-real content in conventional, non-immersive TV productions.

He says: “Our broadcast team and our boxing production team have been taking a look at the assets, working out the best way they could be used in traditional broadcast.

“It might be some of the assets will be used in a conventional broadcast in time for the next fight, or it might be they won’t be ready until the one after. What we’re doing is so new, there’s been a lot of collective head-scratching about the next steps.”

PROTOTYPING AND EXPERIMENTING

The team is now creating renders that Sky can work with, turning theoretical possibilities into practical productions.

“We are still prototyping and experimenting with the results, but we’ve got a render that’s been put together after a lot of work on the initial models and texturing,” explains Graham. “We’re spending a lot of time getting the look absolutely right.”

He says Sky is trying out different versions of the models, working on the sizing, positioning and interactions with the viewer. Putting the boxer into that environment is the main work that’s been done so far, as well as the initial steps on the interactive elements of the VR. As well as all this, Sky and Dimension have been working together on crafting different aspects of the lighting.

“Getting the lighting right is very important as it’s one of the things that makes the most difference,” says Graham. “With reactive lighting, the models look believably like humans, so our design team is working on all that at the moment.

”As there is so much detail in the images we captured of Joshua, it’s important to get the lighting exactly right. We’re getting all the movements right first and building the lighting around that.”

It’s still early days in the development of the different projects Sky is looking at spinning out from the Dimension shoot, but Graham says he’s been impressed with the quality of results so far.

“You have a physical reaction to it when you see what we’ve done in VR already. You do reach out and try to touch the model; it’s completely believable.”

Dimension is one of only three state-of-the- art volumetric and 3D capture studios in the world, which has been built around Microsoft Mixed Reality Capture technology.

The studio is able to capture the shape and form of moving images to heighten the realism of virtual humans and environments.

Human performance is captured photographically in the studio as volumetric or 3D video, creating “believable and emotional” results, says Dimension.

The studio has 106 RAW capture cameras, 53 of which are RGB cameras and 53 are IR cameras (to capture the depth). It also records immersive sound through eight shotgun mics.