DAZN launches AI-enabled baseball highlights

DAZN has launched an AI-enabled highlights feature for its coverage of the Nippon Professional Baseball (NPB).

Inspired by its Key Moments feature for football, DAZN’s NPB highlights feature uses real-time match data from Stats Perform, Amazon Rekognition Custom Labels machine learning (ML), and video analysis technology from Amazon Web Services (AWS).

To view the highlights, fans choose either a recorded or in-progress NPB game and use a scrub bar to navigate to highlight markers to watch important plays, such as home runs, scoring plays, and more.

DAZN worked closely with its colleagues in Japan to build an application to support this functionality. It first determined which plays the application would define as highlights, using available Stats Perform data. The team then had to accurately sync this data with its live game streams and found ML the best way to do so.

The team needed a way to automatically detect the start of a game so the highlights would line up accurately in the video timeline. They explored a range of options and chose to use the first appearance of the score graphic (including the number of balls, strikes and outs) – which coincides with the start of the first pitch – as the starting marker.

With no off-the-shelf tool available to identify this graphic, they built their own model using Amazon Rekognition Custom Labels, a computer vision service that allows developers to build custom models to classify and identify objects in images.

The team used sample images from previous NPB video feeds with the score graphic, and a similar set without the score graphic, to train the AI.

They labeled the images with the score graphic using Amazon Rekognition Custom Labels, and then trained the application to detect the initial appearance of this graphic. From there, the team provided the system with a list of all the key moments and times they occur, and it synchronised them with the video based on the time of the game.

Fred Clarke, head of product, sports data, DAZN, said: “Football fans love ‘Key Moments,’ so we knew if we could translate that feature to baseball, it could take the NPB fan experience to a whole new level. Due to the sheer frequency of NPB games and duration of the season, we identified early on that automating the synchronisation would be crucial, which is where machine learning came in.

“We quickly uncovered that if we could rely on the assumption that this graphic’s appearance as a proxy for the start of the game, we could use it as the benchmark for our timeline. Our only alternative to Custom Labels would have been to have a team member manually enter the appearance of the graphic, which would be less accurate and challenging given the volume of games each year. The results are impressive, and the custom model we built using Amazon Rekognition has provided a level of accuracy we never thought possible.

“NPB fans have responded with extreme enthusiasm. Matches run quite long, so unless you’re an avid fan, tuning into an entire game isn’t a reality, so now, more casual fans can even check in to see highlights, which wasn’t previously possible.”