The rise of machine learning

Much of what is now referred to as Artificial Intelligence (AI) and Machine Learning (ML) is, in reality, just advanced image or metadata analysis. Rather than ‘learning’ by themselves, machines need to be trained in detail to get good results and will only get better through additional training.

Nonetheless, by assisting media companies in automating repetitive manual tasks and freeing up engineers and operators to dedicate more of their time to delivering quality content, the ML capabilities of AI are already making significant contributions.

Examples of this include AI-based content recognition technologies to enhance compliance activities around content and machine-based translation for content subtitling.

Other areas of focus for AI are advertising and content recommendation, where media companies use real-time decision-making to create a better, more relevant experience for the end consumer.

“The combination of machine learning and the elimination of traditional barriers between operational and business systems is helping media companies improve efficiency through the reduction of manual processes such as schedule import and export, which also helps to reduce human error,” says Glodina Lostanlen, chief marketing officer of multiscreen video and ad management technology provider Imagine Communications.

“It’s possible that the application of AI and ML to the data we capture could help to drive decisions on device deployment and resource availability,” adds Ian Crockett, chief technology officer at automation specialist Pebble Beach Systems.

“AI has the potential to help with monitoring activity and log files across the system, with the possibility of pre-empting potential problems, rather than the user waiting for something bad to happen and then reacting to it.”

Security and user experience design specialist Nagra is working with several pay-TV service providers to improve their operations with data analytics.

Senior director, product marketing, Simon Trudelle explains that while not all projects technically qualify as “pure AI”, the general approach “takes usage data sent back through a mobile app, as well as other data sources, to determine the type and frequency of marketing and commercials actions that will drive subscription renewals or cut churn, based on a predictive ML model.”

CONTENT VERSIONING

Content versioning also provides an opportunity to deploy AI and ML tools. Entertainment supply chain association MESA estimates the annual cost of localising TV content in Europe alone at $2.5bn (£1.95bn), rising at a rate of 10% a year.

“For the amount of money you spend on a transcription house, you can use ML to deliver speech-to-text, picture localisation and an almost infinite variety of other data tasks – and you end up with richer content,” says Josh Wiggins, chief content officer of automated metadata collection, curation and search company GrayMeta.

“ML can be used to identify a logo in the background of a scene and redact, blur or replace it.”

Discovery Communications is exploring real-time speech-to-text engines from Google and others. It wants to do this at scale to cover all its content in all languages for worldwide distribution.

This includes in-picture versioning – for example, searching for text on posters or signs in an English-language broadcast and mapping over that part of the frame with a local translation.

Such methods are not yet good enough for automated BBC1 subtitling, admits BBC Worldwide vice-president of global content operation Stephen Stewart.

“However, if you have an opportunity to push content [to territories] where it’s not economically viable at present, and you can inform people about a subject they would not otherwise have seen, then – if not perfect – it is worth it. Machine learning is getting there.”

We could already be there if, as Wiggins suggests, the dominant access systems services would let the young turks (like GrayMeta, he implies) in.

“If you’re a leader in localisation and making X million dollars, are you going to allow new technology to reduce that spend?”

GrayMeta is working with AI developer DimensionalMechanics, which has algorithms analysing ‘sentiment’ to highlight inappropriate comments made in online forums associated with brands or shows.

Similar algorithms can detect adult content either in archived media or generated in real-time, which is useful for user-generated content.

“It’s one of our most popular tools,” says DimensionalMechanics chief executive and co-founder Rajeev Dutt.

The potential features and benefits of ML are growing rapidly. Aside from helping to automate specific workflows, it can make content vastly more discoverable.

AI developed by Finnish start-up Valossa Labs can tag video, providing a “superhuman ability to enrich unstructured data with descriptions”, claims chief executive and chief technology officer Mika Rautiainen.

Such identifications include people, visual and audio context, spoken topics, named entities and general themes, time-coded and at scene level.

Finnish national broadcaster YLE has tested it to automate the segmentation of magazine-style long-form content into separate stories, primed for digital distribution.

“Our technology enables content visualisations, or heat maps, that provide detailed insights into video content,” says Rautiainen.

DimensionalMechanics offers a headline optimisation tool for digital broadcasters. Users enter a headline and the AI suggests ways to optimise the text to achieve the greatest number of clicks.

If a plane crashes killing just the pilots, for example, the engine may suggest not including the number of people who died in the headline to entice people to click through to find out.

Another AI news application is being employed by Euronews for readers to self-verify fake news on social networks. It partnered with start-up Storyzy to launch All Quotes, an automated quote verifier that trawls a database of more than 15 million ‘authentic’ quotes vetted by experts and reputable news sources.

Metadata is the key to making broadcast media artifi cially intelligent. Yet using AI for media management is currently being overlooked by companies, argues Tim Child, chief content officer and co-founder of media asset management (MAM) provider Cantemo.

“Being able to integrate these technologies at management level will make a massive difference throughout the workfl ow,” he says. “For example, AI at ingest means content can be automatically analysed to provide suggestions for metadata tags and where to store that item. AI can enable objects within video and images to be automatically identified.”

Cantemo has integrated Google Video Intelligence (GVI) in its cloud-based media aggregation platform, Iconik. Ingested assets are analysed using GVI and Cantemo collects the timecode-based metadata and associates it with the asset to make it searchable.

“The service returns suggestions with a probability variable, and we provide a workflow allowing users with sufficient rights to approve or reject suggestions from GVI,” Child explains.

“Currently, this is being used to automatically identify objects, making it easy for customers to search for specific items within their assets that would otherwise be undiscoverable. This also makes it possible for users to detect specific shots and scenes within a video.”

According to MAM vendor Piksel, ML can also assist with matching to third-party metadata providers, to further improve accuracy.

Additional uses for AI and ML, notes Piksel head of solutions Kristan Bullett, include automated ID of black frames and scene-level analysis as a contribution to production.

However, AI and ML technology is very much in its infancy and no one should assume it’s going to get it right every time. “Everyone is playing in ‘weak’ AI right now,” says Jason Brahms, chief executive of AI developer Video Gorillas. “We are a long way from consciousness.”

Dutt, who designs AI tools, calls most AI “very stupid” because machines are typically trained on narrow pieces of data.

“Anything outside of that data set and the machine can’t understand it,” he says. “A lot of times, the project will fail because the training does not encompass broader data sets. You need a greater level of specialisation. An AI for Fox News will not necessarily work as accurately with an MSNBC audience – and vice versa.”

Sylvian Thevenot, managing director of multi-screen software vendor Netgem, makes a similar point about automated content recommendation.

“AI can only go so far. It won’t necessarily highlight what the user is interested in, because the algorithm generally focuses on available programming and past user views. If the source is another person, it may be more relevant.”

Netgem’s strategy is to use recommendations based on Twitter. “Users can see what programmes other Twitter users are watching and commenting on,” says Thevenot.

Other vendors are just keeping an eye on the tech, rather than diving in. “AI is at an even earlier hype stage than other industry buzzword,” says Cinegy head of product management Lewis Kirkaldie. “We’re still smarting from speech recognition overpromises nearly 10 years ago.”

He adds: “I can see people trying to pitch systems in which AI will predict when a system is going to fail, but I don’t see it being particularly helpful for traffic management or playout. For the majority of core broadcast services and processes, AI won’t be much help.”

There is also the paranoia of job losses as finance teams evaluate AI’s cost-saving options. Wiggins wants AI to be rebranded as “people are scared of that word”. He suggests human intelligence is essential to “deal with the important 10% of data that machines can’t understand”.

Bullett expects some organisations to be very cautious at the prospect of machines ‘freeing up’ staff time.

“ML/AI does not need to be about resource reduction and can be used to provide powerful tools for operators looking to make better use of data – for example, segmentation information or data to improve quality control.”

Nonetheless, companies will start using AI when it can substantially cut operational costs or help them engage better with customers. “In each case, the use of AI needs to be done as a business case, not a technology strategy,” argues Virtual AI chief marketing officer Niall Duffy.

“In many ways, TV is an ideal industry for AI/ML because so many of our activities are highly programmatic; that is, they follow quite specific processes. We like to think that this is a creative industry but in reality, once a creative idea is conceived, there are fairly strict processes that bring that concept to life.”

ML-driven packages like Lumberjack and Magisto can be used to churn through rushes and serve up story lines or clips, provided the algorithm is told what to look for.

Jukedeck offers machine-written music for TV that comes royalty free. In fact, an audience of post-producers at the HPA Tech Retreat in June couldn’t distinguish between tracks composed by human and computer, showing that the line between science and art is already blurred.