Comment: How AI is enhancing localisation

Huge Netflix productions such as Money Heist and Lupin are bringing dubbed content to new audiences with high viewing figures, becoming some of the most watched content on the platform.

How machine learning and speech synthesis are shaping the future of dubbing

According to Cisco, by next year, online videos will make up more than 82% of all consumer internet traffic — this is 15 times higher than in 2017. This means that more companies are turning to technology and automation methods to reach more people at scale, but haven’t quite figured out how this becomes a long-term part of their content strategy.

When it comes to how automated localisation works, Papercup has simplified this process so customers are able to receive a dubbing service for a fraction of the usual price.

Our proposition pairs Automatic Speech Recognition, machine translation and our patented synthetic speech, which is developed through machine learning, to automate a large part of the dubbing process.

In patenting the synthetic speech technology, customers are able to receive a one-of-a-kind service that makes their content available in a number of languages in as seamless a way as possible.

Nothing will replace local content being created by native speakers in local teams.

The pyramid of localisation starts with subtitles (as the most accessible form of localisation), then moves to human dubbed content, and ends with local content created within the market. However now, thanks to automated dubbing, there is a solution between subtitles and human dubbed content.

We have created the category of AI-powered automated dubbing where most of the heavy lifting is done through synthetic speech and machine translation. In order to ensure optimum accuracy, we deploy a human-in-the-loop system, which helps with ensuring the speech provided is 100% localised, correctly paraphrased, and includes all glossary, brand and technical terms supplied by the creator.

We don’t anticipate machine translation to be able to offer 100% accuracy in the immediate future, hence why it’s incredibly important for us to include the human-in-the-loop system, so customers are guaranteed accuracy.

With our patented speech synthesis algorithm, and a combination of this technology with machine translation and ASR, we are able to turn around projects in days instead of weeks.

Working with Sky

When we partnered with Sky News, pre-existing video content from its UK newsroom had achieved strong reach across a number of platforms. While this was a vital goal for the company, it was now looking to expand its audience. Its existing content was limited to English-speaking audiences despite being of a global interest, meaning that the obvious next step was localising this content for other regions.

Sky identified its Spanish-speaking audience as the one with the most growth potential and partnered with Papercup to use our technology to repurpose its existing global content for its new audience in Spanish.

Papercup has been able to dub this important content into Spanish using synthetic voices for a LATAM audience, and in doing so has seen award-winning content go viral, particularly in regions that are not necessarily able to access trustworthy news sources. Since then, Papercup has been working with Sky News to ensure its content is quickly and accurately translated for its Spanish speaking audience, bringing the award-winning content to new territories and untapped audiences.

 Amir Jirbandey is growth lead at Papercup