​Why the industry is in danger of data overload

Broadcasters are fighting harder than ever for views and subscribers, and that means a need to offer a large choice of premium content to remain relevant and attractive to budget-conscious consumers.

The problem with large amounts of content and higher and higher quality is that it means huge, and ever rising, amounts of data to store, process, and distribute to multiple channels, often around the globe.

All of this means a very real risk of data overload that, if not properly managed, could lead to massive problems throughout the industry, and spiralling costs.

How can broadcasters and post-production studios effectively manage their workflows to avoid data overload without skimping on content production?

The media and entertainment industry is increasingly moving towards cloud storage to enable easy access anywhere in the world.

While this has a myriad of benefits, with the industry facing ever increasing amounts of data, it’s very easy for those storage costs to spiral out of control.

The more you store, the more you pay and that has always been the case, no matter the medium.

Perhaps it is almost the ease at which you can add more storage that makes it even more challenging, like swiping your credit card on contactless without really considering the cost implications.

Storing vast amounts of data means organisations need to effectively manage their storage metrics and select the appropriate storage platform, adjusting it depending on the content and how frequently they need access, and even who needs access.

For many, this likely involves a hybrid model, with those high priority items stored in easy access storage, whether on premise or in the cloud, and other content that is less frequently needed stored in deep archive.

Egress fees are the biggest cost burden for cloud-based workflows, and the rise in data is making this even more challenging.

If media companies are continually uploading and then downloading large video and audio files, this is going to result in high egress fees with each transaction.

The key to minimising these expenses is in understanding egress costs in order to better optimise those workflows.

If you ingest content, move it to cloud storage, then download to work on some edits, upload back to cloud storage, download for QC, upload again, you are just adding more and more egress costs.

Instead, the industry needs to be looking at bringing all of those workflows into the cloud so you upload once, rather than multiple times for each piece of content.

Some cloud platforms have higher egress fees but come with other benefits that make them worthwhile.

If broadcasters and post-production companies only use these for those workflows that require specific capabilities, this will also greatly reduce costs, while utilising cost-effective solutions for less frequently accessed content.

Most post-production companies adopt a hybrid approach, rather than fully cloud-based, but for those brave enough to make the shift, it is delivering numerous benefits, including when it comes to resource management.

Being able to easily spin-up and down workstations as needed helps to prevent bottlenecks in workflows and avoids inefficient over-provisioning.

Additionally, when post-production workflows are based in the cloud, media operators can work on content simultaneously from any location and collaborate with ease.

However, without proper scheduling and resource management, organisations may end up paying for idle workstations and resources. At the same time, organisations need to be ready to cost-effectively handle fluctuations in editing workflows.

Data overload is a very real threat to broadcasters, but it’s not an insurmountable challenge. Content needs to be stored in the most cost effective and efficient way, and workflows need to be optimised so that assets can be processed efficiently. While this has always been the case, as the industry learns to cope with growing content volumes, it’s become even more important.

 Martin Riley is operations director at 7FiveFive