What's the score with AI music

Either as an aid to composition or as an automated soundtrack generator, AI is already a force to be reckoned with in the music industry. In January, French songwriter Benoît Carré (stage name Skygge) created his album Hello World using Sony’s AI Flow Machines.

In June, experimental film Zone Out was scripted, directed and edited entirely by an AI called Benjamin, with its score generated by algorithms.

Facebook, Spotify and Google are directing large-scale projects at giving artists machine-learned songwriting recommendations or enabling non-musicians to compose customised instrumental tracks at the press of a button.

If Mozart could apply mathematical formula to perfect harmony in classical music, then surely a computer can be programmed to create something at least proximate.

‘Functional music’ is considered the prime market for AI-made compositions, for everything from ads for TV and radio to corporate presentations and social media. “Most investment in [music] AI is focused on the functional space,” says Stephen Phillips, chief executive of Brisbane-based AI music company Popgun.

“Demand is driven by customers who want the speed of creating music on-demand, the ownership of all rights and the convenience of being able to control the track in a much tighter way.”

“In any application where the music is mechanical – such as playlists for a shopping centre – AI will dominate,” predicts Sebastian Jaeger, chief executive and founder of Brighton-based ‘adaptive music library’ Filmstro. “That’s not a comment on the quality of the music, simply of its function.”

The trend is set to quickly extend beyond this, to embrace production music for TV, according to Universal Music vice-president of global digital business James Healy.

At Music’s Smart Future, an event run by record industry trade body BPI, he warned: “If you’re writing library music, corporate background music, [AI] is going to start eating your lunch sooner rather than later.”

It’s possible for AI to create the entire music score for a movie, as demonstrated by the producers of Zone Out, which employed Soho-based AI composition company Jukedeck to create the music for the short film.

Bloomberg also used Jukedeck to score eight-part digital series Art + Technology, which was released in May.

AI ATTRACTIONS

The main attractions of AI music are a combination of being able to get something written almost instantaneously to match the brief and precise length required, and the price – global royalty-free licences cost as little as 80p and full copyright costs from £150.

“Producers are able to customise the track to an exact length far quicker than searching stock music for a track that still needs an edit to fit,” explains Jukedeck chief executive Ed Newton-Rex.

“Users can pick mood, style, tempo, duration and when the climax should peak. The system generates a score in 20 seconds and it is delivered as an mp3. Plus, they know the music they create has not been used anywhere else.”

There’s no threat yet to professional custom compositions at the high end of drama and features but, as elsewhere in broadcast, AI’s early potential is seen as reducing the cost and time required to make smarter decisions or meet tight budgets.

“At any point with our technology, users can bring up multiple versions, create light and shade within the track, and immediately play it back”
Sebastian Jaeger, Filmstro

Numerous start-ups are emerging with variations on the customisable music theme. Filmstro’s app creates soundtracks by allowing users to control the momentum, depth and power of royalty-free instrumentals using browser-based ‘sliders’.

“We have taken the major building blocks of music and made them keyframed [visualised graphically], which is the working process that any film-maker is familiar with,” says Jaeger.

“Static library music comes unstuck in editing. There is no real shortcut to finding the precise clip to underscore a scene. At any point with our technology, users can bring up multiple versions, create light and shade within the track, and immediately play it back.”

Melodrive in Berlin is focused on adaptive music for interactive media like games and VR/AR experiences, a market it values at $30bn (£23bn). Adaptive music changes dynamically based on user input and/or to the emotional setting of a media experience.

For example, if the health of a character in a game decreases, the music may change from calm to tense to adapt to what’s happening in the game.

Other start-ups like Amadeus Code, Humtap and Splice act as a creative foil for songwriters and producers. By playing a few notes, Popgun’s AI will guess what might come next, resulting in a human/AI duet.

“If you wanted to train a neural network to create without any human intervention, you would first try to encode the laws of music so that the AI understands scale and keys, timbre and rhythm, and from that encode an algorithm to generate music,” says Popgun’s Phillips.

“The promise is that it might create music that we’ve never heard before.” However, no one in this field suggests AI is anywhere close to replacing humans. “The professional musicians we meet are able to discern encoded music,” says Phillips

. “You can train an AI on a catalogue of 500,000 songs from which you could derive something like a cover-band version, but it would struggle to create something novel.”

“AI-created music occasionally captures the essence of something good, but it seems to miss the point on some human, emotional or production level”
Nick Vince, Platinum Music

Instead, music-based AIs should be thought of as a tool, or perhaps an instrument. “The vitality of art has always been driven by technology, but there’s a snobbish attitude when something new comes along,” says Phillips, referring to the introduction of drum machines in the 1970s. “A lot of emotion and memory are tied up with how we write and receive music, none of which is present in AI,” says Newton-Rex. “People want to buy into an artist’s backstory.

At the same time, artists can make mistakes and computers can randomise, which often leads to invention.”

This calls to mind Eric Morecambe’s famous quip: “I’m playing all the right notes, but not necessarily in the right order”; or Miles Davis’ aphorism: “There are no wrong notes in jazz, only notes in the wrong places.”

If music is subjective, in the ear of the beholder, then in the constant stream of audio churned out by an AI there is likely to be some with more universal appeal.

“Certain pop producers today are better at picking the best song from hundreds submitted,” says Phillips. “In future, a curation music supervisor will be empowered with much more choice over music created by AI, whether for chart release or a TV soundtrack. The more music you create, the more choice we have in finding our individual expression.”

LACKING EMOTION

Platinum Music director Nick Vince admits: “AI-created music occasionally captures the essence of something good, but it seems to miss the point on some human, emotional or production level. There are no clearly defined improvement vectors that define what makes a musical score great, only subtle patterns within the data.”

He says an algorithm would need to be an expert in ‘listening’ and identifying relevant areas of influence within datasets, music theory, writing, arranging, emulating live instruments perfectly (including any instrument or human voice), producing, mastering and even viewing video.

“These skills would then need to come together cohesively,” adds Vince. “In our view, we’re not anywhere near that yet, but the door is open and the AI world can, and is, moving very quickly.”

If the technology gets to the stage where genuine musical artificial intelligence is actually possible – in that it is consistently indistinguishable from the best music created by humans – artificial general intelligence itself (an algorithm that can adapt to any task and improve exponentially) may not be that far away.

In that instance, the fate of production music would be a small concern compared with the wider economic and societal implications.

Outside of composition, music companies are turning to AI to enhance other aspects of their services. Production library Platinum Music is exploring AI for automating metadata tagging and to offer “intelligent music suggestions” to clients.

In March, Warner Music acquired Canadian start-up Sodatone, which uses algorithms to improve the A&R process.

Audio Network is researching AI “for all areas of business”, including honing client search. “We see AI as another tool to augment creativity,” says director of channel strategy Nathan Leong.

“There are definitely some interesting technologies where you can feed in tunes you’ve created and play with melody or tone. But we can’t ever imagine a scenario where an algorithm would replace a composer.”

He stresses: “It’s an ethical point of principle for us. We’d want to have composer creativity at the core of what we do.”