The future has only just begun

It was a very special "game show" that took place in the packed Voltahalle in Basel last March. As part of the Interfinity festival, ten composition commissions were awarded, five to people and five to the ChatGPT programme. Stylistic copies were requested, five-minute works à la Bach, Chopin, Brahms, Bartók and Messiaen. They were performed by musicians who themselves did not know which piece had been created and how. And after each pair of works, the audience was asked to decide which of the two compositions had been written by a person.

The Bach imitations for violin and harpsichord were the first. And yes, the movement structures in the first piece, the rhythms, the way in which harmonic friction arose between sustained notes: They really did sound like Bach. But here and there you thought you could hear atypical chromatic turns and melodic breaks. Had a machine made a mistake - or was a human composer simply not quite as gifted as Bach himself?

The question could be answered after the first few bars of the counterpart: no one would ever think of suggesting such sparkling arpeggios, such soft-edged harmonies as a Bach style. While the details of the first work were irritating, the "aura" of the second was not right, it seemed too artificial, too superficial, too pleasing: pseudo-baroque for the lift. Only a machine could produce something like this, was the judgement of a large majority of the audience. And rightly so, as it turned out.

Mahler, unfinished

Ali Nikrang is not surprised. Music is a highly complex matter, says the Austrian-based musician and AI researcher in the video interview. He once studied piano and composition and was interested in formal patterns early on, for example in Bach's canons; this interest led him to study computer science. Today, he conducts research at the Ars Electronica Futurelab in Linz and holds a professorship for AI and Musical Creation at the University of Music and Theatre in Munich.

In the 1960s, in the pioneering days of artificial intelligence, he explains, music was chosen as one of the first fields of research "because it was underestimated". Scores consist of only a few concrete elements: a few pitches, a few durations, a few dynamic signs and indications of articulation. Texts seemed more complicated, and digital image design even more so. So the plan was to solve the problem of "composing" for a moment and then move on to more difficult questions.

The specialists at the time soon realised that it wasn't quite that simple after all. "In order to write a meaningful work, you have to know a lot about the human perception of music," says Ali Nikrang. "You have to understand how tonality works, what kind of voice leading makes sense, how a piece can develop and how it ends." This is why the first attempts to create music with the help of artificial intelligence failed so thoroughly: "It is not nearly enough to programme music theory concepts."

The first artificial intelligences tried to do just that. They worked on the basis of rules, imitating music theory laws and instructions. The result sounded like all sorts of things, but not like music. Since 2012, the approach has changed: New artificial intelligences are not fed with rules, but with vast numbers of examples. They "learn" from these in a way that even those who programmed them can no longer comprehend - and for some years now, at least, they have increasingly been producing something that can be recognised as music.

Ali Nikrang created one of the first scores in the Ars Electronica Futurelab under the title "Mahler-Unfinished", which caused a furore in the classical music world. in 2019, he worked on the sketches for Mahler's Symphony No. 10 using the MuseNet programme, which provided the musical structures that he then orchestrated by hand. The aim was not to "complete" a fragmentary work, says Nikrang, "I simply wanted to show what is possible: namely that an AI can compose music in the late Romantic style that could also have been written by a human".

The result was premièred by the Bruckner Orchestra Linz under the direction of Markus Poschner. He did not find it truly original, but "quite successful", as he told the "Standard" at the time. He was even more concerned about the process that had led to this result: on the one hand, he was amazed by the technical perfection, but on the other, he was also irritated, to the point of wondering "whether, as a musician, you're allowed to think that's good at all".

The problem of copyright

Is it allowed? The question is currently becoming more controversial almost every week. Because a lot of what is being done is actually surprisingly good. With the Suno programme, for example, entire albums can be produced without anyone having to write, sing or play a song - and in styles far removed from the mainstream. The Udio programme presented in April can do even more. Ali Nikrang sees it as a "game changer": "It's much better than anything that's come before. Classical works and sounds are also imitated extremely well, although this is certainly only a side effect - economically speaking, it's not interesting to invent a Bach aria." In contrast, such a programme is interesting in the field of functional music, such as film music: Nikrang is convinced that entire soundtracks will soon be created by machine.

It is clear that such a development raises fundamental questions. Emotional ones, for example: can music that is produced by machine be touching? Yes, says Ali Nikrang, "although it is also a strange feeling for me when a programme suggests something that appeals to me emotionally". Of course, it is possible to find AI-generated music cold on principle, as long as you know how it was created. "But if you don't know that, I'm convinced that the reaction is the same as with other music: it either triggers something or it doesn't."

A second question concerns copyright: can a machine utilise all the examples it is trained with? In the area of text production, the New York Times has filed a lawsuit against the companies OpenAI and Microsoft for using millions of articles from the newspaper to train ChatGPT. Artificial intelligence also operates in a legal grey area when it comes to music: if you start purely from the sheet music, copyrights expire 70 years after the composer's death. But many AI tools work with recordings - and these are protected even if they are interpretations of old works.

However, deciding whether or not music infringes copyright is neither theoretically nor practically easy. After all, even living composers learn by interacting with other works; it is hard to see why a machine should not be allowed to do this. And the discussions about what constitutes plagiarism and what is a permissible reminiscence are made even more complicated by artificial intelligence than they already were. It will undoubtedly be a while before clear-cut guidelines are developed.

A third question is likely to be on the minds of many people in creative professions: What does it mean if AI-generated music, artwork and lyrics keep getting better? Film music composers would not be the only ones to fear for their jobs.

Is there such a thing as machine creativity?

From there, we soon come to the really big questions: What do such developments mean for the history and self-image of mankind? What actually is creativity? And what role will artistic talents and visions play in the future?

Ali Nikrang knows these questions, and he answers them as a scientist: objectively, differentiated, but clearly. From his point of view, it is clear "that we are living in a time of upheaval - there will be a before and an after". It is equally clear that at the moment it is impossible to predict what the future will bring. If you ask him what will happen in a hundred years' time, he just laughs: "It's incredibly difficult to make predictions for the next five years."

What is certain is that the way in which AI creates art is fundamentally different from the way in which humans do it. Humans always have a goal when they compose music, write a novel or design a picture. An AI, on the other hand, has no such intentions. It is not consciously creative - but it is possible for it to deliver creative, namely surprising, new, valuable solutions. Ali Nikrang compares it to a room in which there are countless possibilities based on the training data: "When we create something with AI, we move around in this room and choose what we can use."

What happens when such a selection is not made was clearly audible at the "gameshow" in the Voltahalle in Basel. There were AI-generated pieces that were far more convincing than the Bach imitation. The Chopin-style piano piece, for example, began quite credibly. And the quartet for violin, clarinet, piano and percussion, modelled on Bartók, actually sounded inspired by folk music. The fact that the majority of the audience was nevertheless able to accurately distinguish the human from the machine compositions in all the examples was mainly due to the fact that the AI works did not take any particular direction.

Regardless of the style, there were always moments when the music got lost in repetitions or tangled up in melodic loops. In the Bartók quartet, the pianist couldn't help but grin during an endless series of staccato dabs. And in the Brahms-style quintet, too, one would have liked to give the music a nudge every now and then when it got bogged down again. What the human style imitators managed effortlessly - to span a musical arc over five minutes - was not possible for ChatGPT.

Communicating with the machine

Ali Nikrang is aware of the problem and even made it a principle in the development of his own award-winning programme Ricercar. It is an interactive programme in which the "composing" human has to choose one of five suggestions after 30 seconds of music. The selection then triggers another five suggestions for a further 30 seconds - and so on. "Purely machine-generated pieces interest me at most from a technical point of view, but not as an artist," says Ali Nikrang. "I see AI as a dialogue partner that can give me ideas that I might not have had myself."

The curiosity about these ideas is understandable. Musicians have always tried out new forms, sounds and technologies - so it's clear that something as powerful as AI is also being tested. Not just by composers, by the way: Programme booklet texts or entire festival programmes could also be generated by AI if people know how to use the technology.

And we can assume that a similar concert project will soon lead to less clear-cut judgements; composition tools will continue to develop rapidly. Ali Nikrang's Ricercar is already pointing the way in this direction, as it is the first programme to communicate musically rather than linguistically: To trigger the first 30 seconds of a composition, you give it a musical example and then use curves to determine at which point in the desired piece this "inspiration" should be taken into account and to what extent.

Above all, this means that you have to prompt well, i.e. ask the right questions. Modern artificial intelligence works on the basis of text, you have to give it instructions; the more precisely these are adapted to the needs of the machine, the better the results. This shows that, contrary to what was assumed in the early days of development, music is not easier to generate than text or images, but far more complicated. It is no problem to get an introduction to Mahler's Symphony No. 7 from ChatGPT. Nor is it difficult to order a picture of a cat skiing on the roof of a house. But how do you communicate what you want musically? How do you make something clear in a programme that defies precise linguistic description?

Another realisation from the Basel concert was that there are limits here. The Latvian composer, improviser and producer Platons Buravickis, who created the AI-generated pieces with ChatGPT, undoubtedly knows a lot about the different styles. So much so, in fact, that he managed to elicit works from a programme that is not geared towards music and certainly not towards copying styles, which, at least for short stretches, came very close to the required compositional manuscripts. Although he never succeeded in deceiving the majority of the audience, there were always a few dozen voices who thought his versions were humanly invented compositions.

George Orwell's vision

Only two questions remain: What is gained? And what will be lost?

The future will provide the answers. "At the moment, we are still mainly imitating old forms with the help of AI," says Ali Nikrang. "But completely new things will develop. Perhaps one day we will no longer watch films, but move around in interactive spaces. Musically, too, the strict concept of a work of art will probably dissolve." Entirely new cultural fields could open up and with them new tasks for those who are being weeded out in the course of digitalisation.

This may sound fascinating to some, but others are likely to swallow empty at the prospect. After all, anyone who enjoys composing, writing, painting or making music does so not only with a view to the result; it is also about the process itself. About what you think and feel while you are working with words, sounds or colours. It's about the meaning you find in these activities, about the satisfaction when, after a long period of practising and struggling, you succeed in creating something with artificial intelligence. Ali Nikrang is convinced that not all of this will be lost: "There will always be people who play the piano or form a band." But there could be fewer than there are today: "If everything can be obtained so quickly and cheaply, the motivation to do things yourself will probably decrease."

At this point at the latest, George Orwell comes to mind. In his essay "The Prevention of Literature", published in 1946, he considered, among other things, machine writing. Artificial intelligence was not yet known at the time, but Orwell's vision is remarkably concrete: "Perhaps a kind of inferior sensational literature will survive, produced in a kind of assembly-line process that minimises human initiative."

Ali Nikrang is not so gloomy, but he too has certain fears. If hundreds of thousands of songs are produced every day and released into the digital world, he says, "then it's possible that good things will go under". And there is another problem caused by the sheer volume of production: the machines are increasingly learning from examples that come from themselves. "If you don't intervene in a targeted way, it will all become monotonous at some point."

Orwell's vision then goes a big step further than purely artistic concerns: for him, only a totalitarian state could want such a machine-influenced culture; if it prevails, "it seems likely that the liberal culture in which we have lived since the Renaissance will come to an end". In view of the totalitarian fantasies of today's political and technological rulers, this is a thought-provoking sentence.

Above all, one hope remains: that there will still be people in the future who use their imagination, their talents and their efforts to create art - with or without the help of artificial intelligence. And one consolation remains: machines will never be able to decide whether what they produce is good. It will always be people who judge art, who are touched by music or not.