EarWRM
Write ┃ Record ┃ Mix
door Steven Maes
Colleagues who are actively engaged in three-dimensional music today will likely agree with me that there is nothing more important than considering, from the very start, how to approach an immersive music project in terms of composition, recording, and mixing. From the outset, even before a single note has been recorded or even played, one must carefully consider the concepts of front-back, left-right, and above-below.
​
The song or composition must be conceived and created in immersive audio. The microphone setup, the space where you will record, and finally, the production process: everything must be viewed through the lens of a three-dimensional musical experience. In stereo, we often do this subconsciously as well, since we place microphones in the space, work with stereo microphones, and already consider the spreading of choirs, guitar riffs, or specific reverb or delay when composing.
We know from research, papers, and reference works a great deal about which stimuli matter. For instance, it is impossible to give low frequencies an exact position in the vertical space. We are only able to accurately place sound above us from 7000 Hz onwards. All these parameters can be found in two excellent reference works: “Immersive Sound: The Art and Science of Binaural and Multi-Channel Audio” and “Perspectives on Music Production in 3D Audio”. Both works are theoretically correct but consistently overlook the fact that the first step is creation. If immersive and spatial possibilities are not considered during composition, 3D music can become quite dull.
Why we must compose in immersive audio
In popular music, the current standard is to upgrade stereo mixes to immersive audio. In a multi-speaker system with 10 or 11 speakers, not much more happens than some flying point sources and stereo vocals spread across the left and right front speakers. Then, the singer is placed in the center channel, and the immersive mix is done. Everything that was intended for the stereo mix is reused, occasionally tweaked with some reverb and EQ, and then reinserted into the immersive mix. This results in a simple yet fundamentally irrelevant upgrade from stereo to immersive. It is far more logical to go from an immersive mix to a stereo mix. The musical information absent in stereo cannot be magically added when upmixing from stereo to immersive. In contrast, a downmix to stereo gives you complete freedom to decide what to do with all that additional information from 3D audio.
​
Apple Music offers extra banners on their online platform when music is submitted in both stereo and spatial audio, which is valuable additional publicity for producers. Record labels go to great lengths to quickly and cheaply upload their new releases in both stereo and immersive formats to Apple Music. There are certainly exceptions within the labels and musicians who know what they are doing. However, the majority are still searching and remain unaware. Artists are hesitant to let their creation morph into something unrecognizable, akin to an octopus with more tentacles than intended.
​
When a mixer or producer must quickly produce an immersive mix at the end of a project for the record label, this creates stress. Artists prefer to play it safe and stay close to the stereo version. Had musicians been aware of the possibilities and artistic value of immersive audio, they would not view it as an additional task or obligation, but rather as an opportunity to express themselves artistically.
​
However, some well-known composers have already begun to experiment.
John Cage (1912-1992)
Immaginary Landscapes Nr. 1 & 2
Karlheinz Stockhousen (1928-2007)
Gesang der Jünglinge
These are compositions made for live performances with audiences and multiple speakers.
​
A project that particularly inspires me is “Poème Électronique” composed by Edgard Varèse (1883-1965). It was an 8-minute work performed live with 425 speakers in the Philips Pavilion during the 1958 World’s Fair in Brussels. The performance was delivered by four individual tape recorders running at different speeds, causing them to gradually fall out of sync. The audio had to traverse pre-determined paths in the space, something you can clearly see in this video.
Content is king?
Bill Gates wrote "Content is King" on the Microsoft website in 1996. Although I am not particularly fond of Gates or the word "content," I must fully agree with him in this instance, especially when it comes to immersive music.
Our apostolic work at present largely consists of demonstrating that things can be done differently and that it is indeed possible to elevate productions artistically and substantively to another level in a simple, efficient, and affordable way.
Demonstrating good immersive productions is often enough to pass on the 3D audio "bug" to musicians and producers. Teaching them to be creative is unnecessary; it is already in their DNA.
What goes wrong?
1/
​
Composers, musicians, and artists cannot, during their creative process, hear what 3D audio could mean for their new composition.
I myself, in the early days, especially with acoustic classical recordings on location, had to rely on my instincts and experience since immersive listening was not available. With a bit of luck, it sometimes worked, but progress is slow, as you cannot build upon a consistent sequence of artistic ideas. Particularly with microphone placement for height, listening is indispensable. Therefore, I devised a solution with six speakers: a quadriphonic horizontal first ring and two speakers for height. The center channel is presented via the left and right front speakers as a phantom source. The height information can be reasonably estimated with two speakers when it comes to acoustic data. On its own, this is a good solution, but still not cheap.
​
2/
​
In pop music, there is an evolution from studio producers to bedroom producers, beatmakers, and DIYers. Investing in expensive speaker setups is clearly not their priority. For example, an electronic musician in his home studio is tinkering with sounds and beats with conventional stereo listening. Why would this person experiment with 360° audio? It would be like an astronaut preparing to go to the moon, but only knowing about hot air balloons. However, give that electronic musician 11 speakers with the necessary software to create music with, and you will trigger a creative eruption. Unfortunately, there is rarely a budget for this. A vicious circle.
In the future, I believe binaural headphone listening with headtracking will be the solution for these musicians. I can imagine that, as a highly creative individual, you are not yet excited by the current 3D headphone experience. You can see that the entire audio industry is working hard to improve and enable this. At present, in our immersive lab, we have a test setup that closely resembles what I just described: a headphone system with headtracking that sounds nearly as good as a multi-speaker system. Sooner or later, a good binaural headphone experience will be available, allowing musicians and producers to fully unleash their creativity with immersive compositions.
Is there no suitable content available?
I am aware that my personal experience, testing, and taste unfortunately do not work in favor of existing released immersive pop and classical productions. As always, commercial mechanisms and ignorance come into play. Re-releasing a back catalog is always cheaper and easier than creating new content. Additionally, there is nothing simpler and cheaper than upgrading existing recordings to so-called 3D audio.
​
Our WRM concept of Composing, Recording, and Mixing in immersive audio is unfortunately still not the standard. Although I am more convinced than ever that this is the way forward. Through the use of space, frequency information, movement, reverb, and panning, you can do so much more in a 3D environment than in stereo.
I hope and dream that this will trigger musicians' creativity so strongly that we will soon be able to speak of a new way of composing. Hopefully, this will cause a small tsunami in the more conservative pop industry. Let the young musical revolutionaries take new, unexplored artistic paths, and before we know it, we’ll have a few new music styles.
