Apple’s new Spatial Audio feature uses Dolby Atmos sound-mixing technology to bring immersive music to the Apple Music platform. These are the early days of immersive music streaming, and there are many questions about Spatial Audio coming from both consumers and professionals. Nathaniel Reichman acknowledges that he doesn’t have all the answers, but he has already mixed and mastered several albums that you can find on Apple Music in Spatial Audio and has learned from the process. Over to you Nathaniel…

Introduction

For the purposes of this article, I’m going to assume most of you already have a rough grasp of the Dolby Atmos workflow. To summarize, the Atmos renderer works with many DAWs and acts as a kind of super-panner/recorder. You can have up to 128 channels of audio and metadata go from the DAW to the renderer.

The renderer software, often the software-only solution Dolby Atmos Production Suite, makes a Dolby Atmos master file (.atmos), an ADM file (which has become the standard delivery format for immersive mixes), MP4’s for consumer device testing, and various re-renders, which can be alternate mixes like 5.1, binaural, or even straight LoRo stereo.

I recommend reading the Atmos user manual and also watching Alan Sallabanks’s excellent videos on this site. After all the Dolby Atmos content here on Production Expert, I do recommend you take a look at the Dolby Atmos Post & Music forum hosted by Dolby as for me it is unmatched in depth and breadth, and the Dolby engineers themselves are active on that forum.

Terminology

Spatial Audio is the name Apple has given to the immersive music on their platform. The term Immersive means any audio presented in higher than conventional 5.1 or 7.1 formats. Auro 3D, Sony 360, Fraunhofer’s contributions to MPEG-H, and many others are all immersive audio formats.

Dolby and Apple worked together to bring Spatial Audio to the Apple ecosystem, and music is branded as being Dolby Atmos on supported devices. So why the new Apple-invented name? We’ll take a look at the name and some of the possible reasons behind it at the end of this article.

Mixing For Apple’s Spatial Audio

Mixing for Spatial Audio is very similar to mixing any Atmos content. It has to be done with a compatible DAW, and the Dolby Atmos Production Suite (DAPS), or the Dolby Atmos Mastering Suite (DAMS).

You will utilize a mix of beds and objects and record a .atmos master file, which appears on your computer as a folder with usually three or four files inside containing the audio and the metadata.

Dolby and Apple ask that everyone mixing in the format check their work on at least a 7.1.4 loudspeaker system as well as headphones. Some producers might be tempted to bypass the loudspeaker check because of the expense and complexity. This is not recommended.

The loudspeaker experience can’t be replicated in headphones, and you can easily make significant mistakes in staging and correlation if you haven’t heard your mix on loudspeakers. Conversely, most consumers will hear your mix on headphones, and if you don’t take that format seriously, you’ll also run into trouble.

I’ve worked on many classical music albums, and the 96kHz sampling rate is considered a minimum standard in that world and in fact a number of my albums actually started in the exotic DXD/DSD formats. Currently, although you can work in Atmos at 96kHz, you can’t deliver to the consumer at that sampling rate. You can only deliver 48kHz files for Atmos and Spatial Audio. I recommend that you try and work at 96kHz and downsample to 48kHz as late as possible in the post-production process as you can. I typically do it with the Dolby Atmos Conversion Tool after I’ve made a .atmos master, but before I make the Audio Definition Model (ADM) deliverable.

You Still Need To Produce A Stereo Master As Well For Apple Music

Apple is not using Spatial Audio as a replacement for your conventional stereo master. This is very important for two reasons:

1. If you’re organized and have recorded your album at high enough sampling rates and bit depths, your stereo master can be eligible for the Apple Digital Masters program, which I highly recommend enrolling in. Apple Digital Masters ensures that the lossy AAC file Apple creates of your mix is done at the highest possible quality.

2. Your stereo master is also usually eligible for the Apple Lossless program. Spatial Audio is not lossless, so for the stereo-purists out there the lossless option is important.

While streaming, Spatial Audio runs in parallel to your stereo master. On an iOS device under Settings/Music/Dolby Atmos, you can switch on-the-fly between the two formats. This is a wonderful feature. I think I spent an entire evening toggling between the stereo and Spatial Audio versions of St. Vincent’s outrageously fun new album Daddy’s Home on my favorite headphones.

Because of this quick switching, Apple asks that your stereo master and Atmos master are as tightly synchronized as possible. However, because Atmos masters can only be edited to the nearest frame, there is a limit to how tight you can get this.

Before I get too much further, I’m going to share a checklist I made for myself which is a mix of Dolby and Apple’s technical guidance and my own discoveries sprinkled in and if you have anything to add then please do share that in the comments below.

Before I deliver anything to a record label, I go over this list carefully to avoid mistakes:

Nathaniel’s Atmos Music Pre-flight Checklist:

• Using BIN settings plug-in, punching-in is somehow disabled. Make inactive.

• Setting an FFOA (first frame of action) causes a 1-frame problem with ADM files. Do not set an FFOA.

• All deliverables MUST be created with a timecode of 24 fps.

• All deliverables MUST use 24-bit PCM resolution at a sampling rate of 48 kHz.

• All deliverables MUST achieve a True Peak measurement that does not exceed -1 dBTP.

• For individual tracks, all deliverables MUST achieve an integrated Dolby Atmos loudness that does not exceed -18 LKFS, based on ITU Recommendation BS.1770-4.

• For albums, integrated loudness MUST be measured on each individual track, rather than the overall album.

• For individual tracks, the amount of peak limiting indicated for binaural headphone playback SHOULD NOT exceed 3 dB.

• All deliverables SHOULD have the Binaural Render Mode setting (Off, Near, Mid, Far) specified for each Bed Channel and Object.

• The DAPS binaural output can be used for QC. It is a reasonably good approximation of Apple Music’s Spatial Audio output when set to Mid. But at the end, listen to an MP4 in iOS Files app with AirPods Pro or AirPods Max for a near-exact experience.

• All deliverables MUST be conformed and synced to the corresponding stereo deliverable.

• All deliverables MUST be provided as an ADM BWAV file that conforms to the Dolby Atmos Master ADM profile.

Dolby Atmos Binaural Settings Plug-in

Starting at the top, Dolby has recommended that everyone use the free Dolby Atmos Binaural Settings plug-in. It is intended to simplify the communication of settings from DAW to the Atmos Renderer. For reasons that I still have not figured out, its use prevents me from punching in on my Atmos master files. And I love punching-in fixes.

Loudness Readings

Next, the loudness measurements can all be read easily in the Renderer in real-time and exported as spreadsheet data as well. With the frustrating exception of binaural peak limiting, which can only be judged by the small orange meter. I suspect Dolby will improve this read-out soon.

What Happens To The Binaural Settings?

Then we get to the weird bit that has caused a great deal of confusion since the launch of Spatial Audio. In a standard Atmos mix, there is the loudspeaker experience and the binaural headphone experience.

The Binaural Render mode settings page lets you determine the distance each object and the beds (as a whole) are heard. The aural differences between the settings of OFF, NEAR, MID and FAR are huge. Unfortunately, Apple’s Spatial Audio encoding algorithm ignores these settings.

Nevertheless, Dolby has explicitly stated that as producers we should continue defining these distances. But then how do we listen to the binaural sound of Spatial Audio if the Dolby Renderer is making a different sound? Unfortunately, the following are best practices as of the summer of 2021:

• Monitor the Dolby renderer’s binaural mix with all the distance settings set to MID

• Record a temporary Atmos master

• Export an MP4 of your master set to 768kbps

• AirDrop or iCloud copy the MP4 to an iOS device with iOS 14.6 or higher

• Open the iOS Files app and play the MP4 using Airpods Pro or Airpods Pro Max.

  • Note that while you can use any headphones to hear Spatial Audio in the Apple Music app, you can only use Airpods Pro or Airpods Pro Max to experience Dolby Atmos in the Files app to properly hear your mix as a consumer would.

  • Any other headphone models besides these two will result in the Files app downmixing your content to stereo.

• After you’re satisfied with the Spatial Audio sound, go back to the Atmos renderer and change the distance settings to taste (OFF, NEAR, MID, FAR).

• Re-record your Atmos master

• Export and then QC an ADM delivery file.

According to Apple, they are creating the binaural mix from a 7.1.4 downmix of the Atmos master.

The Consumer’s Loudspeaker Experience

Regarding the loudspeaker experience, TIDAL and Apple Music sound nearly identical. Lacking a compatible mobile device for TIDAL, I can’t comment on TIDAL’s binaural sound. But I’ve been enjoying TIDAL’s loudspeaker sound for many months now, and gain-adjusted albums on the two platforms almost completely null cancel.

However, Apple Music on most receivers plays about 10db louder than TIDAL, making it more likely to hit the playback limiter. John Loose at Dolby chalks this difference up to the different dialnorm settings between the two platforms. Both platforms are working on this and Apple consumers should set iOS devices to Soundcheck ON for the best experience.

The Affect Of The Bit Rate On the Height Channels

By far, my biggest issue with TIDAL and Apple Music is the 768kbps bitrate, they have chosen to use. (Blu-Ray doesn’t have this limitation). Atmos smartly delivers the most data to the LCR and Ls/Rs. But in a complex mix, soloing the ceiling channels often results in a cringe-worthy MP4 swish-fest.

The entire mix might sound amazing, but don’t solo the Rtr channel with a client in the room. There is also a subtle and intermittent crackling issue heard more often in classical music that Dolby engineers are looking into now at my behest. And John Bowen, the audio wizard behind these great podcasts (here’s a link to the first in a series of excellent binaural podcasts Bowen has produced) has pointed out that…

“The closer your pan is to the ‘listener,’ the more accentuated the crossover from one side to the other. This is most pronounced using FAR mode, and least using NEAR (or OFF). MID seems to most faithfully represent the loudspeaker mix. All of these observations are highly unscientific, and these things do change with updates.”

That said, these are small artifacts in what has otherwise been an excellent technical roll-out of Atmos and now Spatial Audio-compatible consumer devices.

Aesthetics

Listening to Dolby Atmos albums with TIDAL or Apple Music on loudspeakers in a calibrated 7.1.4 or 9.2.6 room is wonderful. It’s a quantum-leap over stereo and a huge improvement over surround sound.

I feel like a kid in a candy store surfing Apple Music in 7.1.4. Spatial Audio on headphones is noticeably better than stereo on headphones, and gives producers new tools, but is a less startling difference when compared to loudspeakers.

Most of the Atmos albums I’ve heard fall into two camps when it comes to staging: beautiful albums like Pink Sweat$ and Albrecht Mayer where the musicians are mostly in front of the listeners, with a conservative use of rears and overheads. Or adventurous albums like Fantastic Negrito and Steven Wilson where the musicians are everywhere and there is no sweet spot in the room because the sound is everywhere.

Too Much Correlation Creates Dead Space in Atmos

However, one of the biggest mistakes I hear (and one that I made early in my immersive journey) is the excess correlation between channels. In straight stereo, complete correlation results in phantom center sounds, something we’re accustomed to. But in Atmos, pan moves like pulling stereo sounds into the middle of the room, or increasing object size too much can result in excessive correlation, which can create a static dead space.

In one of my early mixes, I worked too hard to fit my stereo Lexicon reverbs into a mix and it created large dead spaces but Cinematic Rooms came to the rescue.

One of the huge benefits of classical music-style recording is the common use of Morten Lindberg-style trees in which (11) or more mics are used in the room that produces a natural decorrelation (for one of the best 5.1 music recordings of all time, listen to Lindberg’s Spes). These recordings also usually sound best with binaural distance parameters set to OFF, since they have natural binaural information, whereas conventional pop-music recording with isolated overdubs will benefit more from binaural distance parameters being set, as will spot microphones in a live performance.

Panning In Headphones

In 2017, our studio had a podcast client that loved the binaural mixes we were making in an early version of DAPS. During that time, I came to realise how unnatural conventional stereo panning is in headphones. Whereas panning an Atmos object at MID distance in headphones is much more similar to how the real-world sounds.

Switching to a conventional panner gives an almost painful, vacuous feeling as the sound moves from center to one side. It’s hard to describe without comparing it, but when a conventional panner moves from center to the side, it enters a narrow, unfocussed space and then pops up far to the left or right.

In contrast, an Atmos panner moves the sound smoothly to the side without any lack of focus (once you hear this, you won’t forget it, and all conventional panners will sound odd afterwards).

How Does Binaural Sound In Spatial Audio Compared To The Dolby Atmos Renderer?

So if we can’t tweak the sound of Spatial Audio in binaural, what’s it like compared to the Atmos renderer? Fortunately, not too bad. To my ear, most of the excellent binaural panning is preserved, and while you can’t change the distance parameters, to me, they sound somewhere between NEAR and MID, but much closer to MID.

I have found that the spectral quality of Spatial Audio has a little more bottom end and body than Atmos, which can be dangerous if your Atmos mix is already thumping). But most unsettling is that the object localization is not quite as tight. And some producers I know are complaining about single lead vocal objects coming in slightly lower in level compared to Atmos.

None of these minor complaints amounts to an argument against using Spatial Audio. I’m in the midst of a jazz/funk album and the Spatial Audio rough mix sounds significantly better than the straight stereo mix does. I look forward to a future where Spatial Audio lets us tweak parameters more deeply, though.

Argument - Will Dolby Atmos Become Irrelevant

Quad, Super Audio CD, DVD-Audio… the sceptics and the naysayers have a lot of historical examples to point at and declare that Dolby Atmos will eventually suffer the same fate of irrelevance.

I would make two arguments against this fate:

1. Atmos doesn’t supplant the excellent workflows and traditions we’ve established with stereo, it sits alongside stereo as an opportunity.

2. The device landscape has changed.

Modern cars have lots of loudspeakers, smart speakers are filled with drivers pointing in all directions. Television soundbars have countless drivers. The pandemic has inspired many people to improve their home theater systems. The laptop I’m typing this on has six drivers. With all the computing power in our devices, and the availability of loudspeakers big and small, it would be a huge missed opportunity not to produce content that takes advantage of this new landscape.

The music, television and podcasting clients we’ve been able to give in-person demonstrations to, have been bowled over by the sound of immersive. Granted, only some of them have the resources to demand immersive on a spec sheet, but everyone hears the value.

Conjecture - Why Use The Term Spatial Audio?

Why did Apple adopt only some of Dolby’s technology and then brand it under the new name “Spatial Audio?” I have no insider information, but I suspect it has to do with Apple wanting some independence from Dolby should the corporate relationship sour.

The ADM (Audio Definition Model) format that has become the delivery mechanism for Atmos albums and television shows is an open-source format. Dolby doesn’t own it. Although Dolby does make the best production and distribution tools, but there’s no legal or technical reason stopping producers from creating and playing back ADM files on other platforms.

This brings us to Logic. My guess is that the real-time Spatial Audio binaural-QC-platform is coming and that it’s coming inside Logic Pro.

How the Atmos/Pro Tools ecosystem could incorporate Logic to make an end-to-end Spatial Audio system is anyone’s guess. Watch this space and if you’re serious about producing music in Atmos, take time to read, take time to practice new software workflows, and take time to share your work-in-progress with others.