In this extended tutorial, iZotope’s Mike Metlay dives into the machine learning tech behind Music Rebalance in RX 7, its basic functionality, and some of the cool things you can do with it, from subtle and magical to over-the-top crazy. Let’s learn the basics of how Music Rebalance works, and how to use it, before moving onto getting weird and pushing it beyond the normal settings.

Machine Learning: The Science Behind The Magic

The secret to Music Rebalance is machine learning technology, which “learns” the elements of a track or mix and applies what it learns to various editing algorithms. iZotope first demonstrated machine learning tech in Neutron, a channel-strip-style suite of mixing tools. Mix Assistant in Neutron 3 listens to your mix and makes suggestions to bring out the elements you’re interested in. If you’ve read about or used Mix Assistant before, you’ll recognize certain elements of Music Rebalance right away.

This might sound like magic at first, but we run into machine learning algorithms like these all the time in our day-to-day lives—from virtual assistants that get better at understanding our voices over time (yay!) to bots that figure out which ads to plaster all over our web browsers (blech!). iZotope has pioneered the use of machine learning tech in music production applications.

Initially, users have been a little wary of software like this. There’s an element of “the machine shouldn’t be making my decisions for me” in their attitude, sometimes seasoned with a dash of “this suggestion didn’t work the way I wanted it, therefore the whole idea is no good.”

A Quick Caveat

It’s important to remember that, like every other module in RX, Music Rebalance is a weapon in your audio repair arsenal, not a magic cure-all. It can be used or misused, and the results can be wonderful, horrifying, or so horrifying they’re wonderful (we’ll get to that part later).

One other note before we begin: Music Rebalance is one of the few modules in RX that is so processor-intensive, it can’t do real-time previews at the full audio quality. When you preview the effect, you’ll hear the audio at reduced quality, and for really intense processing, there will be stops and starts as the software grabs chunks of audio to analyze. Actual rendering time depends on the analysis type and which elements are being modified, as well as the length of the file being rebalanced. Nevertheless, the processing time will always be shorter than the length of your audio file.

The Magic of RX 7 Music Rebalance: The Gain Sliders

The First Three Sliders: Voice, Bass, And Percussion

For this section, we’ll work with the first verse of “Guadalupe River” by Todd Barrow, a wonderful country/rock artist from Fort Worth, Texas. Todd was kind enough to grant us permission to work with this track.

Our first audio example is the verse, as released by Todd.

The mix is pretty straightforward, with nicely jangly guitars, a solid drum mix, tightly locked bass, and a welcoming and cheery lead vocal. So let’s mess with it, shall we?

As you can see in the screenshot below, Music Rebalance has a simple interface.

The vertical sliders control the relative gain of four types of elements of the mix: Voice, Bass, Percussion, and Other. Each slider has a Sensitivity setting, and under the “Separation algorithm” menu, you can choose to have Music Rebalance look at both channels at once (Joint Channel) or analyze each channel separately (Channel Independent). There’s also a more detailed—but also more processor-intensive—Advanced Joint Channel algorithm.

We’ll get into the analysis algorithms and Sensitivity sliders in Part 2. For now, let’s focus on the main sliders that control relative gain.

In the following audio examples, we’re moving each of the first three sliders 3 dB, first up and then down.

If you listen carefully, you might hear a bit of artifacting here and there, but for the most part, it just sounds like these elements were mixed that way! The fundamental power of Music Rebalance is that you can do this with a summed stereo file and still get transparent results.

However, if you boost any part of the signal, especially by this much, be sure to adjust your input level so your output doesn't clip.

As you use Music Rebalance, you’ll learn that the original track strongly influences how much the module can do while still sounding musical, rather than artificial. This mix has a clean separation of elements across the frequency spectrum and doesn’t slather on potentially disruptive effects like reverb. This helps Music Rebalance isolate the various mix elements more easily and give more transparent results.

In my tests, I found that I could boost or cut these first three types of elements by up to 3 dB each before getting into real trouble and that my sweet spot for really musical results was about 1.5 to 2 dB. Interestingly, cutting an element created more artifacts than boosting the element in this particular mix. It’s very important to set Music Rebalance so you just begin to hear the effect and then back off by 0.5 to 1 dB. After you render the result, you’ll be glad you didn’t push it too hard!

That Fourth Slider: Other

The Other slider is a tricky one because until you try it, you’ll never precisely know what you’ll get. In simple terms, Music Rebalance treats Other as “everything I didn’t identify as voice, bass, or percussion.” In practical use, messing with this slider can produce some pretty weird effects, especially if you push it too hard—and you can’t push it nearly as hard as the other three in most cases.

Check out these audio examples:

In these examples, the end result sounds like we’ve just pulled back on most of the mix elements, allowing the guitars to come forward. That’s because this mix is as simple and uncluttered as it is. Feeding more complex mixes into Music Rebalance will often give you results that are a lot less predictable.

One important thing to realize about these sliders is that they work in relation to one another. While it’s always tempting to make “everything louder than everything else,” pushing all the faders up will only make the entire mix louder... and introduce artifacts from the module in the process. Aside from the contexts of vocal isolation or removal, Music Rebalance is best used with subtle boosts or cuts to one element, or with a gentle push/pull between elements that are fighting each other.

The Magic of RX 7 Music Rebalance: Sensitivity and Separation

Sensitivity Sliders

Each of the four types of mix elements in Music Rebalance has a Sensitivity slider. The module's machine learning tech is designed to separate different audio sources, and Sensitivity controls how picky Music Rebalance will be when identifying and separating audio information into the four groups. 

Higher Sensitivity settings produce more exact, pronounced source separation. This will make Music Rebalance more selective in what goes into each group, but it may create unwanted audio artifacts.

On the other hand, lower Sensitivity settings produce smoother, more blended separation. This will reduce artifacts, but may result in some groups containing audio content from the other groups—e.g. percussion bleeding into the vocal group. These Sensitivity sliders have a range of 0.0–10.0, with 5.0 being the default value.

This time, we’ll be working with the guitar solo break of “Guadalupe River” by Todd Barrow from Fort Worth, Texas.

We chose this solo because there’s no vocal to get in the way of us hearing the guitar, bass, and drums. If we boost the Percussion group by a way-over-the-top 6 dB—twice as loud as before—we can easily hear when the correct instruments are placed in the Percussion group.

Note: Since we’re boosting a group by 6 dB, we’ll have to adjust our listening levels for comfort. Boosting a group will most likely increase the RMS level, and potentially the peak level as well.

Check out the audio samples, first without any Music Rebalance:

Now listen to the solo with the Sensitivity slider on our Percussion group set to 0.0, and then to 10.0.

Note the difference in the sound of the rhythm guitar. It’s being strummed in a straight, percussive rhythm that follows the kick drum, which makes it a bit more difficult to separate from the percussion.

With Sensitivity at 0.0, the guitar is being accentuated with the drums. This is, of course, a result of boosting the Percussion group by 6 dB. But with Sensitivity at 10.0, Music Rebalance is more selective of what goes in the Percussion group, so the guitar is more muted and set back. Music Rebalance now has an easier time understanding that the guitar isn’t supposed to be a part of the percussion.

Here’s another example: the chorus of the song, where we treat the Voice by boosting it 3 dB. First, listen to the chorus without any Music Rebalance in play.

Now here it is again, first with Sensitivity of 0 and then with Sensitivity of 10:

Here, listen to how the voice sits in the mix. There’s more clarity and detail with the Sensitivity set to 10, particularly in the reverb treatment, as Music Rebalance is pickier about what counts as a vocal. Due to the artifacts though, some might find the result a bit unnatural-sounding.

Messing with Sensitivity takes a lot of patience, as differences between different settings can often be quite subtle. In the majority of conventional applications, the default value of 5.0 will do the job just fine, but it’s good to have that flexibility.

Separation Algorithm

The dropdown menu at the bottom of the module lets you choose three types of separation algorithms: Channel Independent, Joint Channel, and Advanced Joint Channel. What’s the difference, and which should you choose? While your ears should always be your guide, you can make initial guesses based on how the mix was originally created. 

Many genres, particularly those with a lot of stereo digital effects—most electronic genres, ambient/new age—feature a great deal of cross-channel mixing and ambience. In extreme cases, the stereo mix is more like a layer cake than a set of sources whose pan positions are easy to spot. When there’s that much stereo stuff going on, a Joint Channel analysis will produce the most accurate results, because the elements you’re trying to grab will be recognized across the soundstage. When ambiences are really complex, running Advanced Joint Channel analysis may bring more clarity, but this takes significantly longer to render.

In other genres, particularly “retro” styles like traditional country and folk, instruments are generally placed clearly in the stereo field, and their presence in the room is well-defined rather than smeared out. It’s fashionable in some circles to mix records in these genres “LCR," meaning that every sound source is panned all the way to the side or straight down the middle: Left, Center, or Right. In these cases, Channel Independent analysis will often get at the sources on the sides more accurately.

For this example, we’re working again on the chorus of “Guadalupe River," now boosting Other by 3 dB with Sensitivity set to 5.0. The examples use these settings, but with the three different channel analysis options:

What we’re listening for here is the sound of the rhythm guitar parts, which are independently tracked on the far left and right. Using Joint Channel analysis, the voice is brought forward a bit with the guitars, and they’re blended into a well-glued, centre-heavy image—which, to be fair, maybe the musical effect you want.

There’s more separation and less vocal emphasis in the Advanced Joint Channel analysis, but the overall impression of a centred guitar image remains.

Switching to Channel Independent analysis causes the individual guitar parts to leap out on the left and right. In fact, it’s quite jarring in places, an effect that can be softened by lowering the Sensitivity before doing the render.

Tip

As with everything else in audio engineering, practice makes perfect. Working with these parameters is key to understanding them. Start with tracks from your personal reference library, and then move to some of your mixes that you know very well. When you get into Sensitivity and Separation algorithms, you’re in the fine print of what Music Rebalance can do, and well-developed ears and instincts are necessary guides to getting the best results.

The Magic of RX 7 Music Rebalance: Getting Weird

We’re getting weird for part three, as we deliberately abuse the capabilities of Music Rebalance to do things to audio that are difficult to accomplish in other ways—at least, more difficult than moving a few sliders, anyway.

1. Going Full-tilt

Tilt EQ is one of the more popular types of equalization. It dates back to signal processors made for studios in the 1970s and is now available as a plug-in in most DAWs. The basic idea of tilt EQ is that you turn the bass up while turning the treble down, ideally keeping the average loudness the same while “tilting” the frequency response. The tone controls on home stereos worked together to accomplish this, and a fancy version called the Baxandall filter was developed for mastering use (you can find one in the Ozone 9 EQ).

With Music Rebalance, we can do something similar to Tilt EQ, but with finer control. We can add boom and power or bring out detail and sheen, but in a way that goes beyond traditional equalization. Music Rebalance grabs pieces of the high- or low-frequency material and emphasizes them in unusual ways, changing the complexion of the original music.

To demonstrate this technique, let’s check out the latest EP from French indie-garage band Normcore, Neighbors. This album offers its tracks under a Creative Commons Attribution license, meaning people can do whatever they want with the music, as long as they mention Normcore. So with that mentioned, let’s get started.

On this track, “Aside,” we hear wide ambience with booming drums and a male vocal drenched in echo. Lots to work with here, but with a very retro EQ profile:

First, we’ll tilt down toward the bass. In this context, “tilting down” means emphasizing the low end of your mix. So to tilt down on this track, we boost Bass and Percussion by 3 dB each. We then counterbalance that with a 3 dB cut to Voice and Other.

To pick out the details and accentuate interesting elements of the tilt, we’ve driven up the Sensitivity of the lower elements. An increased Sensitivity value results in more precise source separation, so this will focus our boosting more specifically on bass and percussion audio content.

A high Sensitivity setting also has the potential to produce audio artifacts if set too high, so we lower the Sensitivity of the Other group to prevent those artifacts. Originally we had lower Sensitivity on Voice too, but the vocals got too murky and benefited from tighter control. Here’s the result:

There’s a lot of power coming from the bass boost, surrounded by a weird sense of three-dimensionality that exists without being shrill. The original mix sounds trebly, almost sibilant, by comparison. Notice, however, that boosting the Percussion group also boosts the high end of percussive instruments, so this tilt is more nuanced than the standard tilt EQ.

What happens if we tilt up toward the treble? These settings are a bit trickier because the high-end information can distort and clip in less obvious ways than low-end slamming might. We back the Percussion off by 5 dB and the Bass by 3 dB, keeping Sensitivity low so they cast a wide net in pulling down the lows. Higher Sensitivities create an acid-trip-etched character for the chiming guitars and the lead vocal, which gets punchier and more present without becoming shrill:

This sort of tone-tweaking with Music Rebalance goes beyond simply boosting and cutting frequency ranges in a mix. It dives into the mix elements themselves and brings out all kinds of unusual details. It’s easy to overdo, but that’s part of the fun!

2. Sucking Out Instruments To Transform Ambience

We talked in previous parts of this series about how one has to be very careful when cutting or boosting mix elements in Music Rebalance—you may recall that we rarely went over 3 dB in either direction unless we were really emphasizing an effect. Well, it’s time to break that rule in a big way. 

Massive boosts are obvious, and not much fun: they easily produce digital clipping and can destroy a mix. If you really want to distort your audio that badly, fire up Trash 2 and do it properly! But instead, what we want here is a massive cut, all the way down to silence. Music Rebalance will let you do that, and sometimes with wonderfully hideous results.

Choose a really obvious element in your mix and drop it down to zero. Then, play with the Sensitivity to get exactly the right feel. Low Sensitivity will allow a vague hint of the original material to sneak through, but higher Sensitivity will rip even that away, leaving a hole in the music where the targeted element used to be.

We start with a clip of roomy drums and guitar from the song “Work Music” by Anonymous420, from the album Corner Dot Com on the French label Loyalty Freak Music. Loyalty Freak releases all of its albums in the public domain, encouraging audio destruction like this—lucky us! Here’s the original section:

We start by dividing this clip into 8-bar segments, then pull the Percussion group down to silence. Every eight bars, we change Sensitivity from 0 to 10 and back, causing the drums to appear as a vague atmosphere and then vanish into an audio black hole. Boosting the Other group by 3 dB enhances the ambience in a horrible way:

This trick is so easy, yet does so much damage to once-balanced mixes. But why stop there?

3. Taking It Further With Different Separation Algorithms

When you exaggerate these massive holes in the music, you can make the effect even more startling by changing up the Separation algorithm. If you have material with a lot of joint information—heavy stereo ambience or room reverb—but you force Channel Independent analysis, you get the opposite of the nice stereo glue effect we discussed earlier. Instead, your previously lovely sense of space gets shattered into a stereo mess that makes headphone listening, a crick-in-the-neck challenge.

For this next one, we return to Loyalty Freak Music for the delicate ballad “Surfing Blood Tears” from the Tequila Moonrise album I Hate Myself And I Want To Tie-Dye. Here’s the original:

We pull the Voice group down to silence, crank the Sensitivity up to 10 to just affect the vocal level, and adjust the Separation algorithm.

In the following audio example, the first and third parts of the clip use Advanced Joint Channel for the cleanest results. When we hit the second and fourth parts, we change to Channel Independent and wave goodbye to normalcy.

When something’s worth doing, it’s worth doing way the heck too much, right? Right!

Conclusion

Naturally, unless your musical playground lies within one of the really extreme experimental genres–like phlegmbient, aubergine glitch, or Hackensack bounce—you’re not going to be using these tricks everywhere within a full mix. However, as bits of audio spice to catch and hold the interest of your audience, they have a lot of potential. Use these tricks on stems or submixes and gently blend them back into the full track to give your mixes something that will leave audiences dancing, nodding their heads, and saying “Wait, what?”

From subtle, musical mix tweaks to the far ends of experimental madness, the real beauty of Music Rebalance lies in its simplicity and ease of use. A little practice and a good understanding of your music can turn this module from an unassuming bit of magic to a fabulous tool in your audio kit. Now get out there and have fun rebalancing—or unbalancing—your music!