The video below is about guitars but even if you’re not a guitarist there might be a point made here from which anyone involved in audio can learn. The video from YouTuber Jim Lill has attracted a lot of attention from the online guitar community. In it Jim questions why his expensive electric guitar sounds better than a cheap one. Asking some respected guitar builders all of them refer to the materials used to build the guitar as being the most important reason. Jim set out to test this by changing one thing at a time and came to the interesting conclusion that the only things apart from the person playing which make a difference are the strings and the pickup.
Interesting as this might be how does this relate to audio engineering beyond guitars and guitarists?
Scientific Method
We all make assumptions about the tools we use and reference anecdotal evidence and hearsay to justify our beliefs but there is no substitute for actually testing your assumptions. We all know we ought to but most of us rarely do. While we ought to be too busy to exhaustively test every decision we make that doesn’t mean that we shouldn’t occasionally take the time to challenge our expectations. Whether it’s as simple as choosing a stock compressor over the premium alternative and getting a friend to AB between them for you, to something more sophisticated such as using the free version of the Hofa 4U + Blind Test plugin, it’s a useful exercise.
The scientific method is difficult to properly implement in these situations. The testing of a hypothesis under controlled conditions with the results being properly peer reviewed is probably further than we need to go, but seeking to introduce at least a degree of control into some rudimentary testing of our assumptions is very worthwhile indeed.
Blind Testing
The importance of blind testing is the easiest and most beneficial to introduce. If we know which of two examples we are listening to it is impossible to eliminate our personal bias from the listening experience, hence ‘blind’ testing. If the test is being conducted by someone else (as in the example above where I suggest getting the help of a friend to aid with the testing) then they are also a source of potential unconscious bias.
So why would sensible people sometimes fool themselves into believing what they want to believe? When you look at the psychology involved it’s pretty understandable and I’d trust the decisions and opinions of someone who admits to this kind of fallibility over someone who believed they were ever truly objective.
Expectation
Expectation is a huge source of unconscious bias. The jokes about adjusting compressors while in bypass are only funny because we recognise those moments when we’ve done it ourselves! If we are using a premium plugin which we’ve paid good money for, we expect it to sound better, so it does. If it looks pretty and has a cool display that probably helps. If it’s a piece of hardware then we’re even more invested, both physically and financially and if it’s boutique or vintage even more so. If I’m in a studio and they have a Fairchild I’m putting something through it whether it’s sonically appropriate or not!
It’s exactly the same process which is at play as when guitarists go a little wobbly in the presence of a Dumble amplifier or a pre CBS Fender. And it’s completely understandable.
What is really interesting is what happens if an assumption is challenged or even undermined like it is in this video. Many people’s first reaction is to challenge methodology, to cast doubt over an uncomfortable result. If this result doesn’t match my expectations the test must be at fault…
Null Tests
The nuclear option when it comes to comparing alternatives in pro audio is of course the null test. If two level-matched alternatives cancel when one is inverted then they are demonstrably the same yet for some even this doesn’t close down the debate.
Of course the sound isn’t everything and I’d be the first to agree that the difference enjoying using the gear makes to how you work and the decisions you make is probably more significant than the, often marginal, differences between two alternative processing options. In the same way as a guitarist plays differently when using a great feeling instrument, a mix engineer might well make better decisions when using gear which gives them the feeing that something special is happening, and if they feel that way then something special is probably more likely to happen.
People respond to their environment and their experience is by definition subjective so let’s be honest with ourselves about it. Test your assumptions occasionally. You might be surprised at what you find.