Blind Listening Tests – Introduction

One of my hobbies is audio, and I am a regular participant over at Audio Science Review, as well as some other sites.

I was interested in the subject as a young teenager, and used to listen to a reel-to-reel tapes of albums via an old Fisher tube amp (mid-1970s, this was). I used to go to the Lincoln Center music library and listen to records there on weekends, sometimes most of the day. I spent most of my allowance at Disc-o-mat, on Lexington Avenue. When I was 13, I talked my way into a job as a go-fer at an audio shop (Atlantis Sound) near my home. I was working for equipment, particularly a stereo to take to boarding school the next year. I spent a lot of time listening to all the options, and learned how the speakers that stuck out initially when you listen were often annoying after a period of time. I made a good choice, and enjoyed those (Acoustic Research) speakers for four years. I did learn some other life lessons. The store manager from Atlantis Sound was both a pervert and a thief – he threatened to anally rape 13 year-old me several times, and stole the entire store’s contents after I left. Ah, Joey G., where are you now?

In general, I bought the idea that different electronics sounded different – they certainly did to me. Right through college I never really questioned the idea, although I remember the advent of Monster Cable causing me some suspicion. My base assumption was that there were decreasing returns to cost, but there were returns. I didn’t think about the psychological effects of dancing meters, cool knobs, and high prices.

In 1987, this article in Stereophile (see around page 80) came out. They blind-tested golden-ears and skeptics on a variety of amplifiers, with a number of options to appease naysayers, such as extended listening, choice of switching, etc. That was my first exposure to a well-run blind test. When I first read it, I felt a little angry. How could this be? No way they couldn’t tell the cheap Pioneer from the mortgage-busting Futterman OTL. My reaction was sort of like this one, denial, with big helpings of special pleading and self-flattery on the side:

I suggest that audiophiles are music lovers who seek to explore an altered state of musical consciousness, a state that can only be entered with a precarious commingling of emotional commitment, artistic courage, and a musical stimulus of a high order. The intensity of feelings, knowledge, and decisions we make in this state are quite different from “normal” listening experiences.

Nonetheless, it stuck in my head. I began to think more about the non-aural cues I might be picking up as I listened to my own and other equipment. I also noticed they seem to wear off. The speaker you thought you would be happy with needed replacement a year or two down the line. It was a hedonic treadmill. I took my early lessons on listening fatigue to auditions, and wore out salespeople’s patience. And I continued to buy recordings – some LPs still, but mostly CDs.

Oddly enough, in around 1993, I found a combination that I was largely happy with for decades, and I focused on my growing music collection and going to concerts (mostly classical, since my wife is an avid violist, but some jazz). I stuck with the same amp and speakers, but I got new sources, as CD gave way to SACD, my LP collection began to gather dust, then I moved to a music server full of digital files. I dabbled in cables for a few years, but a quick A/B test disabused me of any idea that the expensive ones made a difference, at least to me. I was out of the audio *equipment* scene, rarely read the magazines or lusted after something new. I liked what I had. In 2018, I was moving to an apartment in NYC, my beloved Thiel speakers were broken, the company (who had fixed them twice *for free* was effectively gone to private equity). Plus, the amp generated a ton of heat, not a good feature in an overheated apartment. So, with trepidation, I went to some audio stores, and began to read again.

The places I visited showed me some new speakers, but they all started playing the same game with me – it wasn’t just the speakers, it was the amp (many pained expressions when I told them what it was, despite it being a Nelson Pass [he’s a thing] design with gobs of power), the cables, the source, the *ethernet* cables , power cables, wire, wire, magic wire. A couple of them tried to sell me brands I had been unimpressed with decades ago -they simply insisted I couldn’t have heard correctly – there were such large differences between these things. The weird thing was, it was the minority of salespeople who wanted to help me compare speakers. The others kept trying to swap in electronics. They acted as if they knew my mind better than I did. What’s cooking, Mrs. Stuyvesant-Oglander?

Well, I found some speakers I liked, but my online reading revealed something surprising. That Stereo Review amp test? it had effectively been replicated. Over and over again. Not just amps, preamps, cables, tweaks, the works. There was a wealth of research at my fingertips (more on this later – it isn’t *that* easy to find, or not easy enough for those who don’t want to find it). With the ongoing development of DACs (digital-to-analog converters), it seemed they also were increasingly indistinguishable as well. I work in a profession (money management) where retaining your biases is extremely expensive. I realized the evidence was not behind my priors. I ran some more comparisons at home, and my own results also replicated. In one case, I swore I heard a difference between amps. Level-matching and blind testing proved I had imagined it, or it had been a volume difference (louder usually sounds better). Amplifiers are a particularly contentious topic – here is a good article on hearing amplifier differences.

Of course there are diehards who just won’t accept that electronics don’t make much difference, once you eliminate volume, broken designs, adequate power, and reasonable impedance-matching. Everything from the digital transport/signal through to the speaker terminals is pretty well a solved problem, or can be solved cheaply (e.g. radio-frequency interference). Speakers and rooms are where the big, difficult differences are made.

My M.O. in online discussion is not to bring this up when people are bragging about or “reviewing” their equipment. If they’ve already bought it, I’m not going to tell them they wasted their money. However, when someone posts for advice on what expensive new amp or cable to buy, I will encourage them to consider it doesn’t make a difference. So I only voice my views in response to a request for advice. Nonetheless, at places like Audiogon, it is still the equivalent of dropping a large turd in the punch bowl. They tend to deny this body of evidence exists, and it’s difficult to find a simple citation that leads to a full body of evidence. You have to hunt around.

I wrote on ASR that we should crowdsource an authoritative catalogue of blind tests. There are some older ones, but the links tend to go dead, and most are not sorted in a useful way. After some discussion, it is clear this falls to someone with the energy to do so. So I will undertake, in my own time, to:

  • catalogue by type (cables, amps, etc.)
  • summarize
  • grade my perception of the methodology (rigourous, informal, etc.)
  • rate the credibility of the source (according to me)

I hope some of my fellow audiophiles will keep me on the straight and narrow.

First, let me link a few older catalogues, from which I will borrow, updating links along the way:

This post on Head-Fi is one of the better lists. Still many of the links are dead. It also isn’t sorted in a friendly way.

This one hurts to look at, and covers a variety of topics that are relevant, but aren’t strictly listening tests. Also many dead links.

Another audio forum attempt. Not as complete, but well-organized.

Here’s a bunch in German.

And here’s another. There are a few ‘successful’ tests in here, but note how all of them take advantage of a known bias – to hear a difference. In a nutshell, this is why ABX testing is preferable to AB testing. Still, if you can hear a difference, but can’t reliably pick the same one each time, what is that telling you?

Finally, I want to give special recognition to the invaluable and even-tempered Archimago, who has painstakingly created all kinds of cable/resolution/transport tests you can download or take online.

It would be nice to get links for this list of useful citations.

Leave a Reply

Your email address will not be published. Required fields are marked *