Audio Science Review

[MLXXX]

58 minutes ago, LHC said:

I believe you do.

No one questions Ethan Winer's sincerity. The question is his way the best way? Or is he, just like everyone else, influenced by his own beliefs and biases? Maybe take a read of the exchange between Winer and Mark Waldrep at the bottom of the comment sections here: http://www.realhd-audio.com/?p=6197 

 

 

 

I've started reading the main article you link to above, and early on I see a reference to triangular dither. That would pretty much invalidate the exercise I'd have thought.  I recall that triangular dither is just audible. Best practice is to use a noise-shaped dither. ...

 

Now I see a dramatic-sounding reference to the dynamic range being 50-60dB less; but of course that would typically all be below audibility. Ordinarily @LHC, I'd just stop reading at this point because whoever wrote the article appears to me at this stage to be well out of their depth.   Anyway, I'll keep reading. ...

 

Ok, I've now got to the comments section. How great that Ethan Winer offered to use the files as source material for a blind test he was prepared to run. He said:

 

To anyone who lives near me in New Milford, Connecticut (near Danbury), you are invited to arrange a visit and I’ll play Mark’s files through either or both of my two very high fidelity systems while you listen blind. That avoids any chance of cheating, and then we can report here what happened. This is the only way to know if people can tell HD from CD, short of Mark doing what I did to add fake ultrasonic content to defeat cheating.

 

That's an example of Ethan's openness, though in my opinion he was being very generous with his offer.

 

And this response came back:

Ethan, I’ve acknowledged that this survey is not a rigor study. I asked participants to confirm that they only listened before making their choices and will trust that they did so.  

 

And indeed this would not be a rigorous study. Ethan goes on to question why the author is even bothering to try to test the matter seeing as how it has been tested previously.  The author comes back with this:

You insist that history has already determined that ultrasonics have no impact — maybe they aren’t perceptible. But it’s simply too easy to capture, reproduce, and distribute audio that matches the entire frequency output of a musical performance, I would argue why not?

 

@LHC if you want to yourself you can download the test files and play them back with automated ABX software at normal gain settings (not at elevated settings to enable you to hear quantization noise).

 

I'm afraid this really is an example of flogging a dead horse. And Ethan says so. Paul would not say that. If he did, that would alienate a segment of his support base, and offend the hi-res distribution industry.

 

You will see that the comments petered out. No one reported being able to hear a difference.

Edited August 5, 2019 by MLXXX

[Audiophile Neuroscience]

8 hours ago, MLXXX said:

A time might be reached where I truly won't be able to tell the difference between an entry level system and a mid level system. As I get older I'll lose more of the high frequencies, and eventually I might become legally deaf.

 

For now, though, I don't think I have too much to fear. 

 

 

Ah, so you are saying its okay to dismiss blind testing because you can trust your ears.

[Audiophile Neuroscience]

14 hours ago, LHC said:

Has it completed? Isn't that just your own assumption? In real life are there no unknown unknowns? You should start off by asking me if there are any other factors in play. 

 

I think you get where I was going with that. If we know everything there is to know, then one could argue probability is objective. But if we accept there are things we don't know, some discoverable and some not, then probability must be subjective. This allows us to reconcile when shockingly unexpected events happens. This goes way beyond a simple discourse on significance. 

I don't see the problem reconciling probabilities being objective and known unknowns, unknown unknowns and unknowable unknowns. probabilities merely are an estimate of likelihood and in my view that incorporates unexpected events including things we can't anticipate. 

 

If the sun rises every morning for millennia what we can say this is probably not due to chance. The mechanism behind it may be unknown. It may not rise tomorrow due to some unexpected event we have no knowledge of but it is improbable ie unlikely.

 

Quote

Where this may be relevant to audio testing is that between the objectivists (we know everything and can measure them), and subjectivists (we can hear everything and don't care about measurements), is a middle ground occupied by people like PS Audio Paul McGowan who posits that there still things we are yet to learn and may discover them in the future. 

 

 

 Keeping an open mind IMO is not related to probabilities being objective. In my view the S and O debate is more about subjective beliefs regarding probabilities which in the typical case are unkown. It is speculation one way or another about the outcome of the probabilities such as the probability of intervention X leading to audible difference.

Edited August 6, 2019 by Audiophile Neuroscience
typos

[Audiophile Neuroscience]

8 hours ago, proftournesol said:

Another hypothetical that I raised before, but nobody took up the question: If you auditioned a component that you clearly preferred in your system to your present component, yet it measured considerably worse than your existing component, would you buy it for your system?

It clearly sounds better, it isn't an ambiguous 'different sound'

you can afford it

No squibbing by saying that it couldn't sound better if it measured worse.

Definitely. I always listen first and then see if measurements are concordant, if and where possible.If the mismatch was significant it would give me pause to go back and re-listen.

[MLXXX]

1 hour ago, Audiophile Neuroscience said:

Ah, so you are saying its okay to dismiss blind testing because you can trust your ears.

No, I'm saying that your hypothetical example was too extreme to be credible, in my opinion. As I said it's a bit like suggesting that a $5 bottle of red wine "tasted the same" as various $50 bottles of premium red wine. Unless the contents of the $50 bottles were spoilt or the $5 bottle ridiculously discounted, the situation would not arise in practice.

Edited August 5, 2019 by MLXXX

[Audiophile Neuroscience]

10 minutes ago, MLXXX said:

No, I'm saying that your hypothetical example was too extreme to be credible, in my opinion. As I said it's a bit like suggesting that a $5 bottle of red wine "tasted the same" as various $50 bottles of premium red wine. Unless the contents of the $50 bottles were spoilt or the $5 bottle ridiculously discounted, the situation would not arise in practice.

Ah, so you are saying it is okay to dismiss blind testing when you are convinced you can trust your ears.

[proftournesol]

8 hours ago, MLXXX said:

What type of component could this be? Loudspeakers I could understand.  As for other components, these days most measure very well don't they? 

 

I'm not sure what might measure "considerably worse".  Could you give us an example, @proftournesol?

Hypothetical, take your pick: turntable, cartridge, CD player, amp, speaker, DAC

[MLXXX]

1 minute ago, Audiophile Neuroscience said:

Ah, so you are saying it is okay to dismiss blind testing when you are convinced you can trust your ears.

My posts suggested the blind testing result you posited was too far removed from likelihood or reality to occur. So there was no need for me to decide whether or not to "dismiss" it. 

 

As you may have gathered from other posts of mine, I am a strong supporter of the use of the DBT process. It's just that your hypothetical example struck me as too far-fetched in relation to my own hearing, if the test music included classical orchestral and piano and string chamber works. With certain types of jazz music I might think a cheap system of similar quality. 

 

It's interesting that promotions of low bitrate codecs often include the boom-tish genres of music. These can sound quite impressive even at a low bitrate. Classical music however will "suck" at a low bitrate.

[Audiophile Neuroscience]

1 minute ago, MLXXX said:

My posts suggested the blind testing result you posited was too far removed from likelihood or reality to occur. So there was no need for me to decide whether or not to "dismiss" it. 

but you did dismiss it by deciding it was unnecessary and that in situation X you could clearly trust your ears.

1 minute ago, MLXXX said:

As you may have gathered from other posts of mine, I am a strong supporter of the use of the DBT process. It's just that your hypothetical example struck me as too far-fetched in relation to my own hearing, if the test music included classical orchestral and piano and string chamber works. With certain types of jazz music I might think a cheap system of similar quality. 

You say you are a strong supporter but yet you can pick and choose when its applicable

1 minute ago, MLXXX said:

It's interesting that promotions of low bitrate codecs often include the boom-tish genres of music. These can sound quite impressive even at a low bitrate. Classical music however will "suck" at a low bitrate.

That is a trust your ears statement.

 

Basically I don't think you can have it both ways. If you advocate a blind test as the means to verify audibility you cannot only invoke it when it suits you.

[frednork]

9 hours ago, proftournesol said:

Another hypothetical that I raised before, but nobody took up the question: If you auditioned a component that you clearly preferred in your system to your present component, yet it measured considerably worse than your existing component, would you buy it for your system?

It clearly sounds better, it isn't an ambiguous 'different sound'

you can afford it

No squibbing by saying that it couldn't sound better if it measured worse.

As long as there arent other issues such as aesthetic, absolutely.

[MLXXX]

1 hour ago, proftournesol said:

Hypothetical, take your pick: turntable, cartridge, CD player, amp, speaker, DAC

I think I'd be flat out hearing differences with turntables.

 

I think cartridges would be a good candidate for my own hearing. A mid-price cartridge with a more jagged published frequency response curve might sound better to me across a wide range of music than a much more expensive cartridge with a smoother looking published frequency response curve.

 

So to answer your question, " If you auditioned a component that you clearly preferred in your system to your present component, yet it measured considerably worse than your existing component, would you buy it for your system?", my answer would be yes.

 

I can imagine that happening in relation to a magnetic cartridge [if I were still into playing vinyl], and certainly in relation to speakers.

Edited August 5, 2019 by MLXXX

[MLXXX]

7 minutes ago, Audiophile Neuroscience said:

Basically I don't think you can have it both ways. If you advocate a blind test as the means to verify audibility you cannot only invoke it when it suits you.

You haven't actually responded to the matter I raised as to whether you were positing no audible difference at all between a $500 system and the main system in the listener's (presumably an audiophile) home, or whether you were merely positing that the $500 system was assessed by the listener as sounding the same as regards its overall impression of attractiveness or desirability, i.e. its "sound quality". You may recall I assumed you meant the latter.

 

Can you clarify which meaning you intended?

 

As for invoking it when it suits me, if you read guidelines for use of different protocols you will see that MUSHRA testing is used where sources include noticeably imperfect or impaired versions of sound and the object of the exercise is to get a subjective assessment of which types and degrees of impairment sound better or worse.  That type of testing was used in the development of the HE-AAC codec to determine how much more efficient it was than other codecs such as mp3, at low bitrates, in achieving a similar overall level of subjective sound quality.

 

On the other hand a DBT for whether there is "any audible difference at all" is appropriate for use when the sound sources being compared are very similar in their sound.

 

It would help for me to know whether you are talking in a MUSHRA context or an "any audible difference at all" context with your $500 example.

[Grant Slack]

40 minutes ago, MLXXX said:

You haven't actually responded to the matter I raised as to whether you were positing no audible difference at all between a $500 system and the main system in the listener's (presumably an audiophile) home, or whether you were merely positing that the $500 system was assessed by the listener as sounding the same as regards its overall impression of attractiveness or desirability, i.e. its "sound quality". You may recall I assumed you meant the latter.

 

Can you clarify which meaning you intended?

 

Hello there,

 

I hope you don't mind if I make a gentle suggestion here. David recently started a thread on exactly that topic (if, hypothetically, a $500 system sounded the same as your current system in a DBT taken by you, would you sell your hifi and get the $500 system), and his thread was promptly locked by moderators. The mod described it as "a flame-bait DBT thread, of which there are already a gazillion on the forums and we don't need them".

 

And now we see he is luring you (and anyone else here) into reopening exactly the same topic in this thread. It is a closed topic, and I agree, it is just flame-baiting.

 

So may I suggest that you let it lapse, from this thread, which is already under moderator watch. I get the impression that David first decides whether he likes the result of a test before he decides how well it is done, and the flame-bait $500 hypothetical is designed to make you do the same.

Regards,
Grant

[Audiophile Neuroscience]

54 minutes ago, MLXXX said:

You haven't actually responded to the matter I raised as to whether you were positing no audible difference at all between a $500 system and the main system in the listener's (presumably an audiophile) home, or whether you were merely positing that the $500 system was assessed by the listener as sounding the same as regards its overall impression of attractiveness or desirability, i.e. its "sound quality". You may recall I assumed you meant the latter.

 

Can you clarify which meaning you intended?

 

You can conduct the test anywhere, your choice. Its exclusively on SQ

54 minutes ago, MLXXX said:

As for invoking it when it suits me, if you read guidelines for use of different protocols you will see that MUSHRA testing

Not talking about preference testing in trained listeners or any other protocols other than the usual audio forum ABX methodology

54 minutes ago, MLXXX said:

 

On the other hand a DBT for whether there is "any audible difference at all" is appropriate for use when the sound sources being compared are very similar in their sound.

Not always. It has been touted in this very thread as making night and day differences disappear.  At any rate a blind ABX is the subject of this thought experiment.

 

54 minutes ago, MLXXX said:

It would help for me to know whether you are talking in a MUSHRA context or an "any audible difference at all" context with your $500 example.

Blind ABX for any audible difference. You can control whatever variables you see fit eg level matching but it must be within the scope of what most people can achieve.

[Audiophile Neuroscience]

1 hour ago, MLXXX said:

You haven't actually responded to the matter I raised

Oh, regarding Grant's comment. As you know I said there was no need to play. As you are aware I mentioned the previously posted thread was closed earlier in this thread. This thread is about blind testing (although I did question that early on and was assured it was on topic). There is a moderator participating in the thread who also put up a hypothetical.

 

Now all that said I am happy (once again) for you not to respond to the hypothetical.

 

[MLXXX]

1 minute ago, Audiophile Neuroscience said:

Blind ABX for any audible difference.

Well for me that definitely makes the hypothetical scenario very far removed from the realms of anything at all likely to occur with my own current level of hearing.  It would be very much at odds with my experience to date with the various DBTs I have undergone myself, involving quite subtle differences.

 

Anyway as @Grant Slack has suggested, probably not a good idea to pursue this particular tangent any further. I'll make this my final post on that hypothetical scenario.

[LHC]

12 hours ago, MLXXX said:

Ordinarily @LHC, I'd just stop reading at this point because whoever wrote the article appears to me at this stage to be well out of their depth.   Anyway, I'll keep reading. ...

I am very sorry, it was not intention to have you read the whole article. I was just trying to point you to the exchanges near the end of the comment section between Ethan and Mark. My apologies.

 

12 hours ago, MLXXX said:

That's an example of Ethan's openness, though in my opinion he was being very generous with his offer.

I don't think anyone would doubt his openness or eagerness. I just want contrast the attitude of Mark's willingness to test, and Ethan's lack of such willingness. 

 

12 hours ago, MLXXX said:

I'm afraid this really is an example of flogging a dead horse. And Ethan says so.

See, the problem you and Ethan have is making assumptions. If the problem with Mark's test is that the differences are all too easy to perceive, then sure, there were flawed tests in the past. If Mark's test is different then it is worth doing IMO. 

 

12 hours ago, MLXXX said:

Paul would not say that.

Another assumption. But if PS Audio sells any hi-res products, then it make sense for him to not talk them down.

 

12 hours ago, MLXXX said:

You will see that the comments petered out. No one reported being able to hear a difference.

Yet another assumption. The test result was reported in a separate blog page here: http://www.realhd-audio.com/?p=6274

 

The statistical analysis of the 83 reported responses were given by 'Mans' in the comment section underneath (dated August 2018): the results were not statistically conclusive, but a clear indication of some testers able to hear some difference was noted. 

 

"There are many ways to spin this. For instance, one might say that the number of people (12) getting at least 5 correct exceeded the expected value (6.8) by 76%. Clear proof that high-res is audible! However, if we look at how many got 3 or more correct, the number falls short of the expected by 6.6%. High-res is obviously not worthwhile! Statistics, the liar’s best friend."

 

If I am not mistaken Dr Crawford did participated in the test and was one of the high scorers. 

Edited August 6, 2019 by LHC

[proftournesol]

The reason why I raised my question is that, surely, we listen to systems, not components. This happens irrespective of whether we are listening blind or sighted, and our assessment is always an assessment of the complete system rather than a component. The best (or worst) we can say is that this component change sounds different/no different in this system to me. Of course, I am also a part of 'the system', as is the room. It may sound different to you because you are a different system to me. The same component in a different system will likely sound different. Measurement and A/B/X testing may be helpful in understanding some of the differences, but, at the end of the day it's really about musical enjoyment, a totally subjective experience.

[LHC]

@MLXXX

 

For your awareness (sorry if you are already informed) Mark Waldrep looks to set up a more rigorous version of his previous hi-res listening test here: http://www.realhd-audio.com/?p=6494 

 

I think this may be of interests to you. If you do participate I would be interested to follow your take on the experience. 

[Audiophile Neuroscience]

1 hour ago, proftournesol said:

The reason why I raised my question is that, surely, we listen to systems, not components. This happens irrespective of whether we are listening blind or sighted, and our assessment is always an assessment of the complete system rather than a component. The best (or worst) we can say is that this component change sounds different/no different in this system to me. Of course, I am also a part of 'the system', as is the room. It may sound different to you because you are a different system to me. The same component in a different system will likely sound different. Measurement and A/B/X testing may be helpful in understanding some of the differences, but, at the end of the day it's really about musical enjoyment, a totally subjective experience.

Yes I agree we are very much part of the system and we all perceive things differently.

 

The central nervous system has remarkable abilities to adapt and learn processing patterns and influence them from a top down perspective. This in turn can result in neuroplasticity  changes in the physical structure and hard wiring of the brain. This is well known in Pain Medicine but physical brain changes have also been reported for musicians.

 

It is important what we listen to, but arguably more important is the way that we hear.This is not the same as expectation bias. Once you get used to hearing things it becomes much easier to recognize the same things.

 

We know from neuropsychological testing that many higher order cognitive functions can fail under the stress of testing. It is in fact difficult at times to distinguish some injury or disease related brain impairments from from the effects of anxiety.

 

A little bit of anxiety can increase alertness and unlock memories and facilitate integrating multiple cognitive areas that make up perception. Too much stress shuts processes down. Many of us probably experienced memory freezes during exams.

 

What was easily remembered (including echoic memory) or perceived before can disappear when those many parts of the brain involved in the perception are blocked . you no longer get the same interaction going on.

 

Music is intimately related to our emotional centers and one of my theories about blind testing is that it totally distorts the emotional response that was interacting with auditory and other centers to create the experience. We are literally no longer having the same experience.

[sir sanders zingmore]

47 minutes ago, Audiophile Neuroscience said:

We know from neuropsychological testing that many higher order cognitive functions can fail under the stress of testing. It is in fact difficult at times to distinguish some injury or disease related brain impairments from from the effects of anxiety.

I keep coming back to this, apologies. 

It seems that you must conclude that no objective testing is possible. 

 

[Nigel]

IMO - the sound of speakers in room dominate what we hear. Besides that domination, commonly we may hear less or more bass, which I think relates to current and storage capability in the electronics(Source to speaker). Other than that, there are high frequency distortions that can be created anywhere from the recording studio to your speakers. Probably, the more electronic components, the increased likeliness of treble distortion, which seems the main reason why a vinyl sounds better than digital - less components. So when  Audio Science Review publishes say, Dac measurements,  the only useful (to me) finding is high frequency distortion / harshness/ sibilance and that's all. Everything else, like "warmth" and other descriptions from listening sessions (speakers in room) are too removed from those measurements to have good correlation.

[Ittaku]

10 minutes ago, Nigel said:

So when  Audio Science Review publishes say, Dac measurements,  the only useful (to me) finding is high frequency distortion / harshness/ sibilance and that's all.

Virtually all modern DACs only show measurements of those at levels allegedly far below the level of audibility according to ASR... Heck even all ancient DACs do too...

[Audiophile Neuroscience]

2 minutes ago, Sir Sanders Zingmore said:

I keep coming back to this, apologies. 

It seems that you must conclude that no objective testing is possible. 

 

I spent some time talking about how IMO probabilities and significance levels are objective tests. Subjective interpretation of tests and research results are something else again.

 

Neuropsychological testing for which I have been subject to as part of my training (fortunately not as a patient) and have post grad experience aims to be as objective as possible. Like all testing their are confounders and limits.

 

I have had my IQ tested now twice. I got fairly different results and was much better the second time. I mention IQ because it is a "brain" test that most people have heard of. There is endless debate over the validity of such a measure even in the general population.

[Assisi]

2 hours ago, proftournesol said:

The reason why I raised my question is that, surely, we listen to systems, not components. This happens irrespective of whether we are listening blind or sighted, and our assessment is always an assessment of the complete system rather than a component. The best (or worst) we can say is that this component change sounds different/no different in this system to me. Of course, I am also a part of 'the system', as is the room. It may sound different to you because you are a different system to me. The same component in a different system will likely sound different. Measurement and A/B/X testing may be helpful in understanding some of the differences, but, at the end of the day it's really about musical enjoyment, a totally subjective experience.

@proftournesol,

 

Thank you.  To me the above says all that needs to be said about this topic etc. Especially the last part of the last sentence.   .  The musical listening experience with our respective systems, rooms and brains will be different for each of us.  

 

Hence why started this thread recently.

https://www.stereo.net.au/forums/topic/288817-tuning-our-systems-%E2%80%93-the-elephant-in-the-room/?tab=comments#comment-4352182

 

John