Expectation Bias

[March Audio]

7 minutes ago, Keith_W said:

So is there a middle ground? I believe so. If there are sound theoretical reasons why there should be a difference (and not some made up explanation with an at best tenuous link),

Problem is that much of the time the explanations provided for an alleged sonic effect are tenuous.

 

Jitter caused by Ethernet switches, which I examined in another thread, is an example.  Different switches didn't make one jot of difference to the jitter levels of the dacs.

[March Audio]

11 minutes ago, Keith_W said:

if there are electrically measurable differences at output, if there are sonically measurable differences by microphone, then even a poorly conducted listening test of either type might have persuasive power.

 The problem is that doesn't happen. Person X plugs in device A which is claimed to be beneficial to sound quality.

 

They hear an improvement.  It could be the case, but equally it may not.  And that's the point, no one else can be sure of the veracity of the claim.  Often person X then gets upset because his experience is being questioned.

[Keith_W]

3 minutes ago, March Audio said:

Well, some choose to ignore the reality of scientific evidence, just as they do on any subject.

 

I don't know how many scientific journals you read, but there is a large difference between what is reported in the journal, and what is reported in the popular press. Even the science press like New Scientist. As mentioned previously, I have training in my field and I know what the evidence actually says or does not say. Authors of such papers want to present their experiment in the best light possible so all sorts of biases creep into their conclusions. In some other disciplines, the entire academic field might be afflicted by groupthink (especially prevalent in the social sciences) which prevents dissenting papers from being published. 

I do not have the skills to analyse papers from other disciplines, such as geology, climate change, astronomy, physics, and so on. Yet I continue to follow subjects which interest me as a layperson. In such cases I know full well that I am reading the popular press which was written in such a way that these esoteric subjects are accessible to a layman, and which lack robust scientific analysis and debate. Reef scientists have said that the Barrier Reef is "dead" and have written obituaries for the dead reef for 30 years or more now. Al Gore, based on predictions of climate scientists at the time, warned that the polar ice caps would be completely melted by 2013-15. Then, there are multiple people who deny anthropogenic climate change with their own misinterpretations of scientific studies. This is not to say that we should ignore those warnings. As a climate change layman myself, and lacking the skills to properly interpret academic publications, I am forced to rely on scientific journalists (which usually have an axe to grind either for or against) for my information. I am fully aware that my understanding is incomplete so I refrain from making strong statements for and against either way. 

[vivianbl]

12 minutes ago, March Audio said:

Well, some choose to ignore the reality of scientific evidence, just as they do on any subject.

That’s a weasel way of argument to dismiss anyone who might disagree with what you are postulating . “Scientific evidence” can be disputed.

This does not mean the those who dispute what you claim as “scientific evidence” are choosing “to ignoring scientific evidence”. 

There are lots of methodological issues to overcome in the Hearing/ listening/ perception areas  to draw definitive conclusions. The testing situation itself is a confounding variable.

[March Audio]

1 minute ago, vivianbl said:

That’s a weasel way of argument to dismiss anyone who might disagree with what you are postulating . “Scientific evidence” can be disputed.

This does not mean the those who dispute what you claim as “scientific evidence” are choosing “to ignoring scientific evidence”. 

There are lots of methodological issues to overcome in the Hearing/ listening/ perception areas  to draw definitive conclusions. The testing situation itself is a confounding variable.

Nothing weasel about it.

 

Yes scientific evidence can be disputed, that's a fundamental part of the scientific process.

 

So, please go ahead and scientifically dispute the concept that people are affected by cognitive biases.

[vivianbl]

8 minutes ago, March Audio said:

Nothing weasel about it.

 

Yes scientific evidence can be disputed, that's a fundamental part of the scientific process.

 

So, please go ahead and scientifically dispute the concept that people are affected by cognitive biases.

I’m sorry, but why do I need to do that?

[March Audio]

35 minutes ago, vivianbl said:

I’m sorry, but why do I need to do that?

 Because you are the one who seem to be arguing that there is something to be disputed about the concept that people are affected by cognitive bias.

 

Can I take it in that case that you don't feel there is anything to be disputed about that assertion?

Edited March 3, 2023 by March Audio

[muon*]

15 minutes ago, March Audio said:

 Because you are the one arguing that there is something to be disputed about the concept that people are affected by cognitive bias.

An you are the one arguing the opposite.

 

I gave you one example of a very big confounding variable, in that some are effected strongly and some are effected very little, and the population would inhabit every level between.

 

Edit: actually I'm wrong in the above, It's not confounding, It's a variable. It's only confounding for someone who tries to posit that everyone is effected equally.

Edited March 3, 2023 by muon*

[March Audio]

11 minutes ago, muon* said:

An you are the one arguing the opposite.

 

I gave you one example of a very big confounding variable, in that some are effected strongly and some are effected very little, and the population would inhabit every level between.

I am.  What's your point?

 

That wasn't an example of anything. You haven't even provided the overall outcome of the study.  How do we know if the two individuals were outliers or not?

 

Can you support that claim with some evidence?

 

 

Edited March 3, 2023 by March Audio

[vivianbl]

27 minutes ago, March Audio said:

 Because you are the one who seem to be arguing that there is something to be disputed about the concept that people are affected by cognitive bias.

 

Can I take it in that case that you don't feel there is anything to be disputed about that assertion?

I’m afraid you are going all over the place. I’d suggest you read what I’ve posted before and then workout where I’ve said what you claim I’ve said.

 

[muon*]

OK, Alan. you win at the internet

 

BTW, not everyone that questions some of your assertions questions everything, if my budget allowed I'd be a subscriber to Skeptics Society, big fan of Michael Shermer.

Edited March 3, 2023 by muon*

[MLXXX]

2 hours ago, Keith_W said:

In the audio DBT setting, it might include: recruitment of inadequately trained listeners, recruitment of listeners with an expectation bias towards the null hypothesis, no control over subjects (e.g. have you recruited people with hearing loss?), poorly controlled experiment (e.g. too much lead time between A and B, or loudspeakers in ABX tests positioned differently in the same room), and so on. 

 

A broad study of the general population could be aimed at deriving mean limens of human hearing acuity at different frequencies and intensity levels for sinusoidal test tones, as a function of age and gender (for subjects with healthy hearing).  

 

However the situations that arise for discussion in audiophile forums are rarely if ever of such a broad nature.

 

A typical claim to arise in this forum would be that there is a distinctly audible  (even "night and day") difference from changing part of a hi-fi system. The claim is made without the support of measurement data, or any attempt (even informal) to blind test.

 

Some changes to a hi-fi system can be made quickly (e.g. changing a connecting lead) and would lend themselves to A B X comparisons with minimal delay.   A topical matter over the past couple of years has been the effect of changing Ethernet switches or Ethernet cables for listening to streamed audio.   To test a claim, there would no need to cast about for a cohort of test subjects as one might for a comprehensive medical study. One could simply ask a person who has reported hearing a night and day audible difference to repeat the feat when listening under blind A B X conditions.  They could use their own audio system, their own listening room, and their own listening chair.  A neutral party would change the connections.  That person could be a friend (audiophile or non-audiophile).

 

On the other hand, some system changes can take a long time to carry out, such as changing the room treatment, and ABX testing may not be feasible in a domestic setting.  (However, the particular case of changing the room treatment normally has a pretty obvious effect and there tends to be less call for blind testing anyway!  It's not a matter of establishing whether there's been an audible change. It's more a matter of evaluating whether the change has been beneficial.) 

 

 

Edited March 3, 2023 by MLXXX

[March Audio]

13 minutes ago, vivianbl said:

I’m afraid you are going all over the place. I’d suggest you read what I’ve posted before and then workout where I’ve said what you claim I’ve said.

 

How so?

 

This is a discussion about expectation (cognitive) bias.  You were arguing that the science that backs that up was some how debatable.  I invited you to explain why.

Edited March 3, 2023 by March Audio

[March Audio]

3 minutes ago, muon* said:

OK, Alan. you win at the internet

 

BTW, not everyone that questions some of your assertions questions everything, if my budget allowed I'd be a subscriber to Skeptics Society, big fan of Michael Shermer. So...was that an example of your personal biases?

It's not case of winning.  It's just rational debate.

 

Sorry but I really don't understand the point you are making.

[vivianbl]

13 minutes ago, March Audio said:

How so?

 

This is a discussion about expectation (cognitive) bias.  You were arguing against  the science that backs that up was some how debatable.  I invited you to explain why.

So you haven’t read what I’ve said.

[March Audio]

53 minutes ago, vivianbl said:

So you haven’t read what I’ve said.

Of course I didnt, and you didn't put laughing emojis against most of my posts either. BTW why is there no face palm emotion?

 

So, just in case I misunderstand, would you agree that individuals can be influenced by things such as brand, price, aesthetics, what the reviewer said, a specious technical justification etc?

Edited March 3, 2023 by March Audio

[MLXXX]

2 hours ago, vivianbl said:

There are lots of methodological issues to overcome in the Hearing/ listening/ perception areas  to draw definitive conclusions. The testing situation itself is a confounding variable.

 

I've seen the issue that I've highlighted in bold above, put forward on this forum in the following sorts of ways: 

• I can't hear the difference unless I am listening naturally in a relaxed state, not listening for differences
• If I know I'm being assessed, I get nervous and can't hear properly.

 

Those sorts of explanations are a bit hard to reconcile with previous claims of hearing "night and day differences". They suggest that the differences were not as obvious as the test subject claimed they were, i.e. not "night and day", or that the test subject is of a particularly nervous disposition.

 

As I've mentioned earlier in this thread, I find a rapid ABX test makes it easier to hear differences, not harder, but that's just me I suppose.  I'd be flat out hearing artefacts in 256kbps AAC if I did not have an uncompressed version of the same music recording to compare with, without delay.

 

Interestingly, the claims of night and day differences can be made even when there is a significant delay between hearing one version and the next, e.g. "My system performs much better late at night than it does during the day. I  put that down to less interference from solar panel inverters in my street at night, though I'm worried that growing use of solar panel batteries could eventually disturb my nighttime listening.  I am considering getting a power regenerator.".

 

Edit: A solution I have repeatedly suggested to get around delays is that of recording the output of the system under test under conditions "A" ad "B", and using those recordings for careful ABX comparisons. 

Edited March 3, 2023 by MLXXX

[rantan]

9 minutes ago, March Audio said:

BTW why is there no face palm emotion?

You should use this one

[vivianbl]

8 minutes ago, MLXXX said:

I've seen the issue that I've highlighted in bold above, put forward on this forum in the following sorts of ways: 

• I can't hear the difference unless I am listening naturally in a relaxed state, not listening for differences
• If I know I'm being assessed, I get nervous and can't hear properly

 

Those sorts of explanations are a bit hard to reconcile with previous claims of hearing "night and day differences". They suggest that the differences were not as obvious as the test subject claimed they were were, i.e. not "night and day", or that the test subject is of a particularly nervous disposition.

There may be situational and  behavioural characteristics of individuals, but the other factor is how the brain adapts to the process of testing.
Unlike auditory testing for frequency/ vol differences, It is quite clear that the brain looks for markers in a piece of music to help differentiate between the testing items in question.  I’ve taken part in many blind tests ( in my younger years) and I realised that the markers were not necessarily the ones that gave you the best assessment of the music that was presented. I also found that “no difference” was the most common finding in those sessions!

The hearing/ listening process is very complex because the brain plays an active process. Hence, my comment about the testing “situation/ structure” being a variable that also needs to be taken into account

 

24 minutes ago, MLXXX said:

Interestingly, the claims of night and day differences can be made even when there is a significant delay between hearing one version and the next, e.g. "My system performs much better late at night than it does during the day. I  put that down to less interference from solar panel inverters in my street at night, though I'm worried that growing use of solar panel batteries could eventually disturb my nighttime listening.  I am considering getting a power regenerator.".

Yes, in my situation it was literally a” night and day” difference that I ended up getting a regenerator. It was only after that I realised that incoming voltage was ranging from 226v to 262v! 
When I got my solar panels installed, they picked up that the incoming voltage was well over the threshold and solar system was cutting out. Ausnet crew arrived in quick time to test and adjust the transformers on the pole!

[vivianbl]

53 minutes ago, March Audio said:

Of course I didnt, and you didn't put laughing emojis against most of my posts either. BTW why is there no face palm emotion?

I knew you didn’t read what I had posted!

Didn’t put face palm motion, because what you said made me laugh!

 

53 minutes ago, March Audio said:

So, just in case I misunderstand, would you agree that individuals can be influenced by things such as brand, price, aesthetics, what the reviewer said, a specious technical justification etc?

Yes, lots of factors influence human thinking and behaviour.

Edited March 3, 2023 by vivianbl

[MLXXX]

33 minutes ago, vivianbl said:

The hearing/ listening process is very complex because the brain plays an active process. Hence, my comment about the testing “situation/ structure” being a variable that also needs to be taken into account

Yes, very true, for example in designing protocols for and evaluating results of, systematic testing of a group of listeners (of varying experience levels ) to recorded music.

 

However on this forum we commonly see claims that for a particular enthusiast, all music played on their system with change X in place is noticeably improved. Two phrases often seen are "wider sound stage". and "lower noise floor".  These changes are not expressed as cropping up sporadically but as being ongoing, clearly audible,  differences.  As such, one would not expect it to be especially challenging for the enthusiast to hear some difference, even under sub-optimum blind testing conditions.  However the more usual scenario is a point blank refusal to take part in a blind test.

 

 

50 minutes ago, vivianbl said:

Yes, in my situation it was literally a” night and day” difference that I ended up getting a regenerator. It was only after that I realised that incoming voltage was ranging from 226v to 262v! 
When I got my solar panels installed, they picked up that the incoming voltage was well over the threshold and solar system was cutting out. Ausnet crew arrived in quick time to test and adjust the transformers on the pole!

 

That's a coincidence! I drew my example of suspected solar inverter interference at random, without realizing that you yourself had had involvement with mains issues, or a regenerator.

 

 

[muon*]

2 hours ago, March Audio said:

It's not case of winning.  It's just rational debate.

 

Sorry but I really don't understand the point you are making.

Rational you say

Let me walk you through it step by step.

 

1/ Yes, biases are real, and everyone potentially can be subject to their influences.

2/ Not everyone is effected to the same degree, some are strongly effected, some very little, some less so, some more so.

 

It's not black and white like you seem to be postulating, we are not clones.

[andyr]

2 hours ago, MLXXX said:

Interestingly, the claims of night and day differences can be made even when there is a significant delay between hearing one version and the next, e.g. "My system performs much better late at night than it does during the day. I  put that down to less interference from solar panel inverters in my street at night, though I'm worried that growing use of solar panel batteries could eventually disturb my nighttime listening.  I am considering getting a power regenerator.".

 

Aah, so not only does the use of solar panels to generate electricity cost heaps - and requires the domestic power distribution network to be re-engineered - it also requires people with hifi systems who love listening to music ... to buy a regenerator?  

 

Thank you, Greta!  

 

[Satanica]

4 hours ago, muon* said:

Rational you say

Let me walk you through it step by step.

 

1/ Yes, biases are real, and everyone potentially can be subject to their influences.

2/ Not everyone is effected to the same degree, some are strongly effected, some very little, some less so, some more so.

 

It's not black and white like you seem to be postulating, we are not clones.

That's great, a good explanation of why controlled testing exists. Good to see that you've (slowly) come around, welcome aboard. Of course we can agree to disagree about the "some very little" bit, but that's just a minor point of non-full agreement. I don't actually know if some audiophiles are only inflicted "very little" by expectation bias but I can't see how that it really matters either way, especially if an individual can't prove it on an ongoing basis. So, we mostly agree and that's how the truth seems to work sometimes.

Edited March 3, 2023 by Satanica

[Zed Zed]

10 hours ago, rocky500 said:

I see that a lot of people believe heavily doing a Blind test is the only way to really find out.

 

I have come to think that is one of the biggest confirmation bias's in audio. Thrown in with biggest Expectation bias.

 

 

When the test is properly conducted, there can be no bias since the audience has no idea what is being tested. ABX tests are the gold standard in many industries (eg pharms) for eliminating bias, placebo effect etc. The researcher plays test A and then test B and asks the audience which they heard, A or B. In a proper double-blind test, the researcher has no idea if it is A or B either. This is to prevent subconscious influencing.  Sometimes, A will be played more than once in a row, as will B. It is random. Afterwards, the audience's results are tabulated, Often people will hear a difference between A and B when in fact A and A were played! Generally, the results are 50-50 with no clear indication that the audience heard any differences at all.