muon* Posted March 3, 2023 Posted March 3, 2023 Just a point regarding these things and how they effect different individuals at different levels. This is regarding placebo effect, and in this situation confirmation bias. There was a myth busters episode that showed the differences in how placebo effects different individuals differently, this was not the aim but an interesting out come. They were testing motion sickness remedies, and one of the guys reacted well to a placebo with it being almost as effective as the real pharmaceutical remedy, while the other was pretty much unaffected and reacted the same as if he had no medication at all.
vivianbl Posted March 3, 2023 Posted March 3, 2023 36 minutes ago, muon* said: Lot of bias in this thread Yes, expectations have been confirmed. 1
MLXXX Posted March 3, 2023 Posted March 3, 2023 (edited) 2 hours ago, rocky500 said: In my years of trying them, I seem to come to realization that they tend to mask the differences (smaller ones) I find the reverse. Differences I don't notice in everyday listening I may be able to identify in an ABX test that allows immediate comparisons. For example, AAC stereo at 256kbps normally seems fine to me, but if there is an ABX test allowing immediate comparison between the original file and the AAC 256kbps version there's a good chance I'l hear some subtle differences. PAL Speedup I may sometimes fail to notice PAL speedup on an American sitcom or a movie, shot at 24fps and broadcast in Australia at 25fps. However if there is a comparison available between a PAL DVD of a movie (at 25fps) and a Blu-ray version (at 24fps, or 23.976fps to be precise) the difference in pitch and tempo becomes unmistakeable. Actually the PAL speedup effect is a more than subtle difference and these days when so much video is available at correct speed I normally hear it without any need for a comparison. However I believe that a lot of people have never noticed PAL speedup when watching Australian TV. They would have needed an immediate A B comparison before they'd have noticed any oddness in the sound. So, in my view, a blind comparison that permits rapid A B comparisons would make differences easier to hear than in a lot of real life hi-fi situations where a change in a system setup can take a while to carry out. Edited March 3, 2023 by MLXXX
March Audio Posted March 3, 2023 Posted March 3, 2023 (edited) 1 hour ago, rocky500 said: I see that a lot of people believe heavily doing a Blind test is the only way to really find out. I have come to think that is one of the biggest confirmation bias's in audio. Thrown in with biggest Expectation bias. No, its just standard and *proven* scientific practice. Audio *isnt* a special case. Edited March 3, 2023 by March Audio 1
March Audio Posted March 3, 2023 Posted March 3, 2023 1 hour ago, rocky500 said: In my years of trying them, I seem to come to realization that they tend to mask the differences (smaller ones) when ever I have tried them myself. It seems the smaller ones are the contentious ones online or why you might want to do these tests in the first place. A realisation based on what? 1 hour ago, rocky500 said: One I was just reading recently is in the ability of the brain to fill in gaps in audio tests. This is just one part that could effect the tests and there seems to be lots. This is why you have *controls* built into tests. Known identical and known different.
March Audio Posted March 3, 2023 Posted March 3, 2023 1 hour ago, rocky500 said: Some links here as I just googled. There seems to be quite a bit out there already. Some examples I just googled below. There are many out there. https://www.mentalfloss.com/article/550337/why-your-brain-sometimes-hears-sounds-dont-exist https://www.sciencedaily.com/releases/2009/11/091125134655.htm https://theproaudiofiles.com/music-and-the-brain/ Seems like an equal argument to not trust uncontrolled sighted tests. Are you filling in blanks there also? Put controls in place and you know for sure. 1 1
March Audio Posted March 3, 2023 Posted March 3, 2023 (edited) 51 minutes ago, muon* said: They were testing motion sickness remedies, and one of the guys reacted well to a placebo with it being almost as effective as the real pharmaceutical remedy, while the other was pretty much unaffected and reacted the same as if he had no medication at all. Again this is just an argument for having controls. Should we be basing any conclusion on the result of two individuals in your example? What was the overall result of the study? Edited March 3, 2023 by March Audio
rocky500 Posted March 3, 2023 Posted March 3, 2023 19 minutes ago, March Audio said: No, its just standard and *proven* scientific practice. Audio isnt a special case. Thats the thing. Have you looked how these tests are done in other industries to be effective, The effort put in and time and expense. The scrutiny, etc... What do you think if people where told you can do them yourself at home for these industries as it very simple to do. What do you think people will think of the results then? Will the results be close to valid? 1
March Audio Posted March 3, 2023 Posted March 3, 2023 (edited) 12 minutes ago, rocky500 said: Thats the thing. Have you looked how these tests are done in other industries to be effective, The effort put in and time and expense. The scrutiny, etc... What do you think if people where told you can do them yourself at home for these industries as it very simple to do. What do you think people will think of the results then? Will the results be close to valid? It isnt rocket science to take the most significant biasing factors out of the equation. ie, not knowing what is playing and to accurately match volumes. Doing that is far more likely to give a valid result than sighted. Anyway, I think this somewhat misses the point. None of this is saying people must go out and do rigorously controlled blind listening tests. Its just to try and get people to understand just how susceptible people are to biasing factors. To treat their sighted conclusions with extreme caution. Edited March 3, 2023 by March Audio 3
afa Posted March 3, 2023 Posted March 3, 2023 Here is a site with lots of interesting information about hearing, though, their are a few dead links in some of the chapters. https://www.whyyouhearwhatyouhear.com/
muon* Posted March 3, 2023 Posted March 3, 2023 1 hour ago, March Audio said: Again this is just an argument for having controls. Should we be basing any conclusion on the result of two individuals in your example? What was the overall result of the study? The fact is that it confirms that individuals are not all swayed to the same level by these things. 1
Satanica Posted March 3, 2023 Posted March 3, 2023 1 minute ago, muon* said: The fact is that it confirms that individuals are not all swayed to the same level by these things. I think everyone here agrees that is most likely a fact, but it's no recent discovery and it makes no difference. 1
March Audio Posted March 3, 2023 Posted March 3, 2023 (edited) 21 minutes ago, muon* said: The fact is that it confirms that individuals are not all swayed to the same level by these things. Which is why you use controls. Any study will consist of more than 2 individuals. You can't conclude anything from looking at this example. Edited March 3, 2023 by March Audio
muon* Posted March 3, 2023 Posted March 3, 2023 14 minutes ago, Satanica said: I think everyone here agrees that is most likely a fact, but it's no recent discovery and it makes no difference. Just that some people seem to think these things strongly effect everyone. Glad you are not one of them
vivianbl Posted March 3, 2023 Posted March 3, 2023 2 hours ago, March Audio said: Seems like an equal argument to not trust uncontrolled sighted tests. Are you filling in blanks there also? Put controls in place and you know for sure. Oh yes, we need to have blind faith in blind test and controls.
muon* Posted March 3, 2023 Posted March 3, 2023 49 minutes ago, March Audio said: Which is why you use controls. Any study will consist of more than 2 individuals. You can't conclude anything from looking at this example. I can, they can, they did! I don't bother with blind testing as a hobbyist, if I were developing audio gear I probably would at times, but not as a hobbyist.
Satanica Posted March 3, 2023 Posted March 3, 2023 1 hour ago, muon* said: Just that some people seem to think these things strongly effect everyone. Glad you are not one of them I never wrote that they don't strongly effect everyone. I agreed to: "The fact is that it confirms that individuals are not all swayed to the same level by these things." And that's not quite the same thing.
The Rock Puppy Posted March 3, 2023 Posted March 3, 2023 16 hours ago, andyr said: Port Stephens is somewhere near Gosford right? Not really. Probably two hours or so by car. 1
March Audio Posted March 3, 2023 Posted March 3, 2023 1 hour ago, muon* said: Just that some people seem to think these things strongly effect everyone. Glad you are not one of them .... and some are quite convinced they are not affected when they are. Glad you are not one of them 1
March Audio Posted March 3, 2023 Posted March 3, 2023 (edited) 1 hour ago, vivianbl said: Oh yes, we need to have blind faith in blind test and controls. Erm...... science takes away the problem of erroneous faith based beliefs......... thats the point Edited March 3, 2023 by March Audio 1
March Audio Posted March 3, 2023 Posted March 3, 2023 57 minutes ago, muon* said: I can, they can, they did! I don't bother with blind testing as a hobbyist, if I were developing audio gear I probably would at times, but not as a hobbyist. So what was the result of the study beyond these two subjects?
vivianbl Posted March 3, 2023 Posted March 3, 2023 12 minutes ago, March Audio said: Erm...... science takes away the problem of erroneous faith based beliefs......... thats the point But do they in hearing/perception based issues?
Keith_W Posted March 3, 2023 Posted March 3, 2023 (edited) We are back to blind tests again This is Keith_W's Law: any audio debate eventually degenerates to an argument about blind tests. As a scientist myself with many years experience of reading journal articles, it is well known that: - A double blind test that shows the null hypothesis (no difference) is easy to achieve, and - A sighted test that shows a difference is easy to achieve. In the medical community we call these placebo controlled double blind tests and open label tests. In the former, a cohort of patients are split into two or more arms. One arm is given placebo (or more commonly, standard treatment + placebo), and the other arm is given standard treatment + placebo. The experimenters and the patients are usually blinded as to which arm they are in. In this case, there is a strong bias towards the null hypothesis, especially if the effect is subtle and the experiment is not done properly - for example, insufficient case-controls recruited, both arms not matched, and so on. In the audio DBT setting, it might include: recruitment of inadequately trained listeners, recruitment of listeners with an expectation bias towards the null hypothesis, no control over subjects (e.g. have you recruited people with hearing loss?), poorly controlled experiment (e.g. too much lead time between A and B, or loudspeakers in ABX tests positioned differently in the same room), and so on. In open label tests there is a strong bias towards a positive result. For medics, sometimes there is no choice but to do an open label test, especially with fields like giving chemotherapy. The nurse might be given a mystery bag of fluid to administer to the patient, but it is usually really obvious which bag contains the drug and which one contains the placebo (i.e. normal saline). Even with objective tests like measuring the level of tumour markers, or CT's showing the size of the tumour, or survival rate, there is still a strong bias towards a positive result - especially in subjective metrics such as quality of life, pain, weight loss, and so on. In the audio setting, as mentioned multiple times in this thread, sighted listening opens the door for the McGurk effect and expectation bias to return a positive result, so the persuasive power of a positive sighted test is zero to minimal. I dismiss any such claim immediately and place zero weight on them. Even if I do hear a positive result myself, I know that my assertions should have zero persuasive power and I expect others who are scientifically minded to do the same. (EDIT) the corollary of the above is: a blind test that shows a positive result is more likely to be true than a blind test that shows a negative result. However, the same can not be said about sighted tests - a sighted test that returns a null result is not more likely to be true than a sighted test that shows a positive result. So is there a middle ground? I believe so. If there are sound theoretical reasons why there should be a difference (and not some made up explanation with an at best tenuous link), if there are electrically measurable differences at output, if there are sonically measurable differences by microphone, then even a poorly conducted listening test of either type might have persuasive power. Edited March 3, 2023 by Keith_W 2
March Audio Posted March 3, 2023 Posted March 3, 2023 (edited) 3 hours ago, vivianbl said: But do they in hearing/perception based issues? Yes. Audio is not a special case. However, some choose to ignore the reality of scientific evidence, just as they do on any subject. Edited March 3, 2023 by March Audio 1
March Audio Posted March 3, 2023 Posted March 3, 2023 2 minutes ago, Keith_W said: We are back to blind tests again This is Keith_W's Law: any audio debate eventually degenerates to an argument about blind tests Unfortunately there are some that don't and wont accept the frailty of human perception. It's all proven fact, and frankly common sense, yet they still argue. 1
Recommended Posts