Gene - 17 September 2011 03:32 AM
How would we design the double blind?
Gene, it is a credit to your intellectual honesty that you’d even consider the issue. People are often a bit miffed at the implication that they don’t taste what is obvious to them, even when they know about the placebo effect, confirmation bias, and the post hoc reasoning error. Even scientists get bit by this dog, which is why they follow strict experimental protocols and have their work reviewed by peers for methodological errors. Personally, I’m always second-guessing myself about what I perceive, or think I perceive.
About a year ago, I noticed that my batch of dinner roll dough didn’t rise as quickly as I expected. I instantly “knew” that it was due the store brand of instant yeast that I had just started using. To test my hypothesis, I went and bought a jar of Fleischman’s yeast, and I made two identical batches of a basic bread dough, timing the mixer with a stop watch and placed the doughs into identical bowls, side-by-side. Sure enough, the different doughs…......rose at exactly the same rate. 
In my experiment, I controlled as carefully as I could for other variables that could affect the fermentation rate, but it still was sloppy compared to a real scientific experiment. I probably needed some objective way of measuring the volume change of the dough, rather than my subjective impression, and I probably should have had a thermometer buried in the dough to monitor dough temps. And a mere sample of two wouldn’t get my results in to the 95% confidence range. I also knew which dough was which.
If you were going to test a number of brands of yeast in a double-blind and statistically valid way, you’d probably need a couple of dozen loaves of bread, with the yeast measured and placed in numbered envelopes by someone other than you. It might be more practical to suggest this experiment to Cooks Illustrated. (However, I don’t think their testing methods are as rigorous as, say, Consumer Reports.)
Even if they were able to identify a taste difference, I’d still suggest it’d be the yeast food added to the dried yeast that would most likely account for the difference, rather than the yeast. Even with wild yeasts, like in sourdough, my understanding is that it’s the bacteria that account for the sourness, not the yeast. The only thing the wild yeasts contribute is the ability to thrive in an acidic environment.