In “Confidence in a Small Sample” you write “If the confidence intervals overlap, there is no statistically significant improvement.” This is a common misconception; you can in fact have overlapping confidence intervals and still have a statistically significant difference. To see why this is so, imagine that the intervals overlap just a bit. The probability of this happening if both means are actually the same is about 2 * (alpha/2) * (alpha/2) = alpha*alpha/2 — much smaller than your confidence level alpha. (The factor of two comes in because there are two possibilities for which of the sample means comes in low.)

The right way to tell if a difference is statistically significant is compute the confidence interval for the DIFFERENCE in the means, and check that it doesn’t overlap 0. Here’s one web page that describes how to compute that confidence interval:

]]>I’m sorry, but I don’t plan to do a Kindle version. I’m happily retired and enjoying the other fine things in life. Take care. ]]>

Is there a Kindle version planned? I’d love to read TECPB on mine. That’d be awesome!

Thank you very much for sharing your knowledge and experience!

Olivier

]]>Will be in touch. Thank you Bob.

Thanks,

Srinivas

It would be awesome if I could send my copy to you for an autograph.

]]>cheers,

Uli Schäfer

Technical Project Manager