Saturday, August 3, 2013

CI Selection In Statistical Hypothesis Testing

A GB shows me her result of Hypothesis testing as below.
Based on her 85% CI, she conclude that P-value < 0.15, therefore it shows significant different between two group of sample data.

When I ask her, why she choose 85% as her CI, answer given to me was, if she choose the default value of 95% CI, she is not able to get a conclusion that shows significant different.


Two-Sample T-Test and CI: C1, C2

Two-sample T for C1 vs C2

     N    Mean   StDev  SE Mean
C1  10  5.0086  0.0680    0.022
C2  10  4.9627  0.0449    0.014

Difference = mu (C1) - mu (C2)
Estimate for difference:  0.045879
85% CI for difference:  (-0.009077, 0.100834)
T-Test of difference = 0 (vs not =): T-Value = 1.78  P-Value = 0.095  DF = 15

My question is : Is this the right way to manipulate result by playing around the CI ?
My personal opinion, ethically this is not a right way to manipulate the test conclusion by playing around the CI, and she set the CI beyond the practical limit of 90% ! What is the objective of doing hypothesis testing if we can manipulating the result by changing the CI ? might as well don't do !

In my past experience, I do face many cases where test result shows no different or marginal different, P-value > 0.05. I never advice my GB or BB to adjust the CI for the sake of getting a result of significant different, this is meaningless to me. 

However, we admit the fact, the most, we ask our self, are we still want to implement the solution that shows no or minor different ? what are the benefit we can gain ? if the answer is YES and we can justify our self to proceed, then we go ahead and do it. (handle it with special case)

As a professional Six Sigma practitioner, we should aware Six Sigma is a problem solving methodology that based on Data and Fact, we shouldn't abuse the Fact to mislead the people who do not aware about the result.