#18 p-value calculation bug?

[Anonymous]

Hi,

I'm using jdistlib to test for normalitity. I use the following functions to test:
NormalityTest.shapiro_wilk_statistic and NormalityTest.shapiro_wilk_pvalue
NormalityTest.kolmogorov_smirnov_statistic and NormalityTest.kolmogorov_smirnov_pvalue
NormalityTest.cramer_vonmises_statistic and NormalityTest.cramer_vonmises_statistic

While all _statistic methods return the correct static value (in comparison to SAS), all p-values are wrong (in comparison to SAS, R, and other online tools e.g. http://contchart.com/goodness-of-fit.aspx)

Please enter the data in the file into http://contchart.com/goodness-of-fit.aspx or SAS, and see that jdistlib delievers totally different p-values.

[Roby Joehanes]

Thank you for your report. I am currently investigating this issue.
FYI, the current results given by JDistlib (v0.3.5) is as follows:

Shapiro-Wilk: 0.9837584709338049, p=6.629141861303468E-33
Kolmogorov-Smirnov: 0.03747563698863843, p=0.28158252412529505
Cramer-Von Mises: 0.20110593140953217, p=0.005192748364850318

So, from your online tools, only KS test is at fault (it seems). But I will get to it and double check all the affected functions. Hopefully I can fix it in the next few days. This is one of a few areas where there is little to no test codes in the R side.

Again, thank you for your report.

[Roby Joehanes]

Okay. I found the "problem". In JDistlib, the values in KS test are Z-transformed first (i.e., (x-mean(x))/sd(x)), whereas in R or other statistical packages they are not. You will notice that the following R statement will return the same statistic and p-value:

std <- function(x) (x - mean(x))/sd(x);
ks.test(std(x), "pnorm");

With that in mind, I changed the behavior of JDistlib's KS test in the next version (0.3.6). Even more, I added a generic KS test that can handle other distributions (e.g., Beta, Gamma, etc.) and I added lesser and greater tests (in addition to the two-sided tests).

Hope this helps. I'm sorry that it takes me too long to fix this bug.

[Roby Joehanes]

The bug fix I made for version 0.3.6 was erroneous. Please download version 0.3.8 or above.