changing Prob > A2 values in Anderson-Darling test

[#1]

When performing an Anderson-Darling test on a distribution, I´m getting strange results: I performed the analysis once (continuous distribution, fit normal, goodness of fit), then copied it to a script. Now when I rerun the script several times on the same data set, the Prob > A2 values change. They don´t change every time, but more often than not. No other values change, including the A2. Can you explain what is going on here and how to fix it? Many thanks in advance!

Anne

[#2 · Accepted solution]

This is expected behavior.  You can find the following in the help "For continuous fits, the goodness-of-fit test is the Anderson-Darling test. The p-value for the test is simulated using a parametric bootstrap, similar to the procedure described in Section 4.1 of Stephens (1974). For Normal distributions, the Shapiro-Wilk test for normality is also reported when the sample size is less than or equal to 2000 and there are no fixed parameters."

Since this value is simulated, it is not expected that the value will be the same for each run.

[#3]

Thanks a lot. So I´ll better use the Shapiro-Wilk W test to get consistent results.

[#4]

Thank you for this clear/documented response, it helped me one year later. In a recent experience (n=31) the p-value variations were far from negligible (changes at the second decimal place). It would be interesting to increase the number of Monte-Carlo simulation by default, would likely be unnoticeable in most cases. If I'm wrong, it could be an increase only when p-value estimate is below 0.1, or it could be an option within the continuous fit platform)