Goodness of Fit

[#1]

Dear JMP users,

I have a question regarding the usage of the P-value to check if a dataset fit with a distribution.

For some distribution (norma, lognormal...) in JMP is it possible to check the p-value with the "Goodness of fit". P-value<0.05 the data do not mach the distribtuion selected, if p-value>0.05 the data match the distribution selected.

I tryed to use this function with the "Normal 2 Mixture"... but it in not possible to apply the "Goodness of fit" from the JMP menu.

Some of you have some explanation for this?

Thanks in advance for your feedback.

Best Regards,

Simone

[#2 · Accepted solution]

Many distribution models have a hypothesis test for goodness of fit. The hypothesis test is based on the null hypothesis (that is, the one that is assumed to be true) that the observed data sample is in fact from a population with such a distribution. The alternative is that the population does not have such a distribution. One or more statistics have been developed for some distributions to represent the null hypothesis. The sampling distribution of these statistics is used to compute a p-value. Some statistics cover the p-value better than others. Not all distributions have a statistic, though. Also, a mixture model for a distribution does not have a statistic to test the null hypothesis.

(When such a test is available, I would caution you against always using alpha = 0.05 for every decision about significance.)

In your case, I suggest that you use an information criterion instead of a sample statistic. AICc or BIC may be used for this purpose. The best model based on the criterion alone is the one with the minimum criterion value. This is not a test, though. There is no p-value. There is no such decision as 'significant' or 'not significant.' There is only selection of one model over the other candidate models.

[#3]

Dear Mark,

thank you for the feedback and details.

I will use it as guideline in this distribution analysis.

For my understandig: the p-value is a mathematical calculation... so why some distribution can "generate" the p-valule and other can not generate this p-value calculation?

Thanks again for your feedback.

Best Regards,

Simone

[#4 · Accepted solution]

The p-value is calculated from the sampling distribution of the test statistic for the model under the null hypothesis. For example, there are several test statistics for the goodness of fit to a normal distribution model. The Shapiro-Wilk W statistic or the Anderson-Darling A2 statistic. The test statistic is not available for all models.

[#5]

Thank you Mark.

Clear explanation.

Have you a good day.

Best Regards,

Simone