Share your ideas for the JMP Scripting Unsession at Discovery Summit by September 17th. We hope to see you there!
Choose Language Hide Translation Bar
Highlighted
Level III

## Goodness of Fit

Dear JMP users,

I have a question regarding the usage of the P-value to check if a dataset fit with a distribution.

For some distribution (norma, lognormal...) in JMP is it possible to check the p-value with the "Goodness of fit". P-value<0.05 the data do not mach the distribtuion selected, if p-value>0.05 the data match the distribution selected.

I tryed to use this function with the "Normal 2 Mixture"... but it in not possible to apply the "Goodness of fit" from the JMP menu.

Some of you have some explanation for this?

Best Regards,

Simone

2 ACCEPTED SOLUTIONS

Accepted Solutions
Highlighted
Staff

## Re: Goodness of Fit

Many distribution models have a hypothesis test for goodness of fit. The hypothesis test is based on the null hypothesis (that is, the one that is assumed to be true) that the observed data sample is in fact from a population with such a distribution. The alternative is that the population does not have such a distribution. One or more statistics have been developed for some distributions to represent the null hypothesis. The sampling distribution of these statistics is used to compute a p-value. Some statistics cover the p-value better than others. Not all distributions have a statistic, though. Also, a mixture model for a distribution does not have a statistic to test the null hypothesis.

(When such a test is available, I would caution you against always using alpha = 0.05 for every decision about significance.)

In your case, I suggest that you use an information criterion instead of a sample statistic. AICc or BIC may be used for this purpose. The best model based on the criterion alone is the one with the minimum criterion value. This is not a test, though. There is no p-value. There is no such decision as 'significant' or 'not significant.' There is only selection of one model over the other candidate models.

Learn it once, use it forever!
Highlighted
Staff

## Re: Goodness of Fit

The p-value is calculated from the sampling distribution of the test statistic for the model under the null hypothesis. For example, there are several test statistics for the goodness of fit to a normal distribution model. The Shapiro-Wilk W statistic or the Anderson-Darling A2 statistic. The test statistic is not available for all models.

Learn it once, use it forever!
4 REPLIES 4
Highlighted
Staff

## Re: Goodness of Fit

Many distribution models have a hypothesis test for goodness of fit. The hypothesis test is based on the null hypothesis (that is, the one that is assumed to be true) that the observed data sample is in fact from a population with such a distribution. The alternative is that the population does not have such a distribution. One or more statistics have been developed for some distributions to represent the null hypothesis. The sampling distribution of these statistics is used to compute a p-value. Some statistics cover the p-value better than others. Not all distributions have a statistic, though. Also, a mixture model for a distribution does not have a statistic to test the null hypothesis.

(When such a test is available, I would caution you against always using alpha = 0.05 for every decision about significance.)

In your case, I suggest that you use an information criterion instead of a sample statistic. AICc or BIC may be used for this purpose. The best model based on the criterion alone is the one with the minimum criterion value. This is not a test, though. There is no p-value. There is no such decision as 'significant' or 'not significant.' There is only selection of one model over the other candidate models.

Learn it once, use it forever!
Highlighted
Level III

## Re: Goodness of Fit

Dear Mark,

thank you for the feedback and details.

I will use it as guideline in this distribution analysis.

For my understandig: the p-value is a mathematical calculation... so why some distribution can "generate" the p-valule and other can not generate this p-value calculation?

Best Regards,

Simone

Highlighted
Staff

## Re: Goodness of Fit

The p-value is calculated from the sampling distribution of the test statistic for the model under the null hypothesis. For example, there are several test statistics for the goodness of fit to a normal distribution model. The Shapiro-Wilk W statistic or the Anderson-Darling A2 statistic. The test statistic is not available for all models.

Learn it once, use it forever!
Highlighted
Level III

## Re: Goodness of Fit

Thank you Mark.

Clear explanation.

Have you a good day.

Best Regards,

Simone

Article Labels

There are no labels assigned to this post.