topic Re: Different mixture experiment analysis results JMP 17 & 18 in Discussions

Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Tue, 17 Dec 2024 13:00:11 GMT

Did something in JMP 18 change in analysis of mixture experiment results. Below is shown that analysis results from exactly the same DOE data sets are different

1. DOE three component mixture

JMP 17 results

JMP 18 Results

2. DOE three mixture components + one process effect

JMP 17 results

JMP 18 results

In JMP 18 analysis above the model is different!

What has changed and why? Remark: despite of different plots & model graphs of JMP 17 - 18 prediction profilers are identical.

Re: Different mixture experiment analysis results JMP 17 & 18

Victor_G — Tue, 17 Dec 2024 13:20:06 GMT

Hi @frankderuyck,

In the first case, can you provide anonymized dataset ? Are the columns properties the same ?

In the second case, the models are different in the two captures. You should have similar results by having the same models.

Re: Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Tue, 17 Dec 2024 14:20:57 GMT

My first three component case study above was similar to the one presented in a known earlier JMP webcast vinegar/oil/water case study below, you can see that the plot is also similar to mine but not the same in JMP 18

Re: Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Tue, 17 Dec 2024 14:26:03 GMT

I remeber that the explanation for the wide confidence interval was the strong collinearity among the mixture effects which makes sense; this does not show up anymore in the JMP 18 plot (?)

Re: Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Tue, 17 Dec 2024 14:38:52 GMT

The link to the webinar is https://community.jmp.com/t5/JMP-Blog/Summer-DOE-Series-Mixture-DOE/ba-p/669012

I got green light to use this data set for my DOE training

Re: Different mixture experiment analysis results JMP 17 & 18

Victor_G — Tue, 17 Dec 2024 15:03:20 GMT

Hi @frankderuyck,

I would compare the outcomes of the first use case on something else than p-values, due to multicollinearity in mixture designs.

Again, without any dataset to test, it's difficult to help you since we have only a partial view at this comparison.

For the first use case, if the model are set up correctly and in the same way between JMP 17 and 18, try to look at the Parameter estimates (and standard deviations), at metrics focussed on predictivity and optimization (like RMSE) and residuals/diagnostic plots.

Since you mentioned the Profiler has similar results between versions, I would guess that the differences you are seeing are mostly due to the use of Mixture designs instead of a calculation change between versions.

Re: Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Tue, 17 Dec 2024 16:34:53 GMT

Find in attachment the DOE analysis data set

During my september training I still used JMP 17, trainees got JMP 18, it was a big surprise and annoying that my 17 training results were different! Mine are similar (same data) to those presented in webinar above, think also with JMP 17.

Re: Different mixture experiment analysis results JMP 17 & 18

statman — Tue, 17 Dec 2024 17:40:07 GMT

Strange, I used your data table and analyzed in both 17 and 18. The only difference I found was the confidence limits on the Actual by Predicted plots (leverage). All of the other statistics were identical (I used your scripted fit model).

JMP18

JMP17

Re: Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Tue, 17 Dec 2024 17:57:52 GMT

Indeed question is why?

Re: Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Tue, 17 Dec 2024 18:02:31 GMT

The jmp 17 plot is should be more accurate because of the very strong collinearty among mixture components; limits from jmp 18 are to my opinion too narrow?

Re: Different mixture experiment analysis results JMP 17 & 18

statman — Tue, 17 Dec 2024 18:22:33 GMT

Yes, but in your initial post, the statistics were different as well. That does not appear to be a JMP issue.

Re: Different mixture experiment analysis results JMP 17 & 18

statman — Tue, 17 Dec 2024 18:35:32 GMT

I'm not sure the correct word is accurate...different, yes. It appears the JMP18 version looks more like a leverage plot than the one from JMP17. Typically those plots are narrowest in the center and expand near the ends. Collinearity is typically a cluster of points near the middle.

https://www.jmp.com/support/help/en/18.1/#page/jmp/effect-leverage-plots.shtml?os=mac&source=application%23ww184200

Re: Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Tue, 17 Dec 2024 19:39:35 GMT

The DOE analysis script is 100% reproducible both in JMP 17 and 18. When running second DOE (mixture + process effect) analysis script, developed in jmp 17, in jmp 18 besides the prediction plot also model is different (?) The prediction profilers always are equal.

Re: Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Tue, 17 Dec 2024 20:03:33 GMT

As in case 1 the case 2 data set and analysis script are exactly the same, indeed now results are also different..

Only difference is the two jmp versions

Re: Different mixture experiment analysis results JMP 17 & 18

statman — Wed, 18 Dec 2024 14:03:03 GMT

I'm confused. I posted the output models for both 17 and 18 above. They are identical. The only difference is the leverage plot. You must have some preferences set differently in the 2 versions.

Re: Different mixture experiment analysis results JMP 17 & 18

Victor_G — Thu, 19 Dec 2024 09:24:45 GMT

Hi @frankderuyck,

I might have found a clue about the difference in plots.

In JMP 17, when you extend the ranges of "Actual by Predicted" plot, it seems the plot is constructed by using a response mean of 0 (confidence band is symetrical around 0):

So maybe the modeling without intercept of mixture models was (incorrectly ?) translated into a modeling with intercept = 0, hence the confidence band being large and non-symetrical around the response mean.
In JMP 18, you don't observe this situation as confidence band are symetrical around the response mean (5,88) :

Curious to know if any developers might confirm or inform about this difference in plots. @Ryan_Lekivetz ?

Best,

Re: Different mixture experiment analysis results JMP 17 & 18

Phil_Kay — Thu, 19 Dec 2024 09:25:16 GMT

Hi @frankderuyck ,

For any question like this I would recommend to contact JMP Technical Support (support@jmp.com). Send example data with saved scripts so they can most easily recreate the analysis exactly as you have done it. They will also need information about the version(s) of JMP that you are using.

Regards,

Phil

Re: Different mixture experiment analysis results JMP 17 & 18

frankderuyck — Thu, 19 Dec 2024 15:20:49 GMT

OK Phil, I informed technical support about this discussion

Re: Different mixture experiment analysis results JMP 17 & 18

Dan_Obermiller — Thu, 19 Dec 2024 15:36:30 GMT

Tech support will certainly get to the bottom of this. However, I do believe that @Victor_G might be on to something. I ALWAYS suggest to people fitting mixture models to look at the overall ANOVA report (Regression Reports > Analysis of Variance). It is closed by default , but I wish it were open and visible for your analyses.

If you are fitting a mixture model, that report will have the message: Tested against reduced model Y=mean. This is what you SHOULD see when fitting a Scheffe mixture model.

If it is a no-intercept model, the report will have the message: Tested against reduced model Y=0. We do NOT want this for a Scheffe mixture model. This might occur if the mixture components do not always add to exactly 1 or perhaps some of the components do not have the correct column properties assigned.

Again, I know tech support will get to the correct answer, but I wanted to chime in on this item that is often overlooked when fitting a Scheffe mixture model.

Re: Different mixture experiment analysis results JMP 17 & 18

statman — Thu, 19 Dec 2024 16:15:19 GMT

Agreed, I think Victor figured it out. The JMP 18 leverage plots look "more correct" IMHO.