topic Re: Response surface VS Mixture response surface in Discussions

Response surface VS Mixture response surface

Archy — Sat, 09 Feb 2019 21:44:23 GMT

Hello Gentlemen,

I am writing because I am a little confused about the options that there are in JMP. I am trying to evaluate a DOE and the construction of the model Effect there are several option like response surface which is by default but I did a comparison with Minitab and I found different results between both softwares but I think the Minitab result is the good one because it is more according to the factor plots that I have previously so I started to play a little bit with the option and I found that mixture response surface was the closest one. Why is this happening? I tried to find more information about that in the JMP tutorial and there is lack of information

Note: my DOE is built by 4 factors and I’ve several response

Best regards

FactorialPareto

Re: Response surface VS Mixture response surface

Dan_Obermiller — Fri, 08 Feb 2019 20:18:20 GMT

I'm a bit confused on several aspects.

Please explain what design was created. Was it created in JMP?

Was it a mixture design? A mixture design has a constraint that some or all of the factors add up to a specific amount, usually 1. If this is NOT the case, ignore the mixture response surface option.

Could you possibly share the data?

I am also guessing since you had several responses, the JMP Fit Model dialog had all fo the responses listed in the Y role of the dialog box. If this is the case, then the Effect Summary table that you show is actually listing the LOWEST p-value for each model term ACROSS ALL RESPONSES. (a common point of confusion).

People on this discussion board will be happy to help, but we do need more details in order to help.

Re: Response surface VS Mixture response surface

Archy — Sat, 09 Feb 2019 21:45:26 GMT

It did a full factorial but I cut one part because one parameter was useless in my experiment. It is more like a customized DOE (My DOE is full factorial 3x2x3x3=54 test but I did 45.) and it was created in JMP. My doubt is in the section of Fit model and more specifically in the part of construct model effect, see picture. The option by default is response surface but apparently the result is a different from Minitab so I started to play and I found that mixture response surface is the closest result one, my question is why is that happing? What are the differences between both?. Do you have an email to send you my data?

Fit Model Dialog

Re: Response surface VS Mixture response surface

Dan_Obermiller — Sat, 09 Feb 2019 00:40:07 GMT

You designed a full factorial, but you did not complete a full factorial. That could be leading to your confusing results. Not completing some runs is NOT equivalent to a custom design. Custom designs are optimal for the model that is specified. Missing some of the trials may actually lead to some model terms not being estimable.

The confusion with the model window cannot be answered without understanding the data.

As I said in my first post, a mixture response surface model is to be used only when you have mixture factors. You can ignore it because it does not apply to your situation. If you use it, you will essentially be fitting a no-intercept model which completely changes all of the mathematics and interpretations of your model. Don’t use it.

From your Pareto screen capture, it appears that the last term is not being tested (a missing value for the effect). This leads me to believe that your missing runs are not allowing you to fit the model that you desire. Do you see a Singularity Details part in your JMP report? If so then I am correct. You need to build a model that your data will allow you to estimate.

And finally, the default response surface model in JMP is the default because it is the model associated with your full factorial design. You did not run that full design. Therefore, the model is likely not correct. You changed the design, you need to change your model because those things go together.

Re: Response surface VS Mixture response surface

Archy — Sat, 09 Feb 2019 21:46:48 GMT

How can i get my DOE like i want? let me explain better i am working with machinering parameters so i have three tools, two of them are caplable to cut in tu conditions example 1 and 5 but one tool is a special case so it can only cut at 1, so that was the reason i just delate the part of the 5 from the full factorial for only one tool. How can i develop a DOE with this caracteristics?

Best regards

Desired DOE

Re: Response surface VS Mixture response surface

Archy — Sat, 09 Feb 2019 21:49:08 GMT

Updated information: related to the previous email, i splited my DOE in two full factorial, one DOE with a width of cut of 1 and the other one with 4.3, i did this in order to fit my response with the DOE model and got the following result for a width of cut 1 but it doesn't work because i am also interested in that response

FirstResult 2Result

Re: Response surface VS Mixture response surface

Dan_Obermiller — Sat, 09 Feb 2019 04:55:35 GMT

You would not want a full factorial design because you could not complete it. A custom design with a disallowed combination would be a better choice. You should refer to the design of experiments guide in the Books section under the Help menu.

Also note that you would not be able to estimate the interaction between tool and width of cut because you will not be running all of those combinations. That term should not be in your model.

Re: Response surface VS Mixture response surface

Archy — Sat, 09 Feb 2019 21:50:28 GMT

Hello, I read the tutorial and i made a custom design with a disallowed combination and honestly, i don't know why it gave me a strange table with some duplicate values and sometimes with some misses values i tested the combination of too and width of cut and it works well, then i tried tool, width of cut and cuttining speed and was okay too: however, when i did the 4 together, it starts the disaster and found some duplicated values and some missed values. Could you tell me why, I post all my parameters and the table. could you help me please?

Best regards

DOEParameters

Re: Response surface VS Mixture response surface

Archy — Sat, 09 Feb 2019 21:51:31 GMT

Updated information: Even for three factors, i have some repeat parameters. Is there a way to block repeat values? Uploaded picture

3 factors

Re: Response surface VS Mixture response surface

Archy — Sat, 09 Feb 2019 21:56:45 GMT

Updated information: Even for three factors, i have some repeat parameters. Is there a way to block repeat values? Uploaded picture

3 factors

Re: Response surface VS Mixture response surface

Archy — Sun, 10 Feb 2019 00:04:46 GMT

Even on the option of full factorial in custom design, i got duplicates values. Could you tell me why it is?

Re: Response surface VS Mixture response surface

Dan_Obermiller — Sun, 10 Feb 2019 14:38:27 GMT

Archy,

Instead of sharing pictures, you should share the JMP data tables. That will have the necessary information to determine where things might be going wrong. From the captures that you have provided, I do not notice replicated runs (I did not go through every line in detail). However, if there are, it is because the custom design algorithm has determined that repeating certain trials is more informative for estimating the model than a new design point. Custom design chooses the design based on the MODEL YOU SPECIFY. Which, in one of your screen captures you only have a main effects model specified -- no interactions. For that model you would not need 45 trials, but since you asked for that many, JMP would likely repeat some of the runs.

You also stated that you get these repeats when "specifying the full factorial option in custom design." There is no "full-factorial option" in custom design.

Re: Response surface VS Mixture response surface

Archy — Sun, 10 Feb 2019 21:50:27 GMT

Okay, please find attach the two data tables, one i made in Excel which i must replicate and evaluate using a stadistical sofware like JMP and the custom data table which was made by JMP and is wrong. I put 0 in thereplicate box and i think so it is in picture of the parameters that i posted before. As i said in previous emails, my goal is to replicate and evaluate the excel DOE. What can i do? I did several test and nonestly, i could find something which can solve my issue in JMP. And yes you right. There is no option for full factorial in custome design but i did it because i inverted the order of my factorials in order to use the restriction DOE evaluation

Besr regards and have a nice week

Re: Response surface VS Mixture response surface

Dan_Obermiller — Mon, 11 Feb 2019 19:09:26 GMT

Thank you for providing the files. This helps tremendously.

First, you state that you MUST replicate the Excel file. If that is the case, you are not really interested in creating a designed experiment. You are interested in running the specific trials in the Excel sheet. Why not just open the Excel table into JMP, run those trials, enter the responses and analyze?

You will ALWAYS have difficulty getting JMP to match your Excel table because JMP can do better than that table of runs.

A few issues that I saw from your table:

When specifying your factors, if the factor is continuous (like Feed per Tooth, Cutting Speed, and Width of Cut), then specify that factor as continuous rather than categorical. The only reason to specify them as discrete is to FORCE certain levels into the design. If you truly want to do that, specify a higher-order model and/or use the Discrete Numeric option.

When I made the factors continous (except for Tool Name), I created a 45 run design for the main-effects only model (attached design). I then compared the JMP generated custom design to the Excel table. The JMP custom design had equivalent or better power on all of the model terms. The JMP design had lower prediction variance than the Excel table for a large majority of the design space (and where it was worse, it was extremely close). I have attached the result as a picture. Bottom line is that JMP will not give you an inferior design if a better design exists.

Creating the design in this fashion with JMP does limit the factors to two levels. Experimentation is about EFFICIENCY. Do as few trials as possible to estimate your desired model. Then you use the model to determine the optimal settings. You do not need to try every combination in a grid to optimize a process. With this approach, because you were only specifying a main effects model, there is no need for more than two levels for any of the continuous factors. Conducting experiments at more than two level is wasteful.

Why insist on a third level? Many times it is because you expect a curved response. So, the model to use to estimate curvature is a response surface model. In JMP, I clicked the RSM button to get a response surface model (design attached as JMP RSM Design). That will force a 3rd level because it is needed to estimate a quadratic effect. I then compared this 45 run custom RSM design to the Excel table. Again, the JMP custom RSM design is superior as shown in the attached picture.

Remember that JMP is designed to create the best designed experiment possible, not to match a specific table of runs. The table of runs may not be the best choice. The only way to have JMP "create" a table to match the Excel table is to use DOE > Classical > Full Factorial. Specify all of the factors and create the design. Then you must manually remove the unwanted combination. Note that the last piece must be manual because if you do not do that combination you are NOT running a full factorial design. That is why JMP does not build that design automatically. That is the purpose of the custom design: create designs for your specific situation rather than trying to force fit a design that is not really appropriate.

You might wish to take a class on design of experiments or perhaps look at the Optimal Design of Experiments book by Goos and Jones to gain more insights on design creation. I hope this has been helpful.

Compare JMP Design to Excel TableCompare JMP RSM Design to Excel Table

Re: Response surface VS Mixture response surface

Archy — Mon, 11 Feb 2019 19:33:37 GMT

Hello, thanks you for your tables and your feedback. I reviewed the tables and images that you uploaded and I have some technical difficulties to develop the experiments of RMS table. Simply because the tool which is noted by O cannot exceed more than 1. It is a restriction that we have. We developed a screening before and we stablished the operational parameters. Indeed, we developed all the experiments and we are looking to analysis the data and have some Pareto chart of standardized effect and make the ANOVA and see which parameters have most significant effect. I haven’t try JMP excel tool yet because I thought Fit model was powerful enough to make it. However, I still have some question.

What did you mean with that “ The only reason to specify them as discrete is to FORCE certain levels into the design. If you truly want to do that, specify a higher-order model and/or use the Discrete Numeric option”

How did you get the factor RS in the fit model option?

Finally, retaking the las point,

“Remember that JMP is designed to create the best designed experiment possible, not to match a specific table of runs. The table of runs may not be the best choice. The only way to have JMP "create" a table to match the Excel table is to use DOE > Classical > Full Factorial. Specify all of the factors and create the design. Then you must manually remove the unwanted combination. Note that the last piece must be manual because if you do not do that combination you are NOT running a full factorial design. That is why JMP does not build that design automatically. That is the purpose of the custom design: create designs for your specific situation rather than trying to force fit a design that is not really appropriate.”

Actually, I did that at the beginning of this discussion and I found the results are not correct if I compared the results for example with Minitab; however, I remembered that you told me that I just match the DOE with the model.

Best regards and have a nice day

Re: Response surface VS Mixture response surface

Bill_Worley — Mon, 11 Feb 2019 21:11:37 GMT

If I may, I would like to add to what @Dan_Obermiller has written.

The method you used to first analyze your data in JMP was not a fair comparison to minitab because you only had categorical factors. You cannot estimate quadratic or squared terms with categorical factors so there is no response surface. That is why Dan was saying to change the factors to continuous. With continous factors you can get the desired quadratic estimates if they indeed exist and important to the model. It would be good to see your minitab model output to see if there are any squared terms in your model. If there are that would help explain the difference in the models you saw initially.

The last image you shared looked like you were getting close to what Dan was.

HTH

Bill

Re: Response surface VS Mixture response surface

Archy — Mon, 11 Feb 2019 22:31:59 GMT

Hello Bill,

Yes, I modified my factors from categorical to continuous and see the results but I couldn’t find any differences in the surface response, see attached picture. To your question, Minitab is working in full quadratic as default option, see attached picture too.

Best regards

Re: Response surface VS Mixture response surface

Bill_Worley — Tue, 12 Feb 2019 16:47:32 GMT

Thank you for the images and it looks as if you are more than likely in an overfit situation with your minitab model. The JMP effect summary is showing the factors that are most important and active in the model based on p-value and logworth value. Based on the JMP output the quadratic effects appear to be of little influence on the best overall model. I am not familiar with how minitab helps find the most important factors for their models, but that might be a good place to start to make sure you model is not overfit.

Re: Response surface VS Mixture response surface

Archy — Tue, 12 Feb 2019 17:34:12 GMT

Hello Bill,

You have a good point there. How could i know when a system is over fiting and when it is not? Me neither i don't know how minitab does it. I said in previous email. I just selected the default parameters which are describe in the toturial. I agree with you in some times but the problem that i cannot fix that i have someparameters which are not evaluated by JMP like the width of cut or appears to be no significant. On the other hand, minitab puts this factor as significant. it is 180 degrees of difference.

Best regards

Re: Response surface VS Mixture response surface

Bill_Worley — Thu, 21 Feb 2019 03:22:50 GMT

Hello Archy,

With JMP there are many ways to set up cross-validation to help avoid overfitting your model. Almost every platform within JMP allows K-fold cross-validation. If you have a smaller data set use a k-fold for the number of samples (rows) in your data set. For example if you have 20 rows set your k-fold to 20. This is also called leave-one-out cross-validation.

If you are using JMP Pro you can use a validation column and one other option would be to let JMP randomly select up to 20% of your data and then hide and exclude the selected data to be used as a holdout set to test the predictive capability of your model.

Overfitting is a real issue for many predictive models. Check your p-values for each parameter. With JMP the default p-value for variables that are important for the model is 0.05. Anything above that value is suspect and you should check the model predictive capability with and without variables that have a p-value above 0.05.

One other telltale sign that you might be overfit is if your R-square and R-square adjusted values are significantly different from each other. Those two values will likely be fairly close together if the model is not overfit. There are several other ways to check for overfitting, but the above should help you decide whether JMP or minitab is giving you the better predictive model.

Hope this helps.

Best,

Bill