topic Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs in Discussions

Fit Definitive Screening - Runs are not foldover or centerpoint runs

shoffmeister — Fri, 02 Feb 2018 08:51:43 GMT

Hi JMP/DOE-Experts!

I am currently planning a DSD. I wanted to do some a priori power analysis based on simulated results.

Now when I try to use the "Fit Definitive Screening" plattform JMP complains that some runs are not foldover or centeproint runs. This happens of course because I have to change the mid-values of some factors due to technical requiements (e.g.: pH is varied in between 1 and 12 but we are going to use 7 as "centerpoint" instead of 6.5). Those settings are not too far of from the center, but just not exactly in the middle.

I would really like to evaluate how much I am losing in terms of power when not doing the exact center points.

Now here is my question: Is there an easy way to use the "Fit DSD"-plattform anyways? I do not have the time to write a script for this (...sadly).

Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs

Phil_Kay — Fri, 02 Feb 2018 11:41:44 GMT

Hi,

The Fit DSD requires the design to have a foldover structure - it is the basis of the analysis.

You could use the design with the original centre-points for your power analysis. If the "corrections" to the centre-points are not too big you can assume that the power will not be greatly affected.

However, if you are ultimately going to change the centre-points for your experiment you will not be able to use Fit DSD.

Regards,

Phil

Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs

shoffmeister — Fri, 02 Feb 2018 12:14:24 GMT

Thanks for the answer!

It would be great if anyone could give me a explanation on statistical level why the Fit DSD-approach is not suitable when marginally changing some (mid-level) settings.

I'm familiar with the properties and theory of DSDs and the approach recommended by B. Jones to analyse DSDs (https://community.jmp.com/t5/Discovery-Summit-Europe-2016/Powerful-Analysis-of-Definitive-Screening-Designs-Taking/ta-p/23773). From statistical point of view I do not see why that approach couldn't be used after making marginal changes to the structure of the DOE, but I'm very open to other opinions. Of course the analysis (Power) will be worse, as there is more correlation between model effects but overall I don't see a major problem as long as the orignial values haven't been changed too much.

Thanks for any advice,

Sebastian

Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs

Phil_Kay — Fri, 02 Feb 2018 12:24:05 GMT

As you know from Brad's paper, fit DSD fits the 1st order effects and the 2nd order effects separately by separating the response. This is possible because 1st order effects are orthogonal to 2nd order effects in DSDs. If the centre-points are off-centre, this is no longer the case.

Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs

Peter_Bartell — Fri, 02 Feb 2018 13:06:20 GMT

shoffmeister: To answer this question within your thread, "I would really like to evaluate how much I am losing in terms of power when not doing the exact center points.", my suggestion is to use the Compare Designs platform to compare the JMP generated DSD with your modified design.

I agree with the comments of my colleague @Phil_Kay 's comments. But I think you know there is nothing stopping you from analyzing your modified design using the Fit Model, Standard Least Squares (I'm presuming good old fashioned ordinary least squares is appropriate for you experiment's responses vs. some other alternative modeling subplatform) platform. You can mimic some of the thoughtware behind the Fit Definitive Screening Design platform...but there's a good reason why JMP is 'wired' to not let you use the Fit Definitive Screening Design platform. Again @Phil_Kay raises some of these issues. DSDs and the Fit Definitive Sreeening Design platform for analysis are all kind of tied together with various assumptions that are inconsistent with your modified design. But you still may be able to achieve your practical goals for the experiment using the Fit Model platform and the appropriate interpretation of the various modeling diagnostics.

After all it's a screening design...so if your main goal is to find significant effects in an economical manner at some level of risk (statistical and representation) of 'missing' specific effects, then I say go for it. Also never forget that your domain expertise should also play a very important role in your ultimate interpretation and conclusions. And if you are fully intending to run confirmatory runs or subsequent additional experimentation as a result of what you learn...then I have little issue with the approach.

Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs

Bill_Worley — Fri, 02 Feb 2018 13:07:29 GMT

Maybe an option for you if you are using JMP Pro. You can try using the Generalized Regression > 2 Stage Forward Selection platform.

Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs

shoffmeister — Fri, 02 Feb 2018 13:11:31 GMT

Thanks for the answer again. Please correct my if I am wrong about the following:

I totally get that it is an important part of the procedure that main effects and second order effects are not aliased - because we use the residuals of the main effects fit in the second step of the procedure. But looking at my modified DSD the degree of aliasing between ME and second order terms (quadratic & 2FI) is very marginal. I would therefore assume that the estimation of the main effects should still be more or less valid, while I will have to accept a smaller precision for the estimation of the second order effects.

Am I missing something there? It is my impression that JMP behaves extremly conservative by not allowing me to do the DSD-Fit for this modified DSD.

Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs

shoffmeister — Fri, 02 Feb 2018 13:15:25 GMT

Thanks for all the answers.

Actually I am very confident in my design choice and my ability to analyse the results :-). I'm just trying to understand why JMP is so strict when using the Fit DSD approach.

I sometimes have the feeling that we statisticians are more consevative than nessecary. Therefore I would like to make sure that I am not missing something important when I mimic Brad's approach when I'm going to analyse the design.

Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs

Phil_Kay — Fri, 02 Feb 2018 13:49:44 GMT

Not every JMP user is as confident in their analysis. They need some guidance. They could get into problems analysing designs using Fit DSD when they have correlation between 1st order and 2nd order effects. Probably okay if the correlation is small, as you say. But where should the threshold be beyond which JMP stops you?

Re: Fit Definitive Screening - Runs are not foldover or centerpoint runs

Ryan_Lekivetz — Fri, 02 Feb 2018 22:34:42 GMT

Once you lose that foldover structure, you no longer have the partitioning of the degrees of freedom as described in the original paper, so adjustments would need to be made. In addition, even though you have a small correlation with the quadratic effect, it's not just the quadratic effects but other interactions correlated with that quadratic effect that make their way into the main effect estimates. It wouldn't stop me from using the analysis by treating it as a true center, and seeing how that matches up with other model selection techniques (admittedly doesn't help you here). As Bill mentioned, the 2 stage forward selection has a similar flavor if you have access to Pro.

This situation certainly does show up, so I have opened up a suggestion for us to see about how to appropriately handle this case in a future release.

Cheers,

Ryan