topic Re: DSD Even-Order Effects Estimate and Statistical Significance in Discussions

DSD Even-Order Effects Estimate and Statistical Significance

Kimani — Thu, 08 Jun 2023 21:09:32 GMT

Hi, community,

I am new to definitive screening design and I am trying to get my head around the data I got recently. I had 6 factors, one of which was a categorical variable, and two responses.

I did an 18 run DSD which only identified 3 factors to be active. The second stage for determining the even-order effects identified all possible 2-factor interactions to be included in the model for one of the responses, even though only the intercept was significant.

The even-order effect estimates for the other response only included statistically significant 2-factor interactions and quadratic factors.

I have been reading through the heredity and hierarchical principles but still, I am not able to identify the reason why some of the statistically insignificant interactions and quadratic effects are still included in the model for one response and not for the other.

Any help will be greatly appreciated.

I have tried to include a screenshot of the fit definitive screening design I am referring to.

Re: DSD Even-Order Effects Estimate and Statistical Significance

P_Bartell — Mon, 16 May 2022 11:55:17 GMT

I'm not 100% sure of your question or confusion but I'll try and explain what is the thinking behind the staged approach to building a model with the Fit DSD platform. JMP developers have constructed a workflow that is considered a best practice when analyzing this specific type of design that follows what are called 'stages'. Stage one fits a main effects only model including ALL the main effects but only reports the effects with relatively low p values. The idea here is to respect the property of effect hierarchy encountered in systems under study. This a core concept in DOE in general...that is key to DSD thinking and analysis. Then stage two fits a model to include all the even order and quadratic effects that include the 'significant' effects from stage one. This stage respects the notion of effect heredity. Those are all reported in stage 2. The final 'stage' combines all the fitted effects from stage 1 and 2 into a combined model...from there you can use that model for subsequent 'work'...whatever you choose. Maybe inform sequential additional experimentation? Or prediction if so bold.

I've probably missed a point or two or three so maybe others can chime in?

Hope this helps?

Re: DSD Even-Order Effects Estimate and Statistical Significance

Mark_Bailey — Mon, 16 May 2022 13:59:10 GMT

Your explanation mentions 6 factors in the DSD but the Fit Definitive Screening shows only 3 factors.

Re: DSD Even-Order Effects Estimate and Statistical Significance

Kimani — Tue, 17 May 2022 01:16:25 GMT

@Markbailley

Yes, I had six factors. On running the fit definitive screening, only 3 factors show up. I suppose those are the ones with the lowest p values.

My question is, why does one of the responses show all possible second-order interactions for stage 2 even though their p values are greater than 0.05 (no strong evidence for significance).

Re: DSD Even-Order Effects Estimate and Statistical Significance

Kimani — Tue, 17 May 2022 01:18:57 GMT

Thank you for the explanation and apologies for the confusion.

My question was, why does one of the responses (the one on the left, A) show all possible second-order interactions for stage 2 even though their p values are greater than 0.05 (no strong evidence for significance)?

Re: DSD Even-Order Effects Estimate and Statistical Significance

Mark_Bailey — Tue, 17 May 2022 13:37:27 GMT

What do you see in the Fit Model launch dialog when you click Make Model button in example A?

Re: DSD Even-Order Effects Estimate and Statistical Significance

Kimani — Wed, 18 May 2022 01:53:10 GMT

When I choose to make the model for A, these are the suggested model effects.

Re: DSD Even-Order Effects Estimate and Statistical Significance

Mark_Bailey — Wed, 18 May 2022 13:51:18 GMT

I think that you should contact JMP Technical Support (support@jmp.com) about this issue. Please reply here if they have an answer and a workaround.

Re: DSD Even-Order Effects Estimate and Statistical Significance

Kimani — Thu, 19 May 2022 01:29:57 GMT

Okay. Let me contact them and find out. Thank you for the help.

Re: DSD Even-Order Effects Estimate and Statistical Significance

Kimani — Fri, 20 May 2022 08:11:59 GMT

@Mark_Bailey

I contacted technical and they helped me understand the stage 2 selection process.

The RMSE for stage 2 was larger than that for stage 1.

The effects can later be removed manually based on judgement.

From the book: JMP® 13 Design of Experiments Guide

"Stage 2 uses a guided subset selection procedure. The goal is to continue to add second-order effects to the model as long as the ratio of the RMSE from Stage 2 to the RMSE from Stage 1 is greater than the specified threshold. When the ratio is less than or equal to the threshold, this indicates that there are no additional second-order effects to add to the model. The default threshold is 1. Smaller thresholds increases the number of terms likely to identified as active as compared to larger thresholds"