topic Re: Analysis of CCD results in Discussions

Analysis of CCD results

AaronMB11 — Thu, 08 Jun 2023 21:06:45 GMT

Hi everyone,

I did screening desing and steepest ascent which showed me a saddle point. Then, I conducted experiment (ordinary central composite design) on this area to capture the curvature. But I don't know how to analyse the results, my model isn't significant with quadratic terms and the lack of fit is significant (the model only become significant when I raise my model to the power of 4).

Could someone please help me to understand more?

Re: Analysis of CCD results

frank_wang — Tue, 02 Nov 2021 00:06:04 GMT

Hi, Aaron

看起来温度是一个重要因子。

从3D plot上看，任意2个维度组成的曲线看起来不像一个抛物线。因此二次拟合效果会不好，高次拟合会造成过度拟合。

同理，如果把温度当作Z轴，剩余的两个参数当作X和Y轴，也看不出一个抛物曲面（想象一座山）。因此响应曲面应该不会很好。

高次拟合会取得更好的R平方，但是很有可能是过度拟合。

希望可以帮助到你。

Re: Analysis of CCD results

Dan_Obermiller — Tue, 02 Nov 2021 00:44:54 GMT

I'm not seeing much as being significant here either. In fact, I do NOT see a significant lack of fit as the p-value for that test with the full response surface model is 0.2195. The main issue seems to be that there is not a large change in the response over the ranges you have selected.

Even removing the axial points from the design and fitting a full factorial model does not show any significant effects (and does not show a lack of fit either). I think that the variability on your center points (standard deviation = 5.22) is large relative to the standard deviation in the rest of the data. I will say that the ranges for your variables are not very wide. Typically, you want ranges to be wide enough to be able to detect an effect. A seven degree swing from the low axial point to the high and only a four degree difference between the -1 and +1 levels is not very large.

Re: Analysis of CCD results

Phil_Kay — Tue, 02 Nov 2021 09:18:21 GMT

I agree with @Dan_Obermiller . The factors do not seem to have much effect on the response over the ranges that you have used. As a chemist, the temperature range does seem very narrow - intuitively I would not expect to see a big change in the response if I change the temperature from 175 to 182 degrees.

This might be a good thing, if this is a robustness study. The results here tell you that you could vary the factors over these ranges and get a consistent response.

But it sounds more like this was an optimisation study. In which case you will have to experiment more boldly to obtain a useful model. You could use Augment Design in the DOE menu to add runs from an expanded factor range.

An option is that you could include the data from your screening design. The combined data set will probably not be balanced but I expect it will include data from a wider factor space and might enable you to build a more useful model.

I have attached you data as a JMP data table.

I hope this all helps.

Phil

Re: Analysis of CCD results

AaronMB11 — Tue, 02 Nov 2021 10:12:44 GMT

非常感谢您的回复

Re: Analysis of CCD results

AaronMB11 — Tue, 02 Nov 2021 10:42:50 GMT

Thank you very much @Phil_Kay and @Dan_Obermiller for your insightful comments. Given that I did steepest ascent with 9 steps and saddle point was shown between step 3 and 4 which correspond to my range (176-180°C) (the response decrease from 4 to 9). Is it possible that this is only a local optimum?

Re: Analysis of CCD results

Phil_Kay — Tue, 02 Nov 2021 13:17:16 GMT

I am sure that is possible. I don't really know anything about steepest ascent methods.

I do still think that it would be worth looking at your complete data set with your screening and CCD experiments combined.

(I am assuming that the screening experiment data is not part of the data that you shared - from your description it sounds like you carried out a complete CCD on a different factor region versus the screening experiment.)

Maybe you would also include your steepest ascent data.

Re: Analysis of CCD results

AaronMB11 — Wed, 10 Nov 2021 09:13:54 GMT

Sure!

This model seems to be relevant with a non-significant lack of fit, p-value significant and R2, Radj and R pred also high

So I did steepest ascent to know how to increment each factors . This second experiment bring out the curvature area between experiment 4 and 5.

Re: Analysis of CCD results

Phil_Kay — Wed, 03 Nov 2021 10:38:10 GMT

Interesting.

How do factors A, B, C, D relate to the factors in your CCD?

Which of factors A, B, C, D are not varied in the CCD and what is the setting?

What is the level of factor E for the steepest ascent and CCD experiments? (From your screening design this seems to be the most important factor by far)

Is the response the same across screening, steepest ascent, and CCD? (You achieved much higher values before the CCD)

Thanks,

Phil

Re: Analysis of CCD results

AaronMB11 — Wed, 03 Nov 2021 11:53:26 GMT

Factor D is not varied in CCD due to its low implication in the response, I've set it constant at 0.24.
E was categorical factor so SA was selected instead of SB (despite showing better result...., SA is greener)
So, removing the value of SB, higher values are achieve in screening experiment

Re: Analysis of CCD results

Phil_Kay — Tue, 09 Nov 2021 16:42:41 GMT

Okay. I'm not sure what level of E was used in the steepest ascent experiment - I have assumed SA. I have also made some assumptions about how the CCD factors map to factors A, B, C, D.

I pulled all the data into one data table (attached). I think it has been useful but probably does not give you an answer that you were hoping for.

It seems like something strange happened between the steepest ascent experiment and the CCD. The results from these two phases do not fit the same model. The factor ranges used in the CCD are very similar to the best runs in the steepest ascent but the response is much lower in the CCD (<100 compared with 240 to 260). Again, I am assuming that the setting of factor E is SA for the steepest ascent and the CCD experiments.

In any case, I am not sure that I would recommend the approach that you took. The screening results on their own suggest that only factors E and C are important, so I am not sure why you also carried forward the other factors for the steepest ascent experiment. I don't understand the steepest ascent method as I have not used that approach. Using very narrow factor ranges for the CCD is also not something that I would recommend unless your objective was to determine if the process is robust over these ranges.

I hope that this has been a useful experience for you, even if you did not get the result you were hoping for. You might want to explore some resources to learn more about DOE before your next project. The Statistical Thinking for Industrial Problem Solving course has a good module on DOE.

Re: Analysis of CCD results

statman — Tue, 09 Nov 2021 18:28:19 GMT

I have tried to follow your thought process, but, of course, I don't understand the "science" of your investigation. I will share some of my thoughts.

I see you ran a Res. V fractional factorial with 3 center points (although there is no CP for the one categorical factor). There is no mention of practical significance of the response? How much of a change in the response is of practical importance? Statistical significance is a bit of a challenge because you have 3 replicates of the CP. One of the DF's is used to estimate curvature and the others are unassignable and therefore may be the basis for estimating the MSE. How does the MSE compare with the variation you see in the process normally? E seems to be the most "active" with C second. I don't see much (again I don't have any context with respect to the response variable) with the rest. Why did you continue with A and B? No evidence of curvature. A model based on E and C looks like there are some issues (one unusual event and the residuals are certainly not normally distributed).

What model did you use for steepest ascent? Looks like you used insignificant factors in that model? Not sure why you continued so far past the best results in your SA method?

The most significant factor was set to the least optimum level...huh? I would want to investigate that factor more. Could it be more quantitative?

Not sure why the CCD? How does it relate to the other work you did? Also be careful here as statistical significance does not play as big a role in optimization. By the point of optimization, you already know the factors matter, so what are you looking for? Sweet spots (local maxima which are difficult to maintain) or flat spots that provide more robust results.

Lastly, I don't see any strategy to "handle" noise? No repeats/nesting (short term noise like measurement error) or blocks or split-plots (for long term noise like ambient, raw material lots)?