cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Check out the JMP® Marketplace featured Capability Explorer add-in
Choose Language Hide Translation Bar
Bertelsen92
Level III

Custom design - Help to split-plot/blocking

Hi JMP community, 

 

I am currently working on a custom DOE design. Overall I have 3 factors with varying levels. 

 

Stabilizer solution (I have 7 different stabilizer solution I would like to screen) + potentially adding a reference sample (no stabilizer added). 

Dosage of stabilizer - 2 levels (high and low) => (0,01% and 0,02%)  

Process (2 different heat treatments/autoclavations). 

 

The pilot plant is able to mix 4 unique codes pr. day. (we have 4 days in the pilot plant booked). This means we can maximum mix 16 unique codes that can be run through both heat treatments (giving a total of 32 codes). 

 

However the fact that we can only mix 4 codes pr. day and these need to be processed with both heat treatments (giving 8 codes pr. day) gives me some troubles. The ideal order will off course be so that: 

Day 1: 

Stabilizer 1: 0,01% - heat treatment 1

Stabilizer 4: 0,02% - heat treatment 1

Stabilizer 5: 0,01% - heat treatment 1

Stabilizer 6: 0,02% - heat treatment 1

Stabilizer 1: 0,01% - heat treatment 2

Stabilizer 4: 0,02% - heat treatment 2

Stabilizer 5: 0,01% - heat treatment 2

Stabilizer 6: 0,02% - heat treatment 2

etc... 

 

However I can not get JMP to do this (have tried using blocking, and editing the "changes" from easy to hard etc.). 

This best answer I got myself is attached. 

 

I am not familiar with scripting (JSL), so those solutions will be hard for me. I am using JMP 17. 

 

Thank you in advance. 

Best regards 

Kristian Bertelsen 

Kristian Bertelsen
Technology Specialist
9 REPLIES 9
Victor_G
Super User

Re: Custom design - Help to split-plot/blocking

Hi @Bertelsen92,

 

Which effects would you like to see in the models ?

For the screening of main effects, I'm able to create a 32-runs in 4 whole plots (so 8 runs per day like you mentioned), where you only have one process per day.
Here is the script :


DOE(
	Custom Design,
	{Add Response( Maximize, "Y", ., ., . ),
	Add Factor( Continuous, -1, 1, "X1", 0 ),
	Add Factor( Continuous, -1, 1, "X2", 0 ),
	Add Factor( Continuous, -1, 1, "X3", 0 ),
	Add Factor( Continuous, -1, 1, "X4", 0 ),
	Add Factor( Continuous, -1, 1, "X5", 0 ),
	Add Factor( Continuous, -1, 1, "X6", 0 ),
	Add Factor( Continuous, -1, 1, "X7", 0 ),
	Add Factor( Continuous, -1, 1, "X8", 0 ),
	Add Factor( Categorical, {"L1", "L2"}, "Process", 1 ),
	Set Random Seed( 235758137 ), Number of Starts( 1574 ), Add Term( {1, 0} ),
	Add Term( {1, 1} ), Add Term( {2, 1} ), Add Term( {3, 1} ), Add Term( {4, 1} ),
	Add Term( {5, 1} ), Add Term( {6, 1} ), Add Term( {7, 1} ), Add Term( {8, 1} ),
	Add Term( {9, 1} ), Add Alias Term( {1, 1}, {2, 1} ),
	Add Alias Term( {1, 1}, {3, 1} ), Add Alias Term( {1, 1}, {4, 1} ),
	Add Alias Term( {1, 1}, {5, 1} ), Add Alias Term( {1, 1}, {6, 1} ),
	Add Alias Term( {1, 1}, {7, 1} ), Add Alias Term( {1, 1}, {8, 1} ),
	Add Alias Term( {1, 1}, {9, 1} ), Add Alias Term( {2, 1}, {3, 1} ),
	Add Alias Term( {2, 1}, {4, 1} ), Add Alias Term( {2, 1}, {5, 1} ),
	Add Alias Term( {2, 1}, {6, 1} ), Add Alias Term( {2, 1}, {7, 1} ),
	Add Alias Term( {2, 1}, {8, 1} ), Add Alias Term( {2, 1}, {9, 1} ),
	Add Alias Term( {3, 1}, {4, 1} ), Add Alias Term( {3, 1}, {5, 1} ),
	Add Alias Term( {3, 1}, {6, 1} ), Add Alias Term( {3, 1}, {7, 1} ),
	Add Alias Term( {3, 1}, {8, 1} ), Add Alias Term( {3, 1}, {9, 1} ),
	Add Alias Term( {4, 1}, {5, 1} ), Add Alias Term( {4, 1}, {6, 1} ),
	Add Alias Term( {4, 1}, {7, 1} ), Add Alias Term( {4, 1}, {8, 1} ),
	Add Alias Term( {4, 1}, {9, 1} ), Add Alias Term( {5, 1}, {6, 1} ),
	Add Alias Term( {5, 1}, {7, 1} ), Add Alias Term( {5, 1}, {8, 1} ),
	Add Alias Term( {5, 1}, {9, 1} ), Add Alias Term( {6, 1}, {7, 1} ),
	Add Alias Term( {6, 1}, {8, 1} ), Add Alias Term( {6, 1}, {9, 1} ),
	Add Alias Term( {7, 1}, {8, 1} ), Add Alias Term( {7, 1}, {9, 1} ),
	Add Alias Term( {8, 1}, {9, 1} ), Set N Whole Plots( 4 ), Set Sample Size( 32 ),
	Simulate Responses( 0 ), Save X Matrix( 0 ), Make Design}
);

And attached you can find the datatable.

The trick is to specify your 2-level categorical factor "Process" as a hard-to-change factor, and specify 4 as the number of whole plot (4 days) and 32 as the number of runs (8 runs per day). Note that for screening main effects, you could reduce the number of runs if needed (but it should be a multiple of the number of experimentation day, so if you keep 4 days, you can reduce to 28 or 24 runs for example) :

 

Victor_G_1-1675269106692.png

 

If I misunderstand your objective and/or design specifications, don't hesitate to add more details and explanations, so we can find a better solution.
I hope this will help you,

 

Edit: This design doesn't seem to respect your constraint of having only 4 different mixtures/day, so increasing the number of whole plots to 8 (and doing 2 whole plots of 4 experiments each per day) with the same number of runs could be an alternative, I have to check later if it works.

Sorry for the misunderstanding,

 

Victor GUILLER
L'Oréal Data & Analytics

"It is not unusual for a well-designed experiment to analyze itself" (Box, Hunter and Hunter)
Bertelsen92
Level III

Re: Custom design - Help to split-plot/blocking

Hi Victor, 

Thanks a lot for helping out. 

Keep in mind I want to test 7 different stabilizers + a reference. So shouldn't these be put in as a categorical 8 level factor? 

 

Because we can only add one stabilizer solution to the blend at a time?

 

Yes the 4 mixtures pr day is the limiting factor. However the process (heat treatment - 2 different ones) can be run at the same day. 

 

That's why it would be perfect if you could run 4 mixtures pr. day that will then be heat-treated by both treatments. 

 

I have succeeded in getting this order (please see the attached pictures). However I am afraid that the power score is way to low for the stabilizers which is the main effect I want to investigate. What do you think? 

 

 

Kristian Bertelsen
Technology Specialist
Victor_G
Super User

Re: Custom design - Help to split-plot/blocking

Hi Kristian,

 

Sorry for my misunderstanding, you're completely right about the settings of your factors.

Now that everything seems clear (thanks a lot for your snapshots that helped me define the factors and model needed), I was able to find a design able to fit your constraints : 4 mixtures per day, with the 2 processes involved.

In this case, I didn't set a Factor as "Hard to change", but created a blocking factor "Day", with 8 runs per day, and "naturally" 4 mixtures are done each day.

 

I hope this would be a better solution that fits your needs ?

Please find the datatable attached, you can then compare this design with the one you already have prepared.

Victor GUILLER
L'Oréal Data & Analytics

"It is not unusual for a well-designed experiment to analyze itself" (Box, Hunter and Hunter)
Bertelsen92
Level III

Re: Custom design - Help to split-plot/blocking

Hi Victor, 

 

No worries. Great, however I am open to other suggestions if it could be done smarter. 

 

However I am afraid your suggestions does not solve my problem. Because L6 = 0,01 and L6 = 0,02 are unique mixes and just as different as L7=0,01 and L3=0,02 would be. So within Day 1 there would need to be mixed 8 codes (4 different stabilizers but a 2 levels), which is not possible. 

 

Maybe it cannot be done in JMP or in general. I might need to rethink the design completely. 

Kristian Bertelsen
Technology Specialist
Victor_G
Super User

Re: Custom design - Help to split-plot/blocking

Sorry, I was naively thinking that the problem was to avoid having more than 4 stabilizers per day, but I didn't think about the complete formulation and the relation stabilizer - dosage.

 

I think removing the blocking factor "day" and switching both "Stabilizer" and "Dosage" as "Hard to change" factors should respect the constraints of the 4 formulations per day :

Victor_G_0-1675332129037.png

 

And give you slightly higher power for "stabilizer" main effects (at the expense of lower power for "dosage" main effect) :

Victor_G_1-1675332167883.png

The low power score for stabilizer main effect may be explained by the number of levels of this main effect (if you had no constraints, the highest achievable power for each level of stabilizer for 32 runs with the same model would be 0,454...) and the constraints on randomization.


I can try other options and come back to you if I'm able to find something interesting.

Victor GUILLER
L'Oréal Data & Analytics

"It is not unusual for a well-designed experiment to analyze itself" (Box, Hunter and Hunter)
Bertelsen92
Level III

Re: Custom design - Help to split-plot/blocking

Hi again, 

 

I am also in doubt whenever it makes sense to include the dosage as a factors. Since the main purpose of the trials is to find out what stabilizer works best in terms of stability and potentially giving more viscosity/functionality to the desired product. We could just use one dosage (the one recommended from the supplier of the stabilizers). Then afterwards I would hope to select the 2-3 best solution and do an optimization experiment, where I vary their content and potentially some other process parameters. What do you think? I could do only one dosage but do a replicate for each stabilizers to get a better power score. 

 

The interaction term with the stabilizer and heat treatment is also a bit interesting since we have previously seen that this could give different results (both good or bad). 

I am not sure I fully understand this Power Analysis (have tried reading more about it and also found a good reply in one of your earlier posts together with Phil Kay).
https://www.jmp.com/support/help/14/power-analysis.shtml

https://community.jmp.com/t5/Discussions/Should-I-consider-power-analysis-in-DOE/m-p/501063



I think I understand what the power number means (0-1), where a value of 1 would mean that there is a 100% chance to see if their is an effect or not? 

However I do not fully understand what the Significance level and Anticipated RMSE mean and how they affect the power score of my experiment? 

I am supposed to know a certain value for these based on what I am testing or? 

Really appreciate your time and responses. 

Kristian Bertelsen
Technology Specialist
Victor_G
Super User

Re: Custom design - Help to split-plot/blocking

Hi Kristian,

 

It really depends what is the objective of your study, there is no definitive answer.

If you want to screen and select the best stabilizers in general, perhaps the information about the dosage is indeed not so important. But it could also have several implications/questions :

  • Could the quantity of stabilizer have an influence on the rest of the ingredients of the formulation (synergistic or antagonistic effects) ?
  • Could it be that behaviour of stabilizer is not linear, and fixing an "arbitrary" value of dosage would make you miss a potential important/powerful stabilizer (see graph below) as there could be different "activation"/performance concentrations depending on the chemistry of the stabilizer ? : 

Victor_G_0-1675345941198.png

If the study concentration is fixed and below the threshold of this graph, then stabilizer A would not be retained/kept for optimization, as it can be seen as having no influence on the response. But on the opposite, if you study its effects on a range, you would have a better appreciation of its behaviour and performances, and not only a single point estimation of its performance. 

 

Another option could be to focus first on main effects only in your model (or at least mainly on main effects Stabilizer type, dosage, process, and reduce the interaction terms in the model to have more power for the terms in the model), and then augment your initial design based on identified significant effects ? You would have more power for stabilizers main effect, so the differenciation/discrimination between different stabilizer could be easier, and you could augment the initial design only on stabilizers that show some performances.

 

Or perhaps you already have some data, and you could augmenting your intial dataset with new experiments through the platform "Augment Design" (and use the option "Group new runs into separate block") ?

 

You can have more informations on significance level and anticipated RMSE here : Power Analysis (jmp.com)

  • Power is the probability to detect an effect, if it is significant. Power equal to 1 means that you can detect this effect (if significant) 100% of the time, for a certain value of significance level and Anticipated RMSE. I highly recommend the video from the Youtube channel Statquest, it helps me a lot understand statistical concepts. There is one excellent video on Power analysis : Power Analysis, Clearly Explained!!! - YouTube
  • Significance level shown in this platform is also known as the alpha risk : you commonly see the alpha risk in statistical test like Student T-test for example (to evaluate if there is a difference in a response between two groups). The significance level is the probability of rejecting the null hypothesis when it is true. For example, a significance level of 0.05 indicates a 5% risk of concluding that a difference exists when there is no actual difference. Lower significance levels indicate that you require stronger evidence before you will reject the null hypothesis. More infos here : Significance level - Statistics By Jim
  • Anticipated RMSE is the "level of noise" in your response (square root of the error variation). The higher the value, the higher the noise/variance in your response, the harder it is to determine a significant effect, so the lower your power for your effects will be. You could see it as the ratio Noise/Signal : if you have a lot of noise, Anticipated RMSE will be higher, and your power calculations for the effects will be lower. By default the calculations are done with a value of 1 (which is helpful to compare different designs on the same basis), but if you already have an idea about your measurement variance, you could enter a different value reflecting this knowledge to have more relevant Power calculations. More infos here : Example of Evaluating Power Relative to a Specified Model (jmp.com)

 

I hope these options will help you in your reflexion,

Victor GUILLER
L'Oréal Data & Analytics

"It is not unusual for a well-designed experiment to analyze itself" (Box, Hunter and Hunter)
Bertelsen92
Level III

Re: Custom design - Help to split-plot/blocking

Hi Victor, 

 

Thank you so much for all your inputs!

 

I have watched most of the links you shared. Really helped with the understanding. 

 

I have also succeeded in creating a design which I am satisfied with. I ended up reducing the number of stabilizers to 5 instead of 8. I also convinced our pilot plant to make 5 mixtures pr. day. The power analysis looks strong now. 

 

Really happy how well my first post in the JMP community ended up. 

See you around in here. 

Have a nice weekend. 

Kristian Bertelsen
Technology Specialist
Victor_G
Super User

Re: Custom design - Help to split-plot/blocking

Hi Kristian,

 

Very happy that this discussion was helpful for you and guides you to a (hopefully better) design.

Decreasing the number of stabilizers levels seems indeed a good idea to increase the power : same total number of runs to estimate fewer terms in the model.

Don't hesitate to create a new post if you need further help or advices for the analysis of this DoE (or for any other topic), the JMP Community is full of passionate people !

 

Have a nice week-end,

Victor GUILLER
L'Oréal Data & Analytics

"It is not unusual for a well-designed experiment to analyze itself" (Box, Hunter and Hunter)