DOE Help: enforcing all levels of a factor within random blocks

[#1]

Dear JMP & Community,

  I'm trying to set up a Custom DOE where one of the factors has three levels, and each level of this factor MUST be present in each random block only once.

Situation:

  We have a DOE where we know there will be 4 continuous variables that can range between low and high values. We have an additional factor that is a three-level categorical factor. The DOE must be grouped into random blocks of size 3 (this is due to limitations on the number of runs that can be performed in a day, 3 morning and 3 afternoon) -- of course the total number of runs is also a multiple of 3. The last factor, X5, has 3 levels, L1, L2, L3 which physically represent 3 locations of a sample within a drying oven. These locations are left, center, and right.

  The researcher runs batches in groups of three and each batch gets a position within the drying oven, left, right, or center. Researchers can run up to six batches in a day, so the random blocks are of size 3 (3 morning and 3 afternoon). Whenever three batches are made, all three positions get used. For example, run 1 might go in position L2, run 2 in position L3, and run 3 in position L1. One of the things being tested (in combination with the other factors) is whether the location within the drying oven is having an impact on the outcome.

Problem:

  The problem I'm facing is how to force that when JMP generates a DOE, that it ensures each of the 3 levels occurs once in each random block, yet be randomly assigned to the runs. In the screen shot below, I think it helps to show the problem. For the first six runs for the random blocks 1 & 2, the order of X5 is L2, L2, L3 & L1, L1, L3. Obviously, this can't occur in the design because L1, L2, and L3 refer to left, center, and right positions in the oven and can can only be used once within a random block. so the order of X5 should be something like L1, L3, L2 and L3, L2, L1 for random blocks 1 & 2. I still need each level to be randomly assigned to the runs, but that each level appears only once within each random block. I do not think that adding a blocking factor is the right way, but perhaps I'm wrong.

  In the DOE model, I am also interested in testing all main effects from X1-X5 and all 2nd order interactions between all factors.

Possible solution:

  The only way I can come up with at the moment is to generate a DOE with the four continuous factors and a DOE with the one 3-level categorical factor. Each has the same number of runs and then joining the tables together by matching the column "Random Block". But, when I do that, each run from the four factor DOE is repeated 3 times because of the different levels of categorical factor from the other DOE. This isn't what I want, either. If I click the Drop Multiples from the Main Table, then I go back to the original number of runs, but the ordering of X5 is almost always the same: L2, L3, L1. Also, I don't think the 18 runs will be enough to also capture the 2nd order interaction of X5 with the other four factors. I believe I'll need more like 30 runs total to be able to capture all 2nd order interactions. This still doesn't quite capture what I want, because it's repeating the same run for all three levels within a random block.

  If anyone has had a similar issue and figured out how to overcome this, I'm very interested in learning how you did this. Or, if there is a proper methodology within JMP to achieve this, that is also something I'm interested in learning.

Thanks for your interest and input!,

DS

[#2]

(Withdrawn)

[#3]

I don't completely understand your situation as you have only discussed one factor and have not provided any other context (e.g., What is your objective? Are you screening? What is an experimental unit (is it a batch or a part or a sample? What are your hypotheses regarding oven location?, etc.), so here are my thoughts:

1. Why not use systematic sampling of current production to get an idea about the location within oven effect?

2. I don't understand this as a design factor.  Let's say it is significant and the center gives the best result.  Are you really going to limit your oven runs to just the center negatively affecting production?

3. If the experimental unit is a batch or parts, use split-plots.  The experiment on the 4 factors creates the whole plot and you take samples of the batch and put each sample in each of the 3 oven locations. If it is parts, make multiple parts for each treatment combination of the 4 factors and put a part in each location.

4. Why do you need to test all 3 locations?  Can you assume symmetry and just test 1 level outside (left or right) and the other the center?

[#4]

Hi @statman ,

  I appreciate your input. I'll do my best to answer your questions and provide some context. Because of proprietary reasons, I can't discuss it in too much detail.

  The objective of the experiment is to test how the five factors interact with each other and impact the outcome. In this regard, it would be closer to a screening type of DOE as the researchers are not so interested (at this time) in finding the exact settings to reach a goal, but rather to learn more about how these factors interact and affect the outcome.

  As mentioned previously, an experimental unit is a batch. The researchers can make three different batches simultaneously, each batch is different because of the settings for X1, X2, X3, and X4. Each of these three batches are physically placed inside an oven to dry. The placement of the batches in the oven might have an affect on the outcome, and it might interact with some of the other X1 to X4 factors in the DOE. All positions in the oven must be used during drying. There is some evidence the location in the oven can affect the outcome, but it hasn't been studied, hence this DOE. Each experimental unit (batch) is placed in only ONE of the three positions in the oven. And of course, all three batches are dried at the same oven temperature as it's not physically possible to have each batch, in each position, dried at a different temperature.

  The setting is not production, it's a research lab, so there's a bit more flexibility in this regard as we're not tying up production equipment.

   Regardless if it is found that a certain position is "best", we will not limit our oven runs. We still maximize the amount of batches to dry in each oven run. However, knowing that one location might push the outcome one way or another will be useful knowledge. Understanding this will help the researchers to compensate for this effect.

  Symmetry cannot be assumed in our specific situation because of the physical layout of the oven.

  Unfortunately, there is still no satisfactory answer as to how one can use JMP's DOE platform to generate a proper DOE for the specific situation at hand. At present, it seems that the only way to do this is to perform two separate DOEs that will split some of those 5 factors up -- e.g. one DOE with X1 to X3 and another DOE with X4 and X5. This is unfortunate because there is some evidence that suggests some of the first factors X1, X2, and X3 might interact with the X4 & X5 factors of the second DOE. Ideally, there would be one DOE to test all of these together, but with the stipulations as mentioned in the original post.

  Hopefully this helps to add context and information. Thanks for your interest, and I hope to get an answer to this issue.

DS

[#5]

My guess is that you're including the interactions with X5 and the continuous factors in the model? That's forcing the algorithm to worry more about estimation of those effects versus orthogonality to the random blocks. It's more noticeable here because of the random blocks of an odd size. In my quick testing, if I remove interactions involving the categorical factor I'm seeing the balanced structure. You would have to check on those interactions afterwards, but generally it shouldn't be terrible (and may just require some additional runs). 

You could also try a covariate approach just to force the structure with the blocks specified (and are treated as fixed at that point), but this requires changing factor roles after the fact. 

We appreciate you bringing this to the community, this is related to a general structure that we're certainly thinking about in development for a future release. 

[#6]

Hi @Ryan_Lekivetz ,

  Yes, I am including the interactions with X5 and the other continuous factors in the model.

  Unfortunately, when I run the model with all second order interactions AND remove all interactions with X5, I don't get the balanced structure. It's almost there, but there are a few random blocks that have duplicate levels of the categorical factor. I did generate the table with 30 runs, as this is the suggested number when including all the X5 interactions.

DS

[#7]

A couple of additional thoughts.

1) You can take the "boss" approach, where you manually fix the broken random blocks to give you the structure you want, and check design evaluation that it looks reasonable. 

2) The code below uses your random block as Very hard to change, oven as hard to change, and the continuous effects. Note that you need to remove the effects from the model for the first 2 factors, we're just using them to get to the random block structure. You effectively throw out the random values that appear in those columns.

In either case, you'll have to determine how you want to model these effects based on your treatment structure (In addition to the Box & Jones paper, another good read here: https://www.jmp.com/en_us/whitepapers/jmp/split-plot-designs.html) 

DOE(
Custom Design,
{Add Response( Maximize, "Y", ., ., . ),
Add Factor( Continuous, -1, 1, "Fake VHTC", 2 ),
Add Factor( Continuous, -1, 1, "X5 as random block", 1 ),
Add Factor( Continuous, -1, 1, "X1", 0 ),
Add Factor( Continuous, -1, 1, "X2", 0 ),
Add Factor( Continuous, -1, 1, "X3", 0 ),
Add Factor( Continuous, -1, 1, "X4", 0 ), Set Random Seed( 1471138809 ),
Number of Starts( 940 ), Add Term( {1, 0} ), Add Term( {3, 1} ),
Add Term( {4, 1} ), Add Term( {5, 1} ), Add Term( {6, 1} ),
Add Term( {3, 1}, {4, 1} ), Add Term( {3, 1}, {5, 1} ),
Add Term( {3, 1}, {6, 1} ), Add Term( {4, 1}, {5, 1} ),
Add Term( {4, 1}, {6, 1} ), Add Term( {5, 1}, {6, 1} ),
Add Alias Term( {1, 1}, {2, 1} ), Add Alias Term( {1, 1}, {3, 1} ),
Add Alias Term( {1, 1}, {4, 1} ), Add Alias Term( {1, 1}, {5, 1} ),
Add Alias Term( {1, 1}, {6, 1} ), Add Alias Term( {2, 1}, {3, 1} ),
Add Alias Term( {2, 1}, {4, 1} ), Add Alias Term( {2, 1}, {5, 1} ),
Add Alias Term( {2, 1}, {6, 1} ), Make Strip Plot Design, Set N Whole Plots( 8 ),
Set N Subplots( 3 ), Set Sample Size( 30 ), Simulate Responses( 0 ),
Save X Matrix( 0 )}
);

[#8]

Hi @Ryan_Lekivetz ,

  Thanks for your thoughts and input. I was able to leave X5 as a 3-level categorical factor and change one of the other Xs to a VHTC factor (as it's the oven temp) and things pretty much worked out, however I did have to go in and change a few things for X5 to make it have the structure I need.

  Thanks for the link to the white paper, I will read over this.

DS

[#9]

I suggest you read:

Box, G.E.P., Stephen Jones (1992), “Split-plot designs for robust product experimentation”, Journal of Applied Statistics, Vol. 19, No. 1

Your situation begs for split-plots and/or strip block designs.

You may have to create your own table to accommodate these designs (I do it all the time).

[#10]

Hi @statman ,

  Thanks for the input. I believe I will have to create my own table in the end, as you point out.  And I agree, I think a split and/or strip block design would be appropriate here. It's just too bad that JMP can't do this for me with the constraints that I need.

Thanks!,

DS