Applying DOE at the Bench

[#1]

I am still relatively new to the world of DOE and using JMP.  I have to say that I am a convert on the utility of DOE, but I have had an incredibly hard time finding materials that actually walk you through application of DOE for assay development, or more real world applications in the laboratory.    I think one of the main reasons it is not adopted more rapidly is due to the lack of pragmatic and application specific material.   Does anyone know of a good source of material to learn more about how to apply DOE at the bench in drug development.    I have yet to find one article that discusses setting up a 96 well plate in assay development.  What is the appropriate way to treat 96 or 384 well plates in terms of plots and blocking?   Any help would be appreciated, as I have had a lot of difficulty with this.  I have even noticed that the DOE custom designer freezes quite frequently when you enter in 96 or 384 runs.    Again, any advice on additional materials would be highly appreciated.  

[#2 · Accepted solution]

The principles of DOE translate well in the area of assay development. I worked in R&D in that field for fifteen years and used DOE extensively for both screening and optimization purposes. I, too, am amazed at the dearth of learning materials about DOE as it pertains to pharmaceutical R&D. I have also found that more than half of the articles that I have encountered to be wrong or misleading. (Maybe peer review fails when your peers are ignorant about the subject, too.) Here is a good source of information if you learn well from a book on your own: Optimum Design of Experiments: A Case Study Approach by Peter Goos and Bradley Jones.

I believe you when you say that you have had difficulty with setting up your experiment using microtiter plates. (BTW, take a look in the File Exchange area here in the JMP Community for my article and materials about custom shape files for Graph Builder!) But you should be able to design an experiment for this device. I just designed a two-factor experiment for a second-order polynomial model with an eight-level blocking factor Row and a twelve-level blocking factor Col. (See attached example data table.) The design is randomized within the last block but you can arrange the rows as necessary depending on how you set up your plates. Please examine this example and use it to ask further questions.

It would help us if you could share more specific details about you are trying to do, what you struggle with, or specific situations when JMP fails.

[#3]

Thanks so much for the speedy reply.   I will definitely provide a clear example of an experiment I am trying to set up, and hopefully you can help me set up the DOE as an example.

I have a Nanobret assay designed to measure GR / GR interaction, which I want to first optimize for dynamic range.   I have a positive control, in this case dexamethasone, which should produce the maximum signal in the assay and a vehicle control (which represents the baseline noise in the assay)  Their are a number of variables which could influence the performance of this assay.   These variables include 

1.   Cell number (cont)                   Suggested Range:    5000 - 30,000

2.   Plasmid Ratio (cont)                                                  0.1 - 0.001

3.   Serum Concentration (cont)                                       0% - 4%

4.  Cell Type  (cat)                                                           HELA, HEK293

5.  Integration time (cont)                                               0.1 - 2 seconds

6.  Time Post Trasnfection (cont or cat?)                         24, 48 , 78 hours (this can be continous but experimental workflow is an issue too)

Now I am working in 96 well plates and need to reserve about 8 for controls in this plate ( I could double that so we remove an even number of columns, so that plating cells evenly in two blocks is easier)   The rest of the plate can be used for the DOE.

The one thing I have to keep in mind is experimental workflow, ie. it is near impossible to have 20 different cell number concentrations per plate. (it

would be great to only have two cell types each at once concentration per 96 well plate)  While I can plate both cell types on one plate, I would like to block the plate so it is divided into two (i.e. half the plate has one cell and the other half has the other)   The serum concentration would also be best to keep to two per plate (one different concentration per block of cells, but this might be easier to control in rows) I would like to keep the number of plates below 5 if at all possible, but I am open to suggestions based on the DOE.

I hope that I have laid everything out clearly here.  It would be really helpful if anyone could provide an example of how best to do this, especially keeping experimental workflow in mind.

I am sure that anyone with bench experience will understand where I am coming from with this issue.   

Thanks in advance!

[#4]

Thanks so much for the speedy reply. I will definitely provide a clear example of an experiment I am trying to set up, and hopefully you can help me set up the DOE as an example.
I have a Nanobret assay designed to measure GR / GR interaction, which I want to first optimize for dynamic range. I have a positive control, in this case dexamethasone, which should produce the maximum signal in the assay and a vehicle control (which represents the baseline noise in the assay) Their are a number of variables which could influence the performance of this assay. These variables include
1. Cell number (cont) Suggested Range: 5000 - 30,000
2. Plasmid Ratio (cont) 0.1 - 0.001
3. Serum Concentration (cont) 0% - 4%
4. Cell Type (cat) HELA, HEK293
5. Integration time (cont) 0.1 - 2 seconds
6. Time Post Trasnfection (cont or cat?) 24, 48 , 78 hours (this can be continous but experimental workflow is an issue too)


Now I am working in 96 well plates and need to reserve about 8 for controls in this plate ( I could double that so we remove an even number of columns, so that plating cells evenly in two blocks is easier) The rest of the plate can be used for the DOE.

The one thing I have to keep in mind is experimental workflow, ie. it is near impossible to have 20 different cell number concentrations per plate. (it
would be great to only have two cell types each at once concentration per 96 well plate) While I can plate both cell types on one plate, I would like to block the plate so it is divided into two (i.e. half the plate has one cell and the other half has the other) The serum concentration would also be best to keep to two per plate (one different concentration per block of cells, but this might be easier to control in rows) I would like to keep the number of plates below 5 if at all possible, but I am open to suggestions based on the DOE.

I hope that I have laid everything out clearly here. It would be really helpful if anyone could provide an example of how best to do this, especially keeping experimental workflow in mind.
I am sure that anyone with bench experience will understand where I am coming from with this issue.

Thanks in advance!






Sent from my iPhone

[#5]

I will proceed to use JMP custom design. In many ways, this is an ordinary experiment with restrictions on the randomization. I am trying to keep the thought process and final design as straight-forward as possible. (It might need refinement - perhaps a lot of refinement!)

I used your definitions to make a design. Here are the salient points:

• All factors entered exactly as you defined them with two exceptions.
  • Defined plasmid ratio as the log10 plasmid ratio for a low value of -3 and a high value of -1. Why? Because this factor represents your thinking about dilution steps and this transformation makes that change linear and simplify the modeling.
  • I really don't like zero for the low value of a continuous factor. This definition essentially makes it a categorical factor: absent versus present. I actually left the low value for Serum Concentration at 0% but I recommend a low value like 1%: low but present.
• The number of levels for each factor is determined by the model. Including higher order terms will necessitate more factor levels. On the other hand, many levels are unnecessary (and inefficient - low information content - if the model is relatively simple.
• I don't think we need to explicitly treat the columns and the rows on the plate as blocking factors in the design.
• You prefer to work with half a plate at a time, subtracting out one column from each half for control runs.
  • This choice defines the size of a whole plots in the design, 40 wells.
• The Cell Number should be the same for both cell types on a plate, so this factor is very hard to change.
  • Cell Number becomes the whole plot factor.
  • Its effects will be estimated with the lowest precision.
  • This decision means that you cannot reduce the number of plates much. Four plates might be a practical minimum for the modeling.
  • This design provides 80% power if the effect is at least 7X the standard deviation.
• The Cell Type and Serum Concentration should be the same for half the plate, so these factors are both hard to change.
  • Cell Type and Serum Concentration become the sub-plot factors.
  • Their effects will be estimated with the second lowest precision.
  • This design provides the same 80% if their effects are at least 4X the standard deviation.
• It seems like you have a lot of replication but that is enjoyed only by the easy to change factors, Plasmid Ratio, Integration Time, and Time Post Transfection.
  • I am not sure what replication and randomization mean for these factors because you will be presumably dealing with a plate or half-plate at a time here, too.

The following script will make the design I am talking about for your examination and consideration.

DOE(
	Custom Design,
	{Add Response( Maximize, "Dynamic Range", ., ., . ),
	Add Factor( Continuous, 10000, 50000, "Cell Number", 2 ),
	Add Factor( Continuous, -3, -1, "Log10 PLasmid Ratio", 0 ),
	Add Factor( Continuous, 0, 4, "Serum Concentration", 1 ),
	Add Factor( Categorical, {"HELA", "HEK293"}, "Cell Type", 1 ),
	Add Factor( Continuous, 5000, 30000, "Integration Time", 0 ),
	Add Factor( Discrete Numeric, {24, 48, 72}, "Time Post Transfection", 0 ),
	Set Random Seed( 556681383 ), Number of Starts( 3 ), Add Term( {1, 0} ),
	Add Term( {1, 1} ), Add Term( {2, 1} ), Add Term( {3, 1} ), Add Term( {4, 1} ),
	Add Term( {5, 1} ), Add Term( {6, 1} ), Add Potential Term( {6, 2} ),
	Add Alias Term( {1, 1}, {2, 1} ), Add Alias Term( {1, 1}, {3, 1} ),
	Add Alias Term( {1, 1}, {4, 1} ), Add Alias Term( {1, 1}, {5, 1} ),
	Add Alias Term( {1, 1}, {6, 1} ), Add Alias Term( {2, 1}, {3, 1} ),
	Add Alias Term( {2, 1}, {4, 1} ), Add Alias Term( {2, 1}, {5, 1} ),
	Add Alias Term( {2, 1}, {6, 1} ), Add Alias Term( {3, 1}, {4, 1} ),
	Add Alias Term( {3, 1}, {5, 1} ), Add Alias Term( {3, 1}, {6, 1} ),
	Add Alias Term( {4, 1}, {5, 1} ), Add Alias Term( {4, 1}, {6, 1} ),
	Add Alias Term( {5, 1}, {6, 1} ),
	Set N Whole Plots( 4 ),
	Set N Subplots( 8 ),
	Set Sample Size( 320 ),
	Make Design,
	Change Anticipated Coefficients( [1 3.5 1 2 2 1 1 1] ),
	Set RMSE( 1 ),
	Set Run Order( Keep the Same )}
);

I expect that you will have issues with it, but I am doing the best that I can based on the information you have given me.

You can rename Whole Plots to Plate and Subplots to Half Plate if you like for clarity.

[#6]

I think that this matter of designing a sophisticated experiment with restrictions on the randomization is important and, therefore, this discussion could help a lot of users in a similar situation. It is a great example of what we hoped this discussion area in the JMP Community could be for all of us. I am bewildered that there has been no activity for two weeks. I doubt that it is because the last reply answered all of the remaining questions.

I hope that the discussion picks up again. Here are some points that I hope provoke more replies:

• How is this case like your own if you develop assays? How is it different?
• Do you have the same restrictions on randomizing the runs (wells)? More? Less? Just different?
• The restrictions on the randomization clearly impact how practical the design is to run. Is it just as clear how the restriction impacts the analysis? What might happen to my interpretation of the regression model if the restriction is ignored in the analysis?
• What are some practices that you find beneficial while designing, running, or analyzing an experiment with restrictions?

(Note that there is another related discussion started by the same user. I am going to cross-post this call to action and may the best discussion win!)

[#7]

Part of the reluctance here might be the language that we use. Perhaps that should have been languages. The scientist talks about plates and wells, pipetting and dilution, incubation time, concentration, and so on. The statistician, on the other hand, (indirectly through the JMP user interface for custom design and directly through replies here) talks about types of factors, easy, hard, or very hard to change factor levels, whole plots and sub-plots, blocking, and so on.

Did you get all that? Probably not. That's normal.

So we need to take the time to explain ourselves to the other people in the discussion. The more advanced the topic and the issues, the more explanation is required to fully communicate the problem and the solution.