Verify and use existing Definitive Screen design with JMP?

[#1]

Hello,

I obtained a Definitive Screen design for a DOE Experiment by ChatGPT 4.0. It looks reasonable (after several discussions). Can JMP verify if the Definitive Screen Design is acceptable? if I have an existing Definitive Screen design, can I use it and afterwards evaluate the experimental results of such measurements by JMP? Or does JMP only evaluate its own Designs?

[#2]

Hi @Thommy7571,

First point : If you have JMP, I would strongly recommend to use it instead of relying on Large Language Models to create your DoE, as LLMs are not able (by design) to reason, plan and calculate. Generally speaking, don't use a LLM when you expect a fact, calculation or analysis. Depending on the training set, it might provide acceptable answers, but it's not always the case (and it may not be reproducible).

On the datatable itself, there are also several aspects to check, like :

• Presence of 3 levels for each continuous factors (-1, 0, 1),
• The runs in the design come in foldover pairs with one run at least in the centre of the experimental space.

If you have a dataset and would like to check how it is organized, I would recommend using the platform Evaluate Designs (jmp.com). There are several aspects you might notice when using a Definitive Screening Design and evaluate it :

1. Looking at the Power Analysis for Main Effects, you should have the same (or very similar) power values for same factors types (here X1 to X5 are continuous, X6 is categorical with 2 levels) :
   
2. Color map on Correlations : You should have no correlations between main effects, and between main effects and 2-factors interactions, so you end up with this kind of pattern : 
   

   Here, I have a very small correlation in the main effects area due to the presence of the categorical factor X6, but if you have only continuous factors, you won't have this small border. Looking at the Alias matrix also can help checking if there are no complete confounding (which could be present in classical factorial designs). 

3. Looking at a scatterplot matrix can also help see the particular structure of the DSD and absence of correlation between factors: 
   

See more here : Introducing Definitive Screening Designs - JMP User Community

Finally, if you have JMP and still want to check the design generated by ChatGPT 4, you can generate a DSD with JMP with the same setting and number of factors, and use the platform Compare Designs (jmp.com) to check for any irregularities or strong differences.

Hope this answer may help you,

[#3]

@Victor_G, warning, noob question & apology for hijacking this conversation. What surprises me with DSD in JMP is that the nr of trials for the first 6 factors added to a design invariably stays at 17. Only with the 7th factor does the number of trials increase to 21. For completeness, I included a row for one factor, although this makes limited sense.

Factors	Blocks	Nr. of extra runs (JMP default)	Trials (total)	Power (intercept)	Power (any term but intercept)
1	1	4	17	0,97	0,937
2	1	4	17	0,969	0,935
3	1	4	17	0,967	0,932
4	1	4	17	0,965	0,929
5	1	4	17	0,963	0,925
6	1	4	17	0,959	0,92
7	1	4	21	0,988	0,975
8	1	4	21	0,987	0,973
9	1	4	25	0,997	0,992
10	1	4	25	0,996	0,992

Questions:

• Why does JMP suggest so many runs already with very few factors and why does the number of runs start changing only after the 7th factor is introduced?
• Looking at the power analysis of the factors, the incremental reduction from the first to the sixth factor looks marginal. Why?
• Is there something I am misunderstanding?

[#4]

Hi @Ressel,

I can highly recommend reading the blog posts by Bradley Jones about DSDs to get a better understanding of their use :

Introducing Definitive Screening Designs 

Proper and improper use of Definitive Screening Designs (DSDs) 

About your questions :

• DSDs were not created as a "one-size-fits-all" screening approach, and were aimed at optimizing and reducing the number of runs for screening experiments involving 5+ factors. Below 5 factors, there are "better" or more suitable design alternatives for screening :  D-optimal designs, classical screening designs, or even Response Surface designs. 
  Below 7 factors, JMP is creating a DSD with minimum required runs for 7 factors (17 runs), and consider the non-used factors as "fake factors" to increase power. That explains why there is a small decrease of power in designs involving 2 to 6 factors, as the more factors are being used in the experimental design, the less extra runs are available to increase power and terms estimation precision. For the 7th factor, you need to add extra runs (4 recommended by JMP) to increase power, as there are no fake factors anymore.
• The minimum number of runs is evolving linearly for DSDs depending on the number of factors m :
  • 2m+1 when number of factors m is even,
  • 2m+3 when number of factors m is uneven.

This explains why designs with 7 and 8 factors, or 9 and 10 factors, have the same required number of runs.

• The additional 4 recommended runs by JMP correspond to adding 2 fake factors in a DSD. You can do the test and compare the designs (DSD with 4 extra runs and DSD with 2 fake factors), they should be identical :
  

Hope this answer will help you

[#5]

Yes, your answers are always helpful. Thanks!

[#6]

Short answer, JMP can evaluate any data set.  Whether there is anything to gain from the analysis is a different question.

[#7]

Hallo, danke für die Antwort. Vielleicht ist ChatGPT da nicht optimal, aber es gibt ja auch noch andere Möglichkeiten, sei es ein anderes Programm, sei es das Design eines Kollegen. Heißt das also, dass ich ein Design auch mit einer anderen Software erstellen kann und dann mit JMP testen, bearbeiten und verwenden kann? Es wäre gut, wenn sich auch die Möglichkeit bietet ein Standard-Design (nicht randomisiert) im Nachhinein zu bearbeiten, also Hinzufügen von mehreren Zentralpunkten, Randomisieren etc. Ist das möglich? 

[#8]

The short answer is 'yes'. You can import just about any design into JMP. The general pathway would be to create the design in whatever the originating application is, then save it in some file format. JMP has the ability to natively import many, many different file formats. See the documentation. If nothing else, save the file to some commonly used format, say, a .xls worksheet and then it's as easy as copy/paste into JMP. Once the design is in a JMP data table it's fully editable as any other JMP data table.

Now here's the deal though...when you create a design natively using JMP...lots of metadata is created along with the plain Jane treatment combinations. JMP uses this metadata in the background for all manner of things related to visualization and analysis. None of the meta data typically comes with the imported file...so you may end up spending alot of time adding the metadata to each column to get JMP to behave as you would like in your analysis workflow, results and JMP reports.

So my moral of the story question is...if you already have JMP to do the analysis...why would you go elsewhere to create the design? Just doesn't make sense to me. About the only reason I can think of is somebody else created the design. Ran the experiment and is now asking you to analyze the results.

[#9 · Accepted solution]

Hi @Thommy7571,

Yes, you can evaluate any design with the platform Evaluate Designs (jmp.com).

You can compare several designs with the platform Compare Designs (jmp.com).

Finally, if you want to change factors ranges, model terms, constraints, add centre points, replicates or anything, you can use the platform Augment Designs (jmp.com). Randomization is always proposed at the end of any design construction, before creating the datatable : 

There are several examples in the JMP Help section about use cases for augmenting a design : replicate a design, add centre points, fold over, axial points, space-filling augmentation.

All these operations can be done with any design from any sources, as long as you can upload the data in a format JMP recognizes : Import Your Data (jmp.com) But as @P_Bartell mentions, you may have some manual preparation to do with factors and responses column properties to add specific properties that help JMP recognizes how to deal with the factors and responses from DoE : Column Properties (jmp.com)

Hope this response answers your questions,