cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Choose Language Hide Translation Bar

Custom design based on covariates from two tables

Hi JMP Team and Community !

 

Here is a wish for DoE based on covariates from different tables (I'm using JMP Pro 17).

 

  • What inspired this wish list request? 

Design of Experiments based on covariates factors from two tables - JMP User Community

I have faced a difficult situation to setup a DoE with 5 covariates :

  • 2 covariates, from a table A, are predicted physico-chemical properties of formulations (thanks to the model of a first DoE),
  • 3 covariates, independent from the first ones, from a table B, are characteristics of different packagings.

 

I wanted to create a model (screening main effects and interactions, some set as "if possible" and other "necessary" with 30 runs) with these 5 covariates to evaluate relationships and interactions between formulation covariates and packaging characteristics on the response, but when I use "Custom Design" and want to add covariates, I only have the option to add covariates from the table on which I create a Custom DoE (table A OR table B). @SDF1 suggested to select the top level data table before adding the covariate term, which indeed works, but is not satisfactory for me regarding my settings :

  • Covariates Ax and Bx should be independent from each other, which is not the case anymore with this technique (since JMP seems to do an outer join based on the biggest table (or inner join) with rowID as the join option),
Victor_G_0-1675408656174.png

 

  • When doing the design, due to the joining done on the two tables, not all combinations are explored (covariates from A don't meet all conditions with covariates from B), and I have a poorly efficient design :

Script of the design:

DOE(
	Custom Design,
	{Add Response( Maximize, "Y", ., ., . ),
	Add Factor( Covariate, Covariate A1, 0 ),
	Add Factor( Covariate, Covariate A2, 0 ),
	Add Factor( Covariate, Covariate B1, 0 ),
	Add Factor( Covariate, Covariate B2, 0 ),
	Add Factor( Covariate, Covariate B3, 0 ), Set Random Seed( 695832949 ),
	Number of Starts( 3479 ), Add Term( {1, 0} ), Add Term( {1, 1} ),
	Add Term( {2, 1} ), Add Term( {3, 1} ), Add Term( {4, 1} ), Add Term( {5, 1} ),
	Add Term( {1, 1}, {2, 1} ), Add Term( {1, 1}, {3, 1} ),
	Add Term( {1, 1}, {4, 1} ), Add Term( {1, 1}, {5, 1} ),
	Add Potential Term( {2, 1}, {3, 1} ), Add Potential Term( {2, 1}, {4, 1} ),
	Add Potential Term( {2, 1}, {5, 1} ), Add Potential Term( {3, 1}, {4, 1} ),
	Add Potential Term( {3, 1}, {5, 1} ), Add Potential Term( {4, 1}, {5, 1} ),
	Set Sample Size( 30 ), Simulate Responses( 0 ), Save X Matrix( 0 ), Make Design}
);
Power analysis :
Victor_G_3-1675409864322.png

 

 Since then, I have found an "external" solution by creating a new table with all covariates from the two tables, and so create all possible combinations A/B of the 2 factors from table A with 3 factors from table B thanks to a low-code tool and some looping. When creating the custom design on this table, I find a more satisfactory design :

Victor_G_4-1675410166344.png

 

I joined the different tables  for testing (not the same as for my topic for confidentiality reason, but tried my best to have a mock-up for illustrating the situation).


 

  • What is the improvement you would like to see? 

It would be great if there was a solution where JMP would :

  1. Ask us to choose the table on which the covariates are located (in case of separate tables containing covariates),
  2. Be able to store the data in memory and give us the possibility to do all the combinations in the background when building the design based on two covariates tables. Perhaps an option to check "independent covariates" ? 

 

  • Why is this idea important? 

The Custom design is an excellent tool to deal with covariates and create optimal designs tailored to specific situations. This upgrade would make the tool even more powerful and adaptive, as it would be able to create designs with covariates from different tables and suggest adequate designs.

 

 

I hope my wish is clear, if not, don't hesitate to comment or come back to me,

Thanks a lot in advance !

7 Comments
SDF1
Super User

Hello JMP Team,

 

  I fully support this addition to the JMP Custom DOE toolbox. I think this added capability would really expand the options available to tailoring the DOE for the specific needs of the customer. I can see several situations where my organization could benefit from this added flexibility of the DOE platform. Count me in on wanting this added to JMP!

 

Thanks!,

DS

Status changed to: Acknowledged

Hi @Victor_G, thank you for taking the time to submit this idea! We will review your request and keep you updated on its status. 

mia_stephens
Staff
Status changed to: Investigating
 
mia_stephens
Staff
Status changed to: Not Planned For Now

Hi @Victor_G,

According to the development team, this would not be a trivial fix with our current implementation. It is something the team has been thinking about, but with the current priorities it's not something we'll be able to work on in the immediate future. Moving this to Not Planned for Now, but it will be revisited in a future release.

Thank you,

Mia

Victor_G
Super User

Hi @mia_stephens,

 

Thanks for the update and feedback !

 

I understand that fixing this issue may be complex, as (virtually or physically) creating all possible combinations of values from two (or more) datatables may lead to exponential computation efforts and time (In my example, I had to sample the biggest table to avoid having too many possible runs to choose in JMP during the design creation).

 

As explained, I was able to find a workaround to create the datatable needed for the DoE without code, but a native implementation of this feature in JMP would greatly help and increase reliability (to avoid going in and out from JMP to do some pre-processing steps).

 

Looking forward to seeing it in a next future  

mia_stephens
Staff

Thank you @Victor_G , and I am glad there is currently a workaround. It is certainly on the list for future consideration.

Warm regards,

Mia

kengoldberg
Level II

I asked this of JMP and SAS tech support about a decade ago. Without details of any particular application, simply having 2 independent covariates is what I asked for, "covariates from two tables".

 

For my particular frequently recurring application, I have used manually intensive work-arounds with "Sample covariate matrix" column(s) by "Location covariate matrix" column(s) interactions as "necessary" and "if possible". I copied and pasted below a snippet from a presentation about my workaround. 

 

kengoldberg_0-1695234429316.png

My "sample matrix" is usually highly imbalanced, so it is my first covariate. My location matrix is either a full factorial or sometimes restricted fractional factorial with particular row*cols combinations unused. I add factors 1 at a time to the covariate  (plate, then row, then cols) and manually making switches along the way as necessary when locations are used twice and others are empty.