Get MaxPro value for any Space-Filling design

[#1]

Hi dear JMP Users and expert scripters,

I would like to compare several DoEs (Space-Filling type). Some are generated with JMP (with all the column properties needed for the factors and response), some are created with other external tools (so no column properties after file importation in JMP).

Since these designs are Space-Filling types, I would like to compare them based on discrepancy and MaxPro criterion values (example here with a Fast Flexible Filling design for 4 continuous factors) :

These values are available directly for JMP-generated designs in "DOE Dialog" script, but not with external tools.

I have read the Scripting index and the "<< Get MaxPro Values" function, but I don't know how to use it in a script for "external" DoEs ?

Practically, I would like to have a script that can calculate discrepancy and MaxPro values for any design, and provide a datatable with the results.

Ideally (maybe an idea for Wish List), I would like to have a comparison platform like Compare Designs,  where I could match the factors between different tables, and a window will open with the comparison of these two values, discrepancy and MaxPro criterion.

I have attached two datatable for testing, each is a design with 4 continuous factors, one generated externally and one generated with JMP.
Any help is welcome !

Thanks, 

[#2]

Hi @Victor_G ,

  Below is some JSL code that will do what you wish. It's nothing fancy or anything, and I did explicitly write out all the DOE commands to generate the SFD. You could do it this way, or you could also point to a report window and extract out the number box value from it. But, the <<Get Max Pro Values is a nice command and simplifies that part quite a bit.

  Hopefully this helps you to get started at least.

Names Default to Here(1);

dt = current data table();

dtdoe =
DOE(
	Space Filling Design,
	{Add Response( Maximize, "Y", ., ., . ),
	Add Factor( Continuous, 0, 0.5, "X1", 0 ),
	Add Factor( Continuous, 0, 0.5, "X2", 0 ),
	Add Factor( Continuous, 0, 0.5, "X3", 0 ),
	Add Factor( Continuous, 0, 0.5, "X4", 0 ), Set Random Seed( 1622525169 ),
	FFF Optimality Criterion( MaxPro ),
	Space Filling Design Type( Fast Flexible Filling, 20 ), Simulate Responses( 0 ),
	Set Run Order( Randomize ), Make Table}
);

MaxPro_value = dtdoe <<Get max Pro values;

Print(MaxPro_value[1]);

Hope this helps!,

DS

[#3]

Hi @SDF1,

Thanks for your interest and first ideas !

I do have access to MaxPro (and discrepancy) values for any Space-Filling design created in JMP (they are calculated by default), and I was thinking of extracting them very naively with "Make Into Datatable". I think there may be a more elegant way to do this by identifying what is the property of these values box, and create a JSL script to add to any Space-Filling design in JMP so that these values can be extracted into a separate datatable. Your script is very similar to the "DOE Dialog" one generated after the creation of a Space-Filling design in JMP.

My question was related to the question : Can we calculate MaxPro and discrepancy values for any Space-FIlling design (even those generated outside of JMP), and if yes, how to script or automatize it ? 

EDIT: I just realized I could use the datatable generated outside of JMP and use it as candidate set in the Space-Filling platform (to enforce the runs through the use of covariate factors), so that JMP can calculate MaxPro and discrepancy values (example here with the external datatable provided in my original post) :

However I'm not sure these values are correct, as I obtain the same MaxPro and discrepancy values no matter which design I load into JMP with this "candidate set" approach (even JMP-generated Space-Filling designs). So this seems like a bug and not actual calculated values. If someone has an idea on how to make the calculations and display them, I would be very happy ! 

Best,

[#4]

Hi @Victor_G ,

  Got it. Sorry, that wasn't entirely clear to me that you were wanting to do the calculation yourself -- or at least script it up and not pull it from the report window. I read it as an "either/or" -- either pull it from the report window or calculate it. But, I can see why you want to calculate it on your own as it does seem strange that you get the same MaxPro value no matter what table you analyze.

  The online help has a reference to a journal article: Joseph, V. R., Gul, E., and Ba, S. (2015). “Maximum Projection Designs for Computer Experiments.” Biometrika 102:371–380, that should have the formula (or algorithm) for calculating MaxPro. I don't have access to the Biometrika journal, but if you do, you should be able to set up some JSL to do the calculation on your behalf. From the description in the JMP online help, it almost sounds like they're taking the dot product of the distance between potential design points and trying to maximize this.

  Hopefully this helps moves things forward.

DS

[#5]

Hi @SDF1,

Thanks for the reference.

I might have something to start by reading old posts from the Community, using Hierarchical Clustering platform to get access to Matrix Distance :

A = Current Data Table() <<
	Hierarchical Cluster(
		Y( :X1, :X2, :X3, :X4 ),
		Method( "Ward" ),
		Standardize By( "Columns" ),
		Dendrogram Scale( "Distance Scale" ),
		Number of Clusters( 3 ),
		Set Random Seed( 2088588291 ),
		SendToReport(
			Dispatch( {"Dendrogram"}, "Clust Dendro", FrameBox,
				{Frame Size( 35, 325 )}
			)
		)
	);
	A << Save Distance Matrix;
	A << Close Window;

Then, I'm a bit lost on how to use the Distance Matrix to get the same informations as in the DOE Dialog from Space-Filling Designs : Minimum Distance, Nearest Points, MaxPro value and Discrepancy value.

I have tested using Multidimensional Scaling platform with the Distance Matrix generated previously and defining the number of dimensions as the number of original factors in my design ; I'm able to spot nearest points correctly, but the (minimum) distance seems off compared to calculations done in the DoE Dialog :

Any inputs from @Mark_Bailey, @Ryan_Lekivetz, and other JMP experts ?

EDIT: Using "Unstandardized" in the options of Hierarchical Clustering for the data format give me the right distances... I'm getting closer !

I can now compare designs depending on the min, mean, max distance values, and variability in the distances between points (range, StdDev/Variance, ...). Still no MaxPro calculations, but a better insight about distribution of points in Space-Filling designs.
Also using the platform Distributions and comparing the distributions of factors values to Uniform distributions (Beta with alpha and beta values set to 1) help me getting closer of this "discrepancy" values. There might be something to do with the p-values, log-likelihood, chi-square values that seem correlated to discrepancy :

[#6]

Posted two wish list items on this topic :
Design Explorer for Space-Filling designs 

Compare Designs platform for Space-Filling designs 

Generating several designs and using distances from Hierarchical Clustering platform already helps assessing the "performance" and points distribution of space-filling designs. Here is a visual comparison of the min, mean and max distance values of 2 designs types with increasing number of factors (dimensionality). The first one see a sharp increase of the distances with increasing dimensionality, whereas the second one seems more "robust" with a linear increase of distance regarding the augmentation of factors. 

Looking at the Variance of the distances between points (which seems to be correlated with discrepancy values), design 1 seems to have a more compact way to spread points compared to design 2. Design 2 is able to spread points with the same "variability" when increasing dimensionality : 

[#7]

Sorry for the late (and brief) reply - there is indeed an issue when loading an external design. Note that in particular, that denominator is based on single dimension distances, so if you have any input that has the same value, the denominator is 0 (usually we're looking for the non-collapsing property, which protects against this). From what I recall, that formula can be used on the scaled estimates:

https://www.jmp.com/support/help/en/18.0/index.shtml#page/jmp/fast-flexible-filling-design-details.s...

For discrepancy, we were discussing about adding that formula directly into the documentation - if you're able to access, in the Hickernell, it’s the bottom of page 317. 

Fang, K., Lin, D., Winker, P., and Zhang, Y. (2000). “Uniform Design: Theory and Application.” Technometrics 42:237-248.

Hickernell, F. (1998). “A Generalized Discrepancy and Quadrature Error Bound.” Mathematics of Computation 67:299-322.