Bug: Evaluate design not working for Orthogonal blocks CCD by default

[Victor_G]

What inspired this wish list request? Answering to the topic Re: How do orthogonal axis values change with the number of center points? - JMP User Community, I have created several orthogonal blocks CCD. However, when I try to run the "Evaluate design" script from the design table, the script does nothing. When using the Evaluate design platform, I have an error message :

So evaluation of orthogonal blocks CCD is by default facing an error/bug when the design table is created. 

What is the improvement you would like to see? The fix was quite easy to found: for the block factor, the value of the column property "RunsPerBlock" is set to 0 by default, which makes the Evaluate design platform bug. When setting the value to the value displayed during design creation (in the screenshot 6), then the script 'Evaluate design" from the design table and the Evaluate design platform can work normally:

 
Creating the Orthogonal Blocks CCD for 3 factors and Block size = 6

In the block factor column, RunsPerBlock is set by default to 0, hence the bug. When I change the value to 6, the script "Evaluate design" works normally:

Why is this idea important? I haven't tested extensively all orthogonal blocks CCDs to check if this bug was always present, but it's present for at least Orthogonal Blocks CCD from 2 to 5 factors. Fixing this bug should not be a big issue, the fix is easy to find.

[jszarka]

I will try to add on to the great catch by @Victor_G  as I tested this out and noticed some additional issues.

I ran the same 3-factor blocked CCD as him using the default Axial Value that is Orthogonal. The Design Evaluation in the RSM platform is very unreliable for multiple reasons. In the snippet below, you will see a very strange power of 0.05 for the Block effect. The platform in general is unreliable for updating Design Evaluation (e.g., say I change the axial from 1.633 to 1.5 as it gives 'rounder' values that are easier for operators to implement) the values in the Design Evaluation will not update appropriately. This will happen in other scenarios as well (e.g., the number of center points are adjusted, different radio buttons for axial values are chosen, replicates added). The Design Evaluation can only be trusted after you Make Table and use the Evaluate Design script (except of course, for what was just shown).

A further issue is that after hitting Make Table, the default orthogonal axial value of 1.633 is not used in the table, it is actually 1.82574185835055. Because the axial value has changed, all of the powers will shift as a result. But you can see that after making the RunsPerBlock fix, the power for the Block effect gives a more sensible value. 

[Victor_G]

Hi @jszarka,

Great catch !
The errors you have found come from two sources :

• The power wrongly calculated for block effect may come directly from the bug I reported (RunsPerBlock value set to 0).
• The absence of updating of axial values and design diagnostic may not be a "bug" : axial values are calculated based on the design emphasis (rotatable, orthogonal, etc...) and on the number of factorial, axial and center runs. See Choose a Design for the details about the calculation done for orthogonal axial value.
  So if you change the number of center points or replicate runs, the axial value should indeed change, which is not the case here. Design performance is linked to the repartition of these type of runs and the corresponding axial values, so if these axial values are not changed, then design evaluation won't change either. See https://community.jmp.com/t5/Discussions/why-are-no-star-points-in-custom-design-RSM/m-p/603144/high... for the evaluation of CCD designs with different axial values.

I would frame the second error this way :

• Either it's possible to update axial values depending on the user input (number of centre points, replicate runs, etc...), then these values should be updated, so that design evaluation remains correct no matter the changes done. But I don't think it's possible, so maybe a workaround could be to "split" design generation in two parts, by first defining number of centre/replicate runs and the axial values (calculation of fixed by user), then click on "Make design" to fix the design, and secondly then display the correct design diagnostic of the generated design. The problem of updating the values and design diagnostic may come from the fact that everything is on the same window, so nothing is "fixed" to allow calculation of design diagnostics.
• Or it's not possible to update the axial values, then design diagnostic should display the original design before the design evaluation panel, to know to which design this diagnostic refers to. This is the option in Custom design, where you can see the generated optimal design before the design diagnostics panel :
  

Thanks for diving deep into this topic and sharing your findings ! :)

[jszarka]

The extra click for 'Make Design' is a nice intermediate step and great suggestion to ensure the Design Evaluation criteria is correct!

I think for your second bullet point, my surprise was that choosing the 3-factor CCD-Orthogonal Blocks shows Orthogonal axial value as 1.633, but when the table outputs 1.826, that would be an error as they are not matching. If I do something like CCD-Orthogonal, the default axial is 1.668 with 9 center points and in that case, the table does match. If I adjusted that to 5 center points, the JMP alert pops up indicating the axial value will change, and it correctly updates to 1.471. So, I'm wondering if the axial issue is just a special case in the blocking case since the VIFs are not 1 despite being labeled as orthogonal. Below I show the default snippet of the case. An axial value near 1.525 matches the JMP formula in the help documentation you showed and will make the VIFs 1 for the quadratic terms; other sources list 1.633 as shown in the JMP design. Hopefully it can be resolved as I do feel like it's related to the blocking case only.