topic Re: Generating an equivalence plot for multiple variables in Discussions

Generating an equivalence plot for multiple variables

MANOVATurtle779 — Thu, 08 Jun 2023 16:28:50 GMT

I am working on generating an equivalence plot for multiple attributes of a process. I have seen that it is possible to normalize the equivalence values and plot them all on the same graph. Can someone walk me through how to do this?

Re: Generating an equivalence plot for multiple variables

MRB3855 — Wed, 15 Mar 2023 09:43:44 GMT

If by "normalize the equivalence values" you mean that you have a unique equivalence value (in the "Difference" box in the pic below) for each difference, and you want to scale them such that they are all the same...the easiest way would be one of the following. Either of the below would result in an equivalence value of 1.

1. divide each response by its respective equivalence value.

2. divide each final confidence interval by the respective equivalence value.

Then, if you want to plot them all on the same graph...you could plot the intervals and limit via Graph builder.

Re: Generating an equivalence plot for multiple variables

Byron_JMP — Wed, 15 Mar 2023 11:46:22 GMT

I'm not going to stand behind this suggestion as a valid method for equivalence, but here's how to do it.

1. Stack all your attributes (tables>stack)

2. Range the stacked data column 0-1.

3. Super important to do this -> edit the range formula like this:

(:Data - Col Minimum( :Data, :Label )) / (Col Maximum( :Data, :Label ) - Col Minimum( :Data, :Label ))

This normalizes the data by the attribute that was stacked. If you don't do this the analysis is mud.

4. Analyze>Response Screening. Set the dialog up like this: (note Process in my example has two levels, comparing the old process to the new process.

5. Use this option from the red triangle menu

6. From here there are two options. The first option is a little complex and maybe cryptic as to what it will actually do, and the second is frustratingly limited. It would be useful to specify a value for each level, but that is why since we've already scaled the data to 0-1, chose the actual difference and pick some fraction of 0-1. For example if Process=Old vs Process=New for each scaled attribute is within 5% then they are practically the same, pick 0.05 of the difference to detect.

Now you have some results, except they are all in scaled units so I'm not sure how they would be practically intercepted.

Re: Generating an equivalence plot for multiple variables

MRB3855 — Wed, 15 Mar 2023 12:41:36 GMT

Hi Byron: From JMP help, it's unclear what the null and alternative hypothesis are for the "Practical Differences" test results via the Response screening platform as you describe. Is it the same as the TOST (two one-sided test) approach used on the Fit Model platform? https://www.jmp.com/support/help/en/17.0/index.shtml#page/jmp/equivalence-test-reports.shtml

Re: Generating an equivalence plot for multiple variables

MRB3855 — Thu, 16 Mar 2023 10:44:45 GMT

I found the answer to my query to Byron. @MANOVATurtle779, if you take to route Byron suggests you may want to take a close look at the following links to make sure you answer the "equivalence" comparison(s) of interest relative perhaps to pairwise comparisons, false discovery rates, etc. The first one is from JMP help. The other two are the pertinent references as cited in JMP help.

https://www.jmp.com/support/help/en/17.1/index.shtml#page/jmp/means-differences-data-table.shtml

http://engr.case.edu/ray_soumya/mlrg/controlling_fdr_benjamini95.pdf

https://pubmed.ncbi.nlm.nih.gov/3450848/

Re: Generating an equivalence plot for multiple variables

Byron_JMP — Wed, 15 Mar 2023 19:19:11 GMT

Just to add a note of clarification, I wouldn't use standardized variables for doing equivalence testing against some standardized difference to consider practically zero. I think that would lead to a very confusing mess to interpret.

If I had to do a whole raft of equivalence tests, I'd set it up in Fit Model, something like this:

dt = Data Table( "attribute data 7" );

obj = dt<<Fit Model(
	Y( :Attribute 1, :Attribute 2, :Attribute 3, :Attribute 4, :Attribute 5, :Attribute 6, :Attribute 7 ),
	Effects( :Process ),
	Set Alpha Level( 0.1 ),
	Personality( "Standard Least Squares" ),
	Emphasis( "Minimal Report" ),
	Run(
		:Attribute 1 << {Multiple Comparisons(Estimates( Estimate( Process( "Old" ) ), Estimate( Process( "New" ) ) ),Equivalence Tests( 2 ))}, 
		:Attribute 2 << {Multiple Comparisons( Effect( :Process ), Equivalence Tests( 3 ) )},
		:Attribute 3 << {Multiple Comparisons( Effect( :Process ), Equivalence Tests( 4 ) )},
		:Attribute 4 << {Multiple Comparisons( Effect( :Process ), Equivalence Tests( 5) )},
		:Attribute 5 << {Multiple Comparisons( Effect( :Process ), Equivalence Tests( 5 ) )},
		:Attribute 6 << {Multiple Comparisons( Effect( :Process ), Equivalence Tests( 5 ) )},
		:Attribute 7 << {Multiple Comparisons( Effect( :Process ), Equivalence Tests( 5 ) )}
	)
);

Wait( 1 );
dt2=Report( obj )["Response Attribute 1", "Multiple Comparisons for User-Defined Estimates", "Equivalence Tests",
"TOST Tests", Table Box( 1 )] << Make Combined Data Table;
dt2<<set name("Equivalence Tests");

This approach does an actual TOST with alpha set at 0.1, so its a 95%TOST.

The last bit of the script pulls together a combined table of results from all the variables.