We often use hypothesis tests for a difference but there are times when we want to test for equivalence. It is not proper to use a result that a difference test was not significant to conclude equivalence. Note that JMP performs some equivalence tests, such as comparing two groups in the Oneway platform.
This script performs a test of equivalent for any parameter that meets the assumptions of a t-test:
Observations are independent.
The sampling distribution of the parameter estimates is normal.
Common examples include sample averages and parameter estimates for linear models.
Here is what the initial dialog looks like when you run the script:
The script requires just five inputs:
The hypothesized parameter value
The tolerence value - any observed difference from the hypothesized value that is smaller than the tolerance indicates practical equivalence or similarity
The parameter estimate
The standard error of the estimate
The sample size - number of observations used for the estimate
I will illustrate the use of this script with the following example.
A sample of 100 items with known value (Standard Y) are tested with a new instrument (Test Y).
The new instrument should provide identical results, so the hypothesized regression slope is 1.
The practical equivalence is within 5% of the hypothesized value, so the tolerance is 0.05.
Here is the regression analysis:
The parameter estimate is 0.9942, the standard error of the estimate is 0.00522, and the number of observations is 100. The values in the dialog are changed to match the situation:
This result appears when I click OK:
The plot shows the hypothesized parameter and the interval of practical equivalence with vertical, black lines. The parameter estiamte is shown with a vertical, red line. The red arrows show the lower and upper differences between the parameter estimate and the lower and upper bounds, respectively. The information below the plot records the values input to the dialog and reports the lower and upper differences, t-ratios, and one-sided p-values. Both tests are signficant at α = 0.05, so we conclude that estimated slope is equivalent to the hypothesized slope.