topic Re: When "Set Delta For Power" is set to 2, does this represent a SNR of 2 or 1? in Discussions

When "Set Delta For Power" is set to 2, does this represent a SNR of 2 or 1?

StevenCHowell — Thu, 08 Jun 2023 20:58:29 GMT

In the DOE Guide, on pg 429, under Evaluate Design > Evaluating Relative to a Specified Model, it describes evaluating the "power of detecting an effect whose magnitude is three times the error" by setting the Anticipated Coefficient to 3 and the Anticipated RMSE to 1. This sounds like a signal-to-noise ratio (SNR) of 3.

Later in the DOE Guide, on pg 451, under Evaluate Design Options > Advanced Options > Set Delta for Power, it states to, "Specify a value for the difference you want to detect that is applied to Anticipated Coefficients in the Power Analysis report. The Anticipated Coefficient values are set to Delta/2...." From this it seems that the default Delta=2, is a SNR of 2:1, but in this case the Anticipated Coefficient values of 1 and Anticipated RMSE of 1, which differs from the first option by a factor of 2.

When "Set Delta For Power" is set to 2, does this represent a SNR of 2 or 1?

Re: When "Set Delta For Power" is set to 2, does this represent a SNR of 2 or 1?

cwillden — Wed, 14 Mar 2018 15:55:53 GMT

This has to do with how JMP models are parameterized. All parameters are basically interpreted as ajustments to the grand mean. For example, a parameter estimate "Factor[A] = -1.5" for a 2-level categorical factor (say levels A and B) would indicate that the response when Factor = A is 1.5 units below the average across both levels A and B. This means the average response for Factor = B would be +1.5 over the average, and 3 units above the average response for Factor = A. The delta is 3 between the levels A and B, but the parameter value is half of that. That's why it's saying to divide by 2.

Re: When "Set Delta For Power" is set to 2, does this represent a SNR of 2 or 1?

StevenCHowell — Wed, 14 Mar 2018 16:05:32 GMT

@cwillden, So in your example, when delta is 3, the Anticipated Coefficients for Factors A and B are +1.5 and -1.5, and the Anticipated RMSE is 1, would this represent a SNR of 1.5 or 3?

Re: When "Set Delta For Power" is set to 2, does this represent a SNR of 2 or 1?

cwillden — Wed, 14 Mar 2018 16:20:44 GMT

SNR = 1.5 if anticipated RMSE is 1. SNR for the coefficient t-test is always coef/RMSE.

If delta = 2 and RMSE = 1, SNR = 1.

Also, there’s only 1 coefficient in my example. Level A would be coded as 1, Level B as -1.

Re: When "Set Delta For Power" is set to 2, does this represent a SNR of 2 or 1?

Mark_Bailey — Wed, 14 Mar 2018 17:40:50 GMT

The t-test for the parameter estimate versus zero is the estimated parameter divided by the estimated standard error, not the estimated RMSE. The standard error depends on the RMSE but they are not the same.

If you expect a difference of 3 in the response to the levels A and B, then enter 1.5 for the coefficient as described above. If you think of that change as the signal in the response, then the S/N is 3. The S/N doesn't have to be the same thing as the t-test ratio, although the latter is often referred to as a type of S/N.

Re: When "Set Delta For Power" is set to 2, does this represent a SNR of 2 or 1?

cwillden — Wed, 14 Mar 2018 18:29:14 GMT

Correct, I was referring that the ratio of the coefficient to the RMSE is with respect to the tests for individual parameters rather than the F-tests for whole effects. While the actual denominator for a t-test is obviously not just the RMSE, ithe ratio of the coefficient size to the RMSE can still be thought of as a signal to noise ratio.