topic Re: What's the relation between "coefficient estimate" in the model and "effect size" in the power analysis? in Discussions

What's the relation between "coefficient estimate" in the model and "effect size" in the power analysis?

matthias_bruchh — Tue, 12 May 2020 15:01:03 GMT

Hi,

When using the "Fit Model" platform, I get a table of with the parameter estimates for the model.

Parameter Estimates

Term Estimate
Intercept                                 -2.064624
x1                                            7.0881103
x2                                            11.977504
(x1-0.98875)*(x2-1.04406)      2

So the model coefficients are 7.088... for x1, 11.977... for x2 and 2 for the interaction term.

In the power analysis the effect size delta is given as

x1 -> 6.37483262916979

x2 -> 11.7464525716128

x1*x2 -> 1.82566916425025

So the effect size delta is relatively close to the parameters estimate but not the same. What's the difference?

(I have used random distributions with a mean value of 1 for x1 and x2.) The file is attached.

Another point: Is it possible to do a power analysis from the "fit parametric survival" platform?

Re: What's the relation between "coefficient estimate" in the model and "effect size" in the power analysis?

Mark_Bailey — Tue, 12 May 2020 15:33:23 GMT

The power analysis assumes 'effect coding' so the range in the regression and the power analysis is -1 to +1. So the anticipated coefficient is half the effect size (i.e., the response change when the factor goes from low to high). The Coding column property is automatically added by JMP when the design is saved to the data table. That way, the Fit Least Squares platform will use the coded levels instead of the actual levels in the regression. The coded coefficient estimates should match the power analysis, except for the fact that one should not perform a 'retrospective power analysis.'

Re: What's the relation between "coefficient estimate" in the model and "effect size" in the power analysis?

matthias_bruchh — Wed, 13 May 2020 09:22:51 GMT

Thanks Marc,

However, I still don't quite understand it.

In the attached example I have done the same but the x1 and x2 only have values of +/-1. (So they are the same as their 'effect coded' version you mention, right?).

Now I get the parameter effects

Term                                              Estimate             Std Error
Intercept                                       -5.33e-15            7.224e-8
x1 recode                                      4.36                    5.529e-8        (The real value is 5.)
x2 recode                                     10                        7.224e-8
x1 recode*(x2 recode+0.64)         1                         7.224e-8

The effect sizes delta are given as

x1 recode -> 4.35404318234411

x2 recode -> 7.64426468724727

x1 recode * x2 recode -> 0.764426468724727

From you comment I would have expected the effect sizes to be the same as the parameter estimates, but they are not (not even very close in some cases). So there is something I'm doing wrong.

As to the power analysis as such, the manual distinguishes indeed between "prospective" and "retrospective" power analysis.There is a note:

The results provided by the LSV0.05, LSN, and AdjPower0.05 should not be used in prospective power analysis. They do not reflect the uncertainty inherent in a future study.

As I far as I understood, they should not be mixed up because the there is no guarantee that scatter is the same in different studies. Is that correct or am I missing the point?

Re: What's the relation between "coefficient estimate" in the model and "effect size" in the power analysis?

Mark_Bailey — Wed, 13 May 2020 13:14:32 GMT

I understand what you are doing better now. You are using the Power Analysis command in the menu for the effect, with the leverage plot. Correct?

The value of delta in the power analysis is not the estimated parameter value. The JMP documentation explains this value: