Understanding regression model

[#1]

In multiple regression, I have a variable with high p-value (not significant), but the estimated Power is low. Does that mean we need more data to rule the variable out?

[#2 · Accepted solution]

Retrospective power analysis (here) is a no-no. The best idea in the case of a screening experiment is to design the study with acceptable prospective power and then accept the result. You have ruled out this factor.

[#3 · Accepted solution]

Hi Mark,

Could you explain this a little more? In this particular case, I have an undesigned data, so Power analysis was not done. 

Thank you so much for your help!

[#4 · Accepted solution]

You could use your data set to start a design. You should add the expected DOE column properties first. Add the Response Limits property to the Y column. Add the Coding property to the continuous X factors. Add the appropriate Design Role and Factor Changes properties to all the X factors.

Select DOE > Augment Design and cast the data columns in the correct roles, then click OK. You can modify the factor ranges at this point if you deem that necessary. Note that using reasonably wide ranges for continuous factors enhances power because you will illicit a larger effect if there is one. You can modify the terms in the model. Include terms that you want to test.

Specify the number of runs. This number should include the number of rows in the original data set. For example, if you have 10 rows and want to consider 5 more runs, enter 15. You can use the Design Evaluation section to decide if your changes seem reasonable. The power analysis might be particularly important to you.

[#5 · Accepted solution]

Hi @DOE_USER : There are loads of stuff about this out there.  Here are but a few.

https://library.virginia.edu/data/articles/post-hoc-power-calculations-are-not-useful

https://tidsskriftet.no/en/2019/01/medisin-og-tall/statistical-power-not-after

https://www.sciencedirect.com/science/article/abs/pii/S1527336909001809

One thing you have to remember...that Power is another word for Probability (in particular, the probability of rejecting the null Hypothesis when the alternative Hypothesis is true). But, and this is a big "but", once something has happened (data observed) there is no probability associated with it. Take the weather for example; it's either raining now or not...there is no probability associated with it. A retrospective power analysis is attempting to answer the (non)question of "now that we know it is not raining, what would have been the probability of it raining?". It's a pointless exercise (and full of problems as indicated via the links above).