cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Browse apps to extend the software in the new JMP Marketplace
Choose Language Hide Translation Bar
DOE_USER
Level III

Understanding regression model

In multiple regression, I have a variable with high p-value (not significant), but the estimated Power is low. Does that mean we need more data to rule the variable out?

4 ACCEPTED SOLUTIONS

Accepted Solutions

Re: Understanding regression model

Retrospective power analysis (here) is a no-no. The best idea in the case of a screening experiment is to design the study with acceptable prospective power and then accept the result. You have ruled out this factor.

View solution in original post

DOE_USER
Level III

Re: Understanding regression model

Hi Mark,

 

Could you explain this a little more? In this particular case, I have an undesigned data, so Power analysis was not done. 

 

Thank you so much for your help!

View solution in original post

Re: Understanding regression model

You could use your data set to start a design. You should add the expected DOE column properties first. Add the Response Limits property to the Y column. Add the Coding property to the continuous X factors. Add the appropriate Design Role and Factor Changes properties to all the X factors.

 

Select DOE > Augment Design and cast the data columns in the correct roles, then click OK. You can modify the factor ranges at this point if you deem that necessary. Note that using reasonably wide ranges for continuous factors enhances power because you will illicit a larger effect if there is one. You can modify the terms in the model. Include terms that you want to test.

 

Specify the number of runs. This number should include the number of rows in the original data set. For example, if you have 10 rows and want to consider 5 more runs, enter 15. You can use the Design Evaluation section to decide if your changes seem reasonable. The power analysis might be particularly important to you.

View solution in original post

MRB3855
Super User

Re: Understanding regression model

Hi @DOE_USER : There are loads of stuff about this out there.  Here are but a few.

https://library.virginia.edu/data/articles/post-hoc-power-calculations-are-not-useful

https://tidsskriftet.no/en/2019/01/medisin-og-tall/statistical-power-not-after

https://www.sciencedirect.com/science/article/abs/pii/S1527336909001809

 

One thing you have to remember...that Power is another word for Probability (in particular, the probability of rejecting the null Hypothesis when the alternative Hypothesis is true). But, and this is a big "but", once something has happened (data observed) there is no probability associated with it. Take the weather for example; it's either raining now or not...there is no probability associated with it. A retrospective power analysis is attempting to answer the (non)question of "now that we know it is not raining, what would have been the probability of it raining?". It's a pointless exercise (and full of problems as indicated via the links above).

View solution in original post

4 REPLIES 4

Re: Understanding regression model

Retrospective power analysis (here) is a no-no. The best idea in the case of a screening experiment is to design the study with acceptable prospective power and then accept the result. You have ruled out this factor.

DOE_USER
Level III

Re: Understanding regression model

Hi Mark,

 

Could you explain this a little more? In this particular case, I have an undesigned data, so Power analysis was not done. 

 

Thank you so much for your help!

Re: Understanding regression model

You could use your data set to start a design. You should add the expected DOE column properties first. Add the Response Limits property to the Y column. Add the Coding property to the continuous X factors. Add the appropriate Design Role and Factor Changes properties to all the X factors.

 

Select DOE > Augment Design and cast the data columns in the correct roles, then click OK. You can modify the factor ranges at this point if you deem that necessary. Note that using reasonably wide ranges for continuous factors enhances power because you will illicit a larger effect if there is one. You can modify the terms in the model. Include terms that you want to test.

 

Specify the number of runs. This number should include the number of rows in the original data set. For example, if you have 10 rows and want to consider 5 more runs, enter 15. You can use the Design Evaluation section to decide if your changes seem reasonable. The power analysis might be particularly important to you.

MRB3855
Super User

Re: Understanding regression model

Hi @DOE_USER : There are loads of stuff about this out there.  Here are but a few.

https://library.virginia.edu/data/articles/post-hoc-power-calculations-are-not-useful

https://tidsskriftet.no/en/2019/01/medisin-og-tall/statistical-power-not-after

https://www.sciencedirect.com/science/article/abs/pii/S1527336909001809

 

One thing you have to remember...that Power is another word for Probability (in particular, the probability of rejecting the null Hypothesis when the alternative Hypothesis is true). But, and this is a big "but", once something has happened (data observed) there is no probability associated with it. Take the weather for example; it's either raining now or not...there is no probability associated with it. A retrospective power analysis is attempting to answer the (non)question of "now that we know it is not raining, what would have been the probability of it raining?". It's a pointless exercise (and full of problems as indicated via the links above).