All possible models multiple regression - assessing R2 change between models

[#1]

Hi All,

Relatively new to JMP. I have performed an all subsets multiple regression as I am attempting to find the best model of IVs that predicts my DV. Based on AICc, BIC and Mallows there is a model that appears best. However, there is another model that has one additional variable that has similar AICc etc (though worse), but a better adjusted R2. Is there a way in JMP of statistically comparing the change in R2 between the two models ("R2 change"), as can be done with SPSS? I want to assess if this increase in R2 is statistically significant. I suspect it is not and I should stick with the less complicated model. 

Many thanks,

Phil

[#2]

Hi @philc86 , and welcome!  If the difference between models is just one of the potential variables (as you describe), then the p-value for that additional variable is what you are after (same as testing for a significant increase in R^2). And, in general, see the following:

https://www.stat.uchicago.edu/~yibi/teaching/stat224/L04.pdf 

[#3]

Hi MRB3855,

Thank you for this. Yes, I want to test if there is a significant increase in R2 statistically speaking on adding new variables. In SPSS I can do this by using hierarchical regression (adding all but the additional variable to block 1, then adding the additional variable in block 2). I can't seem to see how to do this in JMP however. Do you know how to do this? I would rather keep to using just the one statistics program. 

Phil 

[#4]

Hi @philc86 : For more than one additional variable, I don't see a way to do it directly in JMP (unless I'm missing something). The article I provided a link for describes how to do it, but you will need to run the two models (full and reduced) to get the pieces of information you need (see pages 31 to 33) to make the calculations yourself..  If you'd like to see that capability in a future version of JMP, you can add it to the JMP "Wish List". https://community.jmp.com/t5/JMP-Wish-List/idb-p/jmp-wish-list

And here is someone who has made a video on how to do it..

 https://www.youtube.com/watch?v=OwXHkoLOse8

[#5]

Welcome in the Community @philc86 ! 

Hi @MRB3855,

Not directly related to your suggestions or @philc86 initial demand, but if conditions to use the platform The Fit Two Level Screening Platform are met (Overview of the Fit Two Level Screening Platform (jmp.com)), it could help identify important active terms and help model(s) selection. Note that effects are entered into the analysis following a hierarchical ordering.

This platform can be found in DoE, Classical, Factor Screening, Fit Two Level Screening.

Hope this may help the discussion,