My interest is in identifying the strongest predictors of annual per capita growth rate for three groups of bird species: residents, short-distance migrants and long-distance migrants. My model has per capita growth rate as the dependent variable, and as predictors bird density during the breeding season, and 13 environmental predictors (various measures of climate, food abundance and nest predation). I am using the "all possible models" approach in the Stepwise program (JMP Pro 17) to reduce the number of predictors to three to minimize the risk of overfitting because our sample size (one value for each year) is only 27 years (the data set has 27 rows and 15 columns). Because the BIC of some 3-predictor models differ little, I plan to include the multiple predictors in these models in a second step of model reduction using the Adaptive Elastic Net (AEN) technique (Generalized Regression program). AEN will also provide regression coefficients (±SE), statistical significance, and a variance inflation factors for each of the predictors in the final model. I have a directional expectation for each of the predictors that comes from long-term research on our study site. For example, warmer temperatures enhance food abundance and reproductive success, higher breeding density reduces breeding success. My questions are about multiple comparisons.
1) Although density is unique to each of the three models (one for each bird group), the environmental predictors are the same. Do I need to correct for multiple comparisons (e.g., Bonferroni correction) because the environmental predictors are used in each of the three models?
2) Does using the "all possible models" approach for model reduction put me in statistical deep water due to many, many uses of the predictors to find the models with three predictors that have lowest BIC?
3) I assume that because predictor 'density' is unique for each model, and because I have an a priori expectation for its influence, that I am on firm ground for inference regarding this predictor, but I am not sure if that holds for the environmental predictors.
I apologize if these questions are naive, I am struggling to find direction in literature about multiple comparisons. I have not found a clear reference about inference after model reduction. Perhaps you know of one. Thanks much for your help.
