Hello bjbreitling,
The first thing I would recommend is doing a Partition analysis under Analyze > Modeling. Use the same response and factors as you have for the original analysis. Make sure to use a validation set of at least 20% by selecting a validation portion in the model dialog. You can also make a validation column that you can then add to the analysis. First things first though. Once you get things set up select OK and then you will see the analysis page. The model building portion has not been run yet. You will need to select Go. If you do not see Go then you have not selected a validation set. If you do not see Go then select Split and keep selecting Split until you can go no further - no more splits will take place when you click split. Go to the red hot spot at the top and select column contributions to see which variables are the most important to your model. If you have JMP Pro do the same analysis but this time do a Bootstrap Forest. You may see a difference in variables that are important.
From this point go back to Fit Model and build a Main Effects only model with the variables that are most important and then build a model with Interactions included. If there are important interactions you will see them here. Again, if you have JMP Pro, use the Generalized Regression platform under the Standard Least Squares drop down in the upper right. This is a good variable shrinkage technique to try with all of your main effects and interactions included.
There is a univariate orthogonal fit under Analyze > Fit Y by X in JMP. Not sure what SAS has to offer there.
Hope this helps,
Bill
