Huge DF denominators for repeated measures ANOVAs with JMP 14

[#1]

I used to run repeated measures ANOVAs with JMP 13 with no proglems.  But now I got JMP 14 and ran an ANOVA with the same data and somehow got "4326" as a DF Den under "Test Denominator Synthesis".  This used to be "42" with JMP 13, and ovbiously the new F ratio got so much larger.  I do not think there is anything different I did (data, how to set up), other than I am now using JMP 14 instead of JMP 13, and I really do not know what to do.

Could anyone please let me know what I am missing?

Thank you! 

[#2]

They are not different. Not at all.

I copied the data from the Excel workbook to a JMP data table. I set up the same full factorial linear model with Subject as the random effect in Fit Model dialog. I used REML to fit the data with JMP 14. I saved the script for the Fit Least Squares analysis and then saved the data table. I closed the data table and JMP 14. I launched JMP 13, opened the data table and ran the script. The results are identical.

I then submitted the Fit Model job to SAS, which uses PROC MIXED. The results are identical. (Except for the different parametrization in the SAS procedure.)

So the DF for the denominators in question are the same (4326) by all three analyses (JMP 13, JMP 14, and SAS 9.4).

Also, as I mentioned before, these DF seem reasonable given the number of observations and the number of parameter estimates.

[#3]

Thank you again for your quick response.

I now have to appologize.  I am sorry I made a mistake in remembering the model I used to use.  I found that my old laptop still had JMP 13 (which will be good until this March), started playing with it, and finally remembered that I used to put in all possible combinations of the factors including Subject as the attached image shows.  In this way, I can get the output I showed you as Image 3 of my RepeatedMeasures.docx file.  If I try to use REML instaed of EMS, JMP 13 complains that there is not enough memory.  I also tried to run the same model with JMP 14 on my new laptop, but then I get the "Unable to allocate enough memory" error message for both EMS and REML.

I like this output as it gives the same results as Unix-ranova and SPSS (General Linea Model>Repeated Measures>Full factorial, SS Type III).  I could try to do all the analyses with JMP 13 on my old laptop for these couple of months as a quick fix, but I do still have altered versions of my original questions as below:

1') Why putting in all possible factors change DF Den's so much (different DF Den's for different factors vs. just one number for all the factors except Disease, and for the Denom MS Synthesis column, different factor combinations for different factors vs. just "Residual" for all the factors except Disease)?  

2') Ideally I would like to continue running my ANOVAs with JMP 14 on my new laptop, but would there be any way to allocate more memory?

Thank you.

[#4]

All that I know about you (i.e., your background) and your purpose is based on this discussion, this data set, and the modeling attempts, so please forgive me if my response covers anything that you already know.

1. This question suggests that you do not have a strong background in linear models.
   1. The DF for the denominator are a function of the number of observations and the number of parameters in the model for a given experimental unit. The parameterization in the JMP Fit Least Squares platform is such that the number of parameters for a categorical factor is one less than the number of levels. So including Channel costs 25 DF alone. Crossing it with other factors quickly adds up to a lot of parameters and, therefore, a large change in the DF.
   2. The choice of the estimation method (EMS or REML) can also affect the DF for the denominator. That difference is the reason that REML is recommended.
   3. Note, too, that the estimation depends on the design of the data. How did you design this study?
2. The amount of memory used by JMP cannot be altered from within JMP other than to use the 64-bit version instead of the 32-bit version. I am not an expert in Windows so I don't know how to allocate more memory to an application from the operating system.

You say that you like this output because it agrees with SPSS and Unix-ranova. I suggest that a model should be based on its own merits and not the software used to estimate the parameters. It seems to me that your model is over-specified.

• The structure of your study indicates no nesting whatsoever. All of the factors are crossed. Why add this complication to the model specification?
• You specified a full factorial model (i.e., all possible interaction effects). Are those terms reasonable?
• You crossed Subject with many of these interactions. Are those terms reasonable?
• You select EMS for estimating the model and variance components. Why use this inferior method?

How did you select the model that you like? What is the basis for selecting this model? What is the model selection process and criterion?

For example, I started with a full factorial model for the fixed factors and add Subject as a random effect. I was able to reduce the model using reasonable model selection criteria (e.g., parsimony, AICc),

Residual analysis indicates no pattern or heteroscedasticity or lack of fit. The residuals indicate some right skew in the errors. This model seems reasonable to me. Also, I would expect to get the same results with this model using JMP, SAS, SPSS, or Unix-ranova with REML procedure.

[#5]

Thank you very much for your very detailed response and trying to run a more appropriate model.

I do agree that I have been trying to blindly follow what has been prevously done in the field and use a full factorial model, and that there is no theoretical reason for the experiment to cross Subject and all the interactions.

But actually I was asking something else in my quetion 1', so please let me ask this again with some more explanations:

Please compare Image 1 and Images 2ab of the attached file.  Image 1 shows that when all possible factors are crossed, the DF Den column lists different numbers for different factors (from the top: 42, 42, 42, 1050, 42, 42 ...), which are also what I get with Unix-ranova and SPSS (General Linea Model>Repeated Measures>Full factorial, SS Type III).  However, when Subject is not crossed with other factors as in Images 2ab, the DF Den column lists 42 for Disease but 4326 for all the other factors.  Also, the Denom MS Synthesis column of Image 1 lists different factor combinations for different factors, while the Denom MS Synthesis columns of Images 2ab just list "Residual" for all the factors except Disease.  Why would this be the case?

Thank you again.

[#6]

Please see from within JMP Help > Books > Fitting Linear Models then the chapter at the end of the book with Statistical Details. The origin of the degrees of freedom is explained in that chapter.

[#7]

Thank you for your response and the reference to the JMP book.

I have looked at the book and found the following on pp. 197-198 of Chapter 3:

“Test Denominator Synthesis

For each effect to be tested, an F statistic is constructed. The denominator for this statistic is the mean square whose expectation is that of the numerator mean square under the null hypothesis. This denominator is constructed, or synthesized, from variance components and values associated with fixed effects.

Source Shows the effect to be tested.

MS Den Gives the estimated mean square for the denominator of the F test.

DF Den Gives the degrees of freedom for the synthesized denominator. These are constructed using Satterthwaite’s method (Satterthwaite 1946).

Denom MS Synthesis Gives the variance components used in the denominator synthesis. The residual error variance is always part of this synthesis.”

I also looked at Satterthwaite (1946), but it was too technical for me to follow.

In the meantime, I have also tried more factorial combinations.  If you could look at Images 1-3 of the attached file, it appears that when a certain factor is crossed with Subject, the corresponding DF Den stays independent while the DF Dens for the other facters are put together as “Residual”.  Could you kindly explain why it becomes like this in my example?

Thank you again.