topic Re: ANOVA with multiple repeated measures factors in Discussions

ANOVA with multiple repeated measures factors

ProfCorrow — Wed, 29 Mar 2017 18:13:31 GMT

I am new to JMP and having a hard time running a repeated measures ANOVA. The problem is that I have two factors and BOTH are repeated measures. All of the examples find online show repeated measures ANOVA with one between subjects factor and one within. Can anyone help? Here is a description.

One group of subjects. This is a face recognition task with eye tracking. The dependent variable is the duration of looking time.

Repeated measures factor 1: Face Half (upper face vs lower face)

Repeated measures factor 2: Phase (Learning, Target, and Distractor)

In short, we are interested in seeing if subjects look longer to the upper or lower face and also if their duration of looking time differs across phase (when the learn the face, for target faces, or for distractor faces). Plus the interaction.

Help!

Re: ANOVA with multiple repeated measures factors

ProfCorrow — Wed, 29 Mar 2017 18:15:23 GMT

Also, I should have clarified that the difficulty is with entering these variables in JMP to do the analysis. I can figure out how to do it with one repeated measure, but can find an option to specify that I have two. I was able to make it work using stacked data but only if I specify subject as a random effect, which I do not want to do.

Re: ANOVA with multiple repeated measures factors

txnelson — Wed, 29 Mar 2017 18:19:10 GMT

If you haven't looked at this addin, I suggest you down load it from the file exchangw

Full Factorial Repeated Measures ANOVA Add-In

Re: ANOVA with multiple repeated measures factors

ProfCorrow — Wed, 29 Mar 2017 18:33:58 GMT

Thanks! I'll look into it!

Re: ANOVA with multiple repeated measures factors

ProfCorrow — Wed, 29 Mar 2017 18:40:49 GMT

This solved it. Thank you so much!

Re: ANOVA with multiple repeated measures factors

Mark_Bailey — Wed, 29 Mar 2017 18:58:41 GMT

Jim solved your problem but I just want to make sure that his note in the JMP Knowledge Base is also part of your answer.

Why wouldn't you want to treat subject as a random effect? How is a fixed effect of subject meaningful?

Re: ANOVA with multiple repeated measures factors

ProfCorrow — Wed, 29 Mar 2017 19:43:55 GMT

I am so glad you asked! This question has been killing us! We have been trying to figure out why our output in SPSS comes out so different than in JMP. In the end, we figured it must have to do with JMP treating subject as a random effect. Turns out, if I use the add-in recommended above, the result comes out identical to that of SPSS. So, can you please tell us!? Why does subject need to be included as a random effect? I have been looking in statistics books, online resources, everywhere I can think of to figure out why JMP does this, when other statistical packages do not! If someone here can provide the answer, you will make my day!

Re: ANOVA with multiple repeated measures factors

Mark_Bailey — Wed, 29 Mar 2017 20:07:22 GMT

The fact is that software was generally unable to properly estimate the random effects properly except in special cases until a recent times. So most analyses used a standard ANOVA or regression analysis and (incorrectly) treated all effects as fixed effects. Many textbooks have yet to catch up with more modern practice.

The choice between fixed or random is based on your inference about the effect of a source of variation. We are generally interested in the fixed effects of factors such as treatment. We see this effect as fixed because our inference is about specific levels and we believe that each level always produces the same effect. On the other hand, some sources of variation represent only a sample. The effect of subject is such an case. We are generally uninterested in the effect of an individual subject because it doesn't generalize but instead care about the population. In this case, we are interested in the variance across subjects. The random effect is this variance.

Re: ANOVA with multiple repeated measures factors

ProfCorrow — Thu, 30 Mar 2017 18:04:07 GMT

Thank so much for the quick reply. Do you have any resources you can point me to on this topic? I ended up analyzing my data first in SPSS with a "traditional" repeated measures ANOVA (I'll call this "SPSS Traditional" for simplicity). Then, I conducted the same analysis in JMP, but including subject as a random effect (JMP Mixed Effects) and, of course, got a slightly different result. Then, I used the add-in recommended above to do a repeated measures ANOVA (JMP Traditional) and got a result that is identical to my SPSS Traditional analysis. Finally, I was able to use SPSS Mixed to do a repeated measures ANOVA with subject as a random effect (SPSS Mixed Effects) and produced an identical result as what I had gotten with "JMP Mixed Effects." So, in essence, the two programs are doing the same thing, it is simply easier to do the Mixed Effects analysis in JMP. Now, what I am still confused about is WHY subject should be included as a random effect when it seems to me that this is not what most people are doing (or at least not what the majority of SPSS users are doing). Doesn't a "traditional" repeated measures ANOVA already treat subject as a random effect to some degree? Can you point me to any resources that demonstrate the difference between what the traditional model is doing and what including subject as a random effect is doing? Many thanks for helping me to sort this out. I want to make sure I really understand it.

Re: ANOVA with multiple repeated measures factors

ProfCorrow — Thu, 30 Mar 2017 18:13:17 GMT

P.S. Thanks also for your comment above. I understand what you are describing in principle, but am confused on the notion that I always thought that a standard repeated measures ANOVA treated subject as a random effect already. As a fellow stats friend recently put it: "I am unsure of why SPSS would *ever* treat subject as a fixed effect. I am pretty sure it is treating subject as a random effect. Treating subjects as random comes from the idea of random sampling, where we assume that the variance of scores in the subjects is a true reflection of error variance in the population - that allows us to generalize to the population, rather than to just the N people in our sample. Treating subjects as fixed vs random factors generally changes the way we construct F-ratios. The tests you get from either program should be using the error term as the denominator in the F-ratio. So for example the F-ratio for face half should be MS(face half)/MS(error), with 1, 2x3x(N-1) degrees of freedom; F-ratio for phase should MS(phase)/MS(error) with 2, 2x3x(N-1) degrees of freedom; and the F-ratio for the interaction should be MS(interaction)/MS(error) with 2, 2x3x(N-1) degrees of freedom. Whatever gives you that is doing it right!"

Re: ANOVA with multiple repeated measures factors

Mark_Bailey — Fri, 31 Mar 2017 09:48:19 GMT

I don't know for certain that the majority of SPSS users treat subject as a fixed effect but I believe you. The majority is not always right and in statistics the majority is often simply behind in terms of adopting new methods. It takes about twenty years for methods to prevail and another decade for them to be covered in textbooks and taught in school.

You are correct: if one used a traditional MANOVA for a repeated measures analysis, the subject is a random effect.

It isn't wrong to treat subject as a fixed effect but what is the use of this interpretation? Subject has nothing to do with treatments. Subject is only part of the experimental unit.

Re: ANOVA with multiple repeated measures factors

Mark_Bailey — Fri, 31 Mar 2017 09:52:19 GMT

OK, so it sounds like you are convinced that subject, as part of the experimental unit, should be a random effect. It is implicitly a random effect in the MANOVA. It is explicitly a random effect in the mixed effects model.

There are situations in which the block effect should be treated as a fixed effect but they are nothing like your study.

Re: ANOVA with multiple repeated measures factors

Mark_Bailey — Fri, 31 Mar 2017 12:40:39 GMT

The statistics instructors in SAS Education recommend this book:

Stroup, Walter W. (2013) Generalized Linear Mixed Modes: Modern Concepts, Methods and Applications, CRC Press, Boca Raton, FL, ISBN 978-1-4398-1512-0 (hardback)

Re: ANOVA with multiple repeated measures factors

ProfCorrow — Fri, 31 Mar 2017 17:31:02 GMT

Ok, I'm with you on your first point. I can see how it would take a long time for these methods to be updated and become mainstream.

Regarding this comment, "if one used a traditional MANOVA for a repeated measures analysis, the subject is a random effect." Can I ask my question a different way perhaps? If I use the JMP add-in for full factorial repeated measures ANOVA to do my analysis (as I did at the beginning of this thread), would subject be treated as a fixed or random effect?

If fixed, then I understand your comment about this method being "behind the times." If random, then what is the difference between what the add-in does and running the analysis using this method (http://www.jmp.com/support/notes/30/584.html) with the univariate split-plot approach? The results come out a bit different. My only conclusion is that the add-in treats subject as a fixed effect and the second method treats it as random.

I am also a bit thrown by JMP's use of the term "MANOVA" as I always thought of MANOVA as involving two dependent variables (http://online.sfsu.edu/efc/classes/biol710/m…), whereas I only have one DV. I take it JMP calls it MANOVA but if you enter only on DV, then it really is an ANOVA you are running?

Sorry for the long string of questions. I hope this thread will be useful to someone with the same questions as me in the future. And I am learning a lot in the process!

Re: ANOVA with multiple repeated measures factors

Mark_Bailey — Fri, 31 Mar 2017 18:11:05 GMT

The add-in treats the subject effect as fixed, which is why you get the same results as you did with SPSS.

I believe that the add-in was created to reproduce the SPSS style of analysis for users who expect results when subject effects are treated as fixed. The note from the JMP Knowledge Base treats subject effects as random.

Repeated measures is a common study design that can use MANOVA. You measure the response of the same subject at multiple times. You can include one or more factors and covariates. The subject effect is again treated as random.

Sorry but it is difficult to keep up with you. I am having muliple and deep computer prolems - at the BIOS level!

Re: ANOVA with multiple repeated measures factors

ProfCorrow — Fri, 31 Mar 2017 18:18:35 GMT

This solves it for me. Thank you so much for all of your time and effort. I understand now.

Sincerely,
Sherryse

Re: ANOVA with multiple repeated measures factors

Kevin_Anderson — Sat, 01 Apr 2017 23:28:50 GMT

Just to chime in...

If the investigator chose four subjects and wanted the interpretation of the analysis to apply to ONLY those four subjects, the subject factor should be FIXED.

If the investigator chose four subjects at random from the population of all subjects in order to generalize the interpretation of the analysis to that population, the subject factor should be RANDOM.

I would also recommend Milliken, G.A., Johnson, D.E(1997); Analysis of Messy Data, Volume 1: Designed Experiments; Chapman & Hall; pp 322-350.

Re: ANOVA with multiple repeated measures factors

jules — Mon, 08 May 2017 19:43:58 GMT

Hi All,

I just wanted to clarify something about the full factorial repeated measures add-in. That add-in *does not* treat subject as a fixed effect. The model generated by the add-in treats subject as a random effect, and models all interactions with subjects as random (random slopes by subject). If you launch the model dialog after running the model (or from the add-in directly) you can see how this structure would be defined in the standard Fit Model dialog. Results from analyses set up with this add-in will certainly differ from treating subjects as a fixed effect (which is, of course, not recommended).

Julian

Re: ANOVA with multiple repeated measures factors

ProfCorrow — Mon, 08 May 2017 21:49:03 GMT

Thanks Julian. I've done a bit of digging into linear mixed modeling since then and have clarified this issue a lot. For those reading this post who have similar questions, consult Andy Field's Discovering Statistics 4th ed (Chapter on Multilevel Linear Models) as a starting point. It now clear to me that SPSS and the JMP add-in are not treating subject as a fixed effect when running an ANOVA. However, if you suspect that there may be a correlation between your scores within a subject, then subject should be modeled as a random effect using linear mixed modeling . . . or some other test that would be appropriate. If your within subjects measures are not correlated, then an ANOVA should be sufficient. If anyone feels I am wrong on this point, please correct me!