ANOVA, Dunnett, and Overall Type 1 Error

[#1]

Hello,

I have equally-spaced data (8 points in time) and four treatment groups (control, low, mid, and high). The number of subjects in each treatment group is identical, with no missing data points.

I have three key response variables on which I would like to perform statistical testing.

My thought was to perform a one-way ANOVA for each time point (8 tests per response variable). If any ANOVAs come back with a significant p-value, I would then perform Dunnett's tests to compare the 3 treatments (low, mid, high) against the control for a given time point.

I am a bit confused when it comes to calculating the worst-case Type 1 error for this scenario. Wondering if my line of thought is correct:

Worst-case: all response variable one-way ANOVAs are significant at each time point...

so... 3 (response variables) x 8 (time points) + 3 (response variables) x 3 (Dunnett's tests) x 8 (time points) = 24 + 72 = 96 total tests

Significance level = 0.05

So for a Dunnett's test to be statistically significant, it would need to have a p-value less than 0.0005 (0.05/96)???

Hope this makes sense. Little bit confused by all of this... JMP seems to use an LSD term, but I am not sure if it is taking all of the Type 1 errors into account.

Thank you!

[#2]

I suspect that you have a repeated measures design. This topic arises often in the JMP Community discussions. Please see this Knowledge Base note. If you have more questions after reading this note, please come back here!