Power analysis for repeated measures

[#1]

I would like to run a study to estimate the change over time of my samples. The change is expected to be linear.

I have estimated a measurement error standard deviation of 2.5, and the final goal is determine if the change over a year is below 5.

I have tried to use the power explorer for one sample mean, where I have used 2.5 as standard deviation, and difference to detect 1.25 (my thinking is that I don't want to wait one year, but only 3 months, so 5 / 4 = 1.25).

Wiht an 80% power, I get a sample size of 25.

First of all, is it correct to use the power explorer?

Then, if is correct, does it mean that I have to measure each sample 25 times every time I remeasure it over the 3 months? or that I need 25 sample and I will measure them only one time every time I measure them over the 3 months?

[#2]

Hi @bbenny7 : The short answer to your question (is it correct to use the power explorer?) is "no".

There is a much longer answer...that may well be beyond (from a practical point of view) what we can do on a forum like this. And, a clarification is necessary. Is this repeated measures? i.e., is the same experimental unit(s) tested at each timepoint?  Or, are different units tested at each timepoint?  The former situation is repeated measures.  With a repeated measures study, there is the additional complication of how the correlation between timepoints is modelled.

Your null and alternative hypothesise are as follows:

H0: slope >1.25

Ha: slope <=1.25 (the thing you want to "prove")

That said, in the latter case:

Your measurement error is 2.5. But the standard error (SE) of the slope is a function of the spread of Y (MSE) around the line, the standard deviation of X, and sample size as shown here. We may be able to assume MSE=2.5^2 =6.25 (the numerator in the SE calculation). 

https://en.wikipedia.org/wiki/Simple_linear_regression#Normality_assumption

So it's not just sample size. All other things being equal, power would be highest when you have n/2 points at t=0, and n/2 points at 3 months, since that minimizes the SE.  

And slope/SE has a t distribution so power can be calculated, but JMP's module is not designed for dealing with slopes (though with care it may be possible...but I haven't worked that out).

No easy answers I'm afraid...

https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=13cd0c13e221defea40116e3802aab0b34fa2...

[#3]

Hi @MRB3855 ,

Thanks for your answer!

For clarification, it is the same units tested at each time-point. As I understand, your explanation is referring to when different units are tested, correct?

Is there a way to simply the study? For example, if I remove the linear fitting and I measure only two time-points, is there a way to include the measurement error in the statistical analysis?

[#4]

Hi @bbenny7 : Yes, my explanation was for different units.

Yes, your case can be simplified to a paired t-test. Each of n units tested at 0 months, then at three months.

H0: mean delta (3 months - 0 months) > 1.25

Ha: mean delta <=1.25.

The way the data analysis will proceed is as follows (this will help to guide us to what information we need for power calculations.

1. For each unit, calculate delta (3 months - 0 months).

2. Treat these n deltas as your new data. This reduces the problem to a one sample test.

3. From these n deltas, calculate a 95% upper bound on the mean (JMP Distribution Platform).  If that upper bound is < 1.25 then you reject H0 in favor of Ha at 95% confidence (you could choose another level of confidence, but 95% is common). 

So, to use the "power for one sample equivalence" platform,  choose: 

-Non-inferiority

-Lower is better

-Margin 1.25

-Alpha=0.05

-Population SD assumed known? No

This sets up the desired hypothesis test:

H0: mean delta (3 months - 0 months) > 1.25

Ha: mean delta <=1.25

In the bottom section is Sample Size, Difference to detect, and Std Dev.

The Difference to detect is what you believe to be true in the entire population of units (must be < 1.25 since otherwise there'd be no point in doing the experiment).

The Std Dev can be a tricky one. It is the assumed Std Dev of the deltas. Or, since delta = Y3 - Y0, the Std Dev of the deltas = sqrt( Var[Y30] + Var[Y0] - 2*correlation[Y30, Y0]*SD[Y30]*SD[Y0] )

Soooo, either you have to know the (1) SD of the deltas, or the (2) SD of each and the correlation coefficient.  If you don't know correlation, 0.5 might not be a bad choice. 

Questions? Come back

[#5]

Hi @MRB3855 ,

How would the measurement error fit in this analysis?

I have estimated a measurement error standard deviation of 2.5; is this value too large compared to the difference I want to detect?

[#6]

Hi @bbenny7 : Exactly what do you mean by "measurement error"?  Do you mean the expected SD of your data at each timepoint is 2.5?  If so, then using the formula in my last response,

SD(delta) =   sqrt( Var[Y30] + Var[Y0] - 2*correlation[Y30, Y0]*SD[Y30]*SD[Y0] )

                =   sqrt(2.5^2 + 2.5^2 - 2*0.5*2.5*2.5) = 2.5.  So that is your Std Dev.

Or, is measurement error defined some other way? 

Either way, to use the formula above you will need to know (assume) the value of the SD at each timepoint. Presumably, you have some data to get an idea of this?

Your question, "Is this value too large compared to the difference I want to detect?" Not necessarily; from a purely stat perspective, you'll just need a large enough sample size.  However, from a resource/cost/time perspective, the required sample size may be prohibitive.  

[#7]

You need 25 samples, each measured once per time point.

View more...

Yes, using the power explorer is appropriate for estimating sample size, as it helps determine if you can detect the desired change with sufficient power. For your case, you need 25 samples, not 25 measurements per sample. You would measure each sample once at each time point, totalling 25 samples measured over the 3-month period.

[#8]

Hi @DawsonCade : n=25? Based on what assumptions?