Different REML Variance Component results for different order of same factors

[#1]

Hello, 

I designed a study with 5 factors (week, lot, etc.) for two different labs, and one of the labs followed the design out of order. They still completed all the parts of the study and the design is still balanced, they just switched the order of the design for weeks 2 and 3. I am analyzing the data for both labs combined. So lab 1 ran the design in order of weeks 1-2-3-4, but lab 2 ran the design in order of weeks 1-3-2-4 (per the planned design). I analyzed the data in 2 ways: once keeping the true week numbers that were performed, and once overwriting the lab 2 data to follow the design (swapping the data for weeks 2 and 3, so it looks like they ran the design in the correct 1-2-3-4 order). I did not think this switch would have an impact on the REML variance component analysis results, since all parts were still completed and the design is still balanced, but the variance component estimates and percentages are slightly different between the two analyses. 

Can someone explain to me why the variance component estimates would be different when I'm just switching the order of the weeks? Isn't the variability attributable to week assessed in the same way? 

Thank you!

[#2]

Without seeing the data and the complete specification of the model you are trying to fit, this is difficult to answer.  

Based on the information you have provided, this may be due to whether or not the "Week" factor is specified as nested or not.  If it is not specified as nested, then the week factor is assumed to be shared by by labs.  Nesting implies that Week 1 in Lab 1 is not the same as Week 1 in Lab 2, for instance.   Week 1 could have a different effect on Lab1 than Week 1 has on Lab 2.

Without nesting you are assuming that Week 1 has the same effect on Lab1 and Lab2.  That is, you are estimating a common week-to-week variability across both labs.   

It is probably reasonable think of the week levels in Lab 1 as being different than the week levels in Lab 2, but maybe not if there is some common lurking factor that impacts both labs each week in a similar way.  

[#3]

I do not have week as nested. I am running the REML analysis in JMP with all 5 factors as random. I think I have a hard time determining what should be nested or not. Attached is the planned study design. Should week be nested within lab? 

[#4]

Actually, this design is more accurate, as the labs used different lots, operators, and equipment.

[#5]

Then Lot, Operator, and Equipment are nested within Lab

[#6]

Thank you! From this discussion and my own reading in the meantime, I believe this (image attached) is the correct way to run the variance component analysis. Site is the Lab variable, and 1 run was performed for each combination of factors shown in the design. Am I on the right track? 

[#7]

There is precious little information to determine if you are on the right track or not; but (and if I understand correctly), if 1 "run" was performed for each combination, what is the Run[Site, Week, Lot, Operator, Equipment] effect?  i.e., it is variability between Runs?  If so, and there is only 1 run per combination, then how can it be variability between runs? And, if you don't mind me asking, why is site random? If you are trying to compare those two particular sites only, then it is not random. And, if it is random...then estimating a variance component with n=2 sites is ill-advised.   

[#8]

I would remove the Run term from the model. The Run will implicitly be used to estimate the repeatability. This change will address @MRB3855 observation that you need more than one run for each combination to estimate the variance.

[#9]

Could have to do with which terms are crossed or nested?