I'm dealing with very large datasets in which we have conducted many different studies (study_id) where we compare various 'test_article' vs a control (ie PBS). This is done in multiple replicates (subject_id). For each of subject_id, in each of those studies, we are making measurement for up 50000 different type of measurement (Veggies). The result of each measurement is reported as a log2 (of the measurement of a given subject normalized by the mean of the control group from the same study, but the nature of the measurement is not that important, I think).
On a small dataset, I would run a One-way anova (one per study) for each veggy to compare the effect of treatment (test_article) and follow that up with a Dunnett's. On a large data, it seems to become intractable for 2 reason (multiple measurement - so FDR becomes a requisite) and unable (ie don't know how) to only report the p-value and bypassing the graphical phase( which is not practical for that many tests).
The 'response screening' seems to be a slightly better alternative but is missing the option to define a reference control group as I am not interested in knowing if test-article A is better than B but only if any of the test_article are different from the control.
Finally, I would like to be able code the process for reproducibility.
Hi Sebastien, Your example seems to call for a repeated measure mixed linear model. There is a really good Add-In (https://community.jmp.com/t5/JMP-Add-Ins/Full-Factorial-Repeated-Measures-ANOVA-Add-In/ta-p/23904) that should let you design the right combination of factors taking into account that you are evaluating the same subject across different conditions. Of note: the Within Subject Factors are those that span across Subjects such as Sampling Time. In contrast, the Between Subject Factors are those that are only found for a specific Subject such as Treatment or Experiment.