Paired T-Test with 3 Processes and Multiple Response

[#1]

Hi Everybody, 

I ran an experiment where the goal was to compare whether three separate processes produced similar results. I made 6 different products (with different specifications) on each of the three processes (6 products x 3 processes = 18 total). Additionally, I have at least two different responses I want to compare the output of. I am almost certain this data is paired, but I don't know if I can run a paired T-test or Anova on these. 

On idea I had was to subtract the differences from each permutation and then do an Anova. Alternatively, I could run 3 different paired T-tests as well, but I'm not sure if this is the correct way to do it. 

Thanks for the help!

[#2]

Hi @tnorthen,

A paired t-test is a comparison of two treatments on blocks (products). When there are more than two treatments, this is typically run and analyzed as "randomized block" design. If I understand your question correctly your data is this type of design. This type of design can be analyzed with Fit Y by X with Product as a block factor and Process as a treatment (x) factor. If the ANOVA test (comparison of between treatments is larger than expected random product effects (within treatment group) then a multiple comparison test (I like Tukey-Kramer HSD) is run to estimate/quantify the differences in the treatments.  

This is a high level description that hopefully gives you some leads to investigate.  I have attached a tricked up test table. The table has the analysis script embedded.  Note there are many ways the analysis can be biased in the way the data was generated. For example, suppose each process involves different equipment. This begs the question whether any observed differences are due to the process or due to the equipment? If the same equipment was used, were the processes run in random order over the different products?  Without going into details, process order, cross contamination, etc could influence results.  I suggest if possible find someone in your org that might be familiar with these methods to work thru the analyses and details of your data collection.

[#3]

I do not think that a paired test is appropriate. How would you pair observations?

You should not use ANOVA because it is based on a hypothesis test for a difference. You should instead use a test of equivalence. This test is available in the Oneway platform. Each product, with its own specification, must be tested separately. How many replicates do you have for each product from each of the three processes?

I mocked up your situation: 3 processes, 6 products, and a sample size of 5 for each treatment. The equivalence test in Oneway produces this report for each product I used a normally distributed response (mean 50, standard deviation 5) for all treatments and a practical difference of 5 to illustrate this approach..

[#4]

I attached the data table with the mock up of your data and a table script to launch the appropriate Oneway platform.

[#5]

Wow! These are fantastic replies! Thank you all so much!