Seeking input on how to define limits for comparability testing. Challenge: Small number of pre-change lots.

[#1]

Hi,

I've dug through the forum and some other sites and am having some difficulty finding clear guidance.  Anything that the JMP community can offer here would be appreciated.

I need to do a comparability assessment on pre vs post change product lots (release data).  I'll be testing for Equivalence but am unsure of the best approach for defining equivalence margins for a few attributes.  There are only 4 pre-change lots.  I've seen different advice on using tolerance intervals vs. a 3SD approach.  

What have people here done when in similar situations?  Any other approaches that I should consider?

Thanks in advance.

[#2]

Hi @ClusterFerret68 : Defining equivalence limits can be very challenging.  Strictly speaking, it is not a calculation and it really isn't even a statistical issue; it is a scientific issue.

Here is a discussion you may find helpful.

https://community.jmp.com/t5/Discussions/calculate-the-Margin-of-Difference-for-the-TOST/m-p/845690

[#3]

Each comparability scenario requires different questions to be asked, most importantly will you be using an equivalence test or a quality range to establish comparability. The answer to that depends on the criticality of the evaluation and sample sizes. For quality ranges, it is typical to use a reference interval (mean +/- 3 standard deviations for example). There are lots of references for this. For equivalence testing, it will be more involved to determine your EACs. Here is a good paper that explains:

Feel free to reach out for clarification, thanks, Heath