how small? (3, 30 or 300) and how big of a shift is meaningful?
Typical one stability study (3 - 10). The meaningfulness of the shift is
debatable but I am trying to adhere to industry standards for significance
which I believe is a Confidence Interval of 0.95 or alpha being 0.05.
I think the conclusion is going to depend a lot on the type of question you are asking as well as the analysis that you use. Are you trying to determine if a point is out of control or out of trend, or maybe different from another group?
I am attempting to identify out of trend results and write a robust
procedure utilizing JMP to maintain the data and save a script analysis for
ease of use. From what I have read out of trend can be characterized many
ways but typically revolves around being identified as a statistically
significant result from the hypothesis (in this case the population mean).
If you expect that the mean and variation will be constant from measurement to measurement (along the trend), then control charts like the IR chart in JMP (also Control Chart Builder) will be useful for identifying when the process you are measuring drifts out of control. Keep in mind that this isn't a hypothesis test.
If you're doing a stability study, the Degradation platform (Analyze/Reliability) will be useful in determining when your sample is likely to degrade below a spec limit.
If you are trying to compare one group of samples to another, the Fit Y by X platform (Analyze menu) tab is the place to go to do an ANOVA.
If you know the analysis you want to do, but can't find it, take a look at the JMP Starter Menu (under the view tab). This is a convenient place to find lots of types of analysis.
Hope this helps,