Thank you for using our script. I am glad that it is useful. One of the many reasons for the File Exchange is to provide useful tools that supplement JMP. If you want this outlier test built into JMP, then visit the JMP Wish List here in the JMP Community and request it. You might search for it first (I have not) and if you find such a request already submitted, then vote for it to boost its chances.
I was not aware of the paper you cited (Rorabacher 1991) so it was not the basis for my script. I used one of the sources he cited, though. I used Bartlett and Lewis (1994), albeit a newer edition than Rorabacher used. I agree with him that such tests are important and can be useful. I wish that more scientists were taught what they mean and how to properly use them. I generally see behavior that some have called "p-value fishing" where tests are applied until one of them is "statistically significant." This behavior is especially rampant in the of outlier tests.
The tests have been developed and re-visited for many decades. Improvements in computing techniques and statistical theory benefit older methods so we all benefit. The tests are objectively based but they rely on subjective choices:
- What is the minimum size of a sample to reasonably test for outliers?
- What risk of a type I error and type II error am I willing to take?
- How well are the assumptions of the test met?
- How well can I determine how well the assumptions are met?
- Which test should I use? That is, what is my belief about the specific contamination process at work on these data?
As to your specific issues, you said, "The first issue is that the critical values in the script seem to refer to the one-tailed test, or actually CL of 90% when choosing the 0.05 option. Please refer to Rorabacher for the list of critical values, especially for the case of 95% CL."
I carefully reviewed the script and confirmed that it is using the correct set of critical values for both 95% and 99% confidence. I further tested published examples that agreed with the result from the script.
Also, note that only the first test provided by the script ("Upper or Lower Outlier") is correctly applied as a two-sided test. All the other tests are one-sided, so doubling alpha for these tests or using a two-sided confidence interval is non-sense. Rorabacher seems to make this mistake on page 141. It is important to be clear about the null and alternative hypotheses in any test or construction of any confidence interval.
You said, "In the enclosed example, the value 11.66 is clearly an outlier."
I wish it were that simple and clear. In my years as an analytical chemist, I saw many cases in which I wanted to call an observation an outlier, but based on only 3 observations, such a call was unwarranted. How do I know that the two negative values are not the outliers? Random variation such as seen in a sample from a population can provide surprising but still valid observations. Prior knowledge about the population could be used to augment the test but these tests do not use information external to the sample.
The use of a critical value also creates the illusion that an observation is an outlier or it is not an outlier. The fact is that the greater the distance used in any of these tests, the stronger the evidence. It is a continuous and non-linear change in belief.
You said, "The critical value that the script shows is Q0=1 where in fact it should be 0.971 and when compared to the test result Q=0.98 should indicate on an outlier."
Rorabacher points out how different critical values have appeared over the decades, so I would not say that, "in fact, it should be 0.971." Each author presented an honest effort to develop critical values that were based on the subjective choices such as those mentioned above. That value 0.971 represents another subjective choice about these tests. I stand by my choice of the source of the q0 values that I use in the script. But you are welcome to modify the script if you prefer Rorabacher's values instead. One of the beauties of a script is that you do not have to wait for JMP developers to add or enhance the feature.
