MSA results JMP vs Minitab

[#1]

@Martin Demel

Dear JMP community, maybe some of you can help me out.

I am routinely using Minitab. Now I am supporting a friend who is using JMP in his company and has asked me to explain to his team how to do the procedures in JMP which I explained in Minitab.

First subject I am struggling with is MSA (in JMP Variability Study).

I have an example with strong appraiser*parts interaction, since somehow sample assignment was mixed up for 3 samples.

No matter whether I remove that mixup or not I observe the following scheme:

All GRR contributions (based on 6*stdev) are smaller in JMP, with the exception of the appraiser*parts interaction, which is bigger in JMP.

Repeatibility seems almost identical.

Now people are obviously asking me, which software is right, and I have not answer.

Minitab in this example is getting negative variance for the appraiser contribution, which they then set to zero (feedback by their support).

But that explains only part of the problem. The biggest difference seems to be in the interaction part.

Can somebody explain to me, why the two are different? And maybe the JMP team can explain why they think they are doing it right.
Excel file with sample data is attached, with evaluation results in a separate worksheet.

I am using JMP 16.1 and Minitab 21.

[#2 · Accepted solution]

Welcome to the community.  You are asking a question about Minitab and we may not have that software to determine what is being done differently, but I will say there are multiple ways to estimate components of variation and some software programs differ on what they use as a default.  My guess is Minitab uses EMS and does not employ REML (Restricted Maximum Likelihood) or Bayesian estimates when negative components are estimated.  I would start by understanding this:

https://www.jmp.com/support/help/en/16.2/?os=mac&source=application&utm_source=helpmenu&utm_medium=a...

About REML:

https://www.jmp.com/support/help/en/16.2/?os=mac&source=application&utm_source=helpmenu&utm_medium=a...

Also realize all of the estimates are ESTIMATES, none are necessarily "correct".  Do the statistics change the interpretation of your analysis (which components are bigger, et. al.)?  I would also look into Wheeler's EMP methodology:

https://www.jmp.com/support/help/en/16.2/?os=mac&source=application&utm_source=helpmenu&utm_medium=a...

[#3 · Accepted solution]

Adding a little bit of information to @statman 's excellent information. To see your options when you Choose Variability / Attribute Gauge Chart, click the Analysis Settings button in the lower left of the dialog box. You will see that JMP offers several ways to calculate the variance components just as @statman said. The radio button is set at "best". You will see that you cannot force JMP to use EMS, only a choice of EMS or REML. The EMS approach is older and negative variance components are possible. So, REML will help avoid that situation. Just my opinion, but I would rather use REML to try and avoid negative variance components. 

If you truly want to use EMS, you could always use Fit Model in JMP. Specify your model and make sure you declare your factors as random effects. When you do, you should get a dropdown box for Method. The default is REML, but you can change that to EMS (traditional). When you run the model it will most likely match the Minitab output.

[#4]

Thank you @statman and @Dan_Obermiller .
I was suspecting something in that direction. Information from Minitab is, that they are using the methods of AIAG MSA4.
I am not deeply enough into statistics to know which of the above model types it is. But I would consider it helpful, if following AIAG MSA standards - which is a very widely accepted de facto standard - were available at least as an option.

And sure, I am aware that all the data are estimates and that small differences are always possible and have to be accepted.
Thank you again for your fast response.

Hermann Hülsing

[#5]

The AIAG (Automotive Industry Action Group) standard was originally intended for the automotive industry.  It is not a standard writing body (e.g., ANSI, ISO, IEC).  If you get a chance to read the MSA document, you will find multiple options for calculating components of variation, not one.  There is even a reference to the EMP methodology.

[#6]

Adding to @statman's reply, other industries disagree with the AIAG standard, for example, the semiconductor industry. They found better definitions for MSA for their purposes. Please use the standard that is expected by your industry, but understand that there are differences of opinion and different interpretations across industries.

[#7 · Accepted solution]

That much about sitting on an island... (see my response to @statman )

[#8 · Accepted solution]

I am aware, that AIAG is not a national or international standards writing body, like ISO or ANSI. That is why I said "de facto".

That is because they do write industry standards, like e.g. the European Computer Manufacturer Association ECMA, where I was involved.

In 40 years of working in Quality Management or Consulting I have never run into anyone using Wheeler's EMP method.
But I also have to admit that working in a certain industry is often like sitting on an island...

Anyway, following your advice I shall check the AIAG MSA document again.
And Minitab is indeed using the EMS method. It has been confirmed by them. So at least it is clear, why there are differences.

That is what I wanted.

Thanks for your support. I was impressed about its speed and clarity.