A Cringeworthy Statistic Statement: #2 in a series

[#1]

Thanks again to all who joined in my first "Cringeworthy" discussion topic.  Very interesting discussion

points were contributed!  I have published a blog post on why I cringe when I hear that first statement.  

Here is a second scenario and statement that makes me cringe:  

A company experienced a problem with yield on a production line.  The working hypothesis was that it was related to a difference in subassemblies received from two different suppliers.  An engineer is asked to do a t-test, from which the p-value comes back as 0.001.  The manager immediately says "That is more than 95% significant!  Go figure out why they are different.  We have to get these two suppliers to be providing the same parts!"

What is wrong with the manager's thinking?

JMP #Cringeworthy

[#2]

Excellent point, @markschwab !  Thanks for the reply!

[#3]

Thanks to all that replied to this discussion thread.  (And if you are just now seeing it, feel free to continue the discussion!)

If you are interested in my take on why I cringe when I hear things like this, see my latest blog post.

And if you liked this one, check out my 3rd "cringeworthy" statement discussion!

[#4]

Perhaps I’m missing something, it seems the manager is equating significantly different yields with the sub-assemblies being different. As has been pointed out, there are many factors that affect yield which have nothing to do with form, fit, or functional differences in the assemblies.

[#5]

Thanks for the reply, @MarkDayton .  Yes, my thought when writing this statement was to point out that people jump at a "Significantly Different" result without thinking through what that means, whether it is important, etc.