Hi, I'm Jerry Fish. I'm a support engineer with JMP, helping customers in the central part of the United States. Today's talk is entitled My Gauge Isn't as Good as It Could Be— Will Its Errors Cost us Money and how much?
I'm Jason Wiggins, also a senior systems engineer, and I support semiconductor users in the Western United States. This talk is a follow on to one we did for discovery Americas in 2022. In our first talk, we introduced the notion that measurement systems are integral to our businesses. In fact, we have many measurement systems we interact with in our daily lives. Along with that idea, we introduced the notion that measurement system or gauge variation can impact decisions in real world inspection situations. We introduced gauge performance curves as a way of visualizing gauge variation and relative to specification limits. In this talk, we'll extend that and explore the costs associated with gauge variation through a fun role play conversation between a quality manager of an automobile manufacturing plant, that'll be Jerry, and I'll be acting as a quality consultant. To kick things off, I'll get on a quick team call with Jerry. Hi, Jerry.
Hi, Jason. How are you doing?
I'm doing pretty good. Thanks for spending a few minutes with me. As a quality consultant, I help quality stakeholders like yourself understand and improve processes. Now, I prefer using JMP as it's a general purpose, easy to use data analytics package that has many quality and process control features. JMP makes quick work of the analytics part of process improvement, so more time can be dedicated to actually improving the process.
Well, it is nice to meet you, Jason. Just to let you know, though, we already have software in place for our internal quality programs, so I'm not really sure what your software can do that we cannot already do. Can we make this quick?
I understand completely, Jerry. I'll try to make the most of your time today. First, can you tell me a little bit about your company and your quality program?
Sure, happy to. Acme Motors has built a reputation with our customers of manufacturing the highest quality cars. We're always concerned with quality. We have various gauges that we use to ensure our quality stays high. We've been doing this for years, and frankly, we think we're pretty good at it.
I'm familiar w ith Acme Motors and your high quality reputation. My consulting team and I have recently been working with manufacturing companies like yours to advance the use and effectiveness of gauge studies. Measurement systems analysis, another way of saying that. One of the things we seek to understand are the monetary costs associated with the gauges used to measure process quality characteristics in your manufacturing plant. Have you quantified how much any of your gauges are costing your business?
I'm not sure what you mean.
Well, gauges are not perfect. They make mistakes. Sometimes they'll throw away good parts and sometimes they'll pass bad parts. Unless you have a perfect gauge and really no one has these, these mistakes are inevitable.
I suppose so, but we've done gauge studies that say our gauges are good. Well, some of them are actually categorized as adequate by the AI AG guidelines. Doesn't that mean we're okay to use them?
Well, possibly, but there is a lot more to the story than just using good, adequate, and poor AI AG gauge assessment criteria. For example, have you seen a gauge performance curve?
I can't say that I have. No.
Now, this is what one looks like. The X axis shows the true part values and the lower and upper specification limits are shown with these lines. The Y axis shows the probability of passing a part. If you have a part that is truly good but very close to the lower spec limit, there's almost a 50 % chance the gauge will recommend that you throw it away. That's one way of thinking about it. But also, there's nearly a 50 % chance that you will accept a part that is truly bad and near the lower spec.
Very interesting. What happens if we could change the variation of the gauge then?
Well, the shape of this curve definitely depends on how good your gauge is. Let's play with this just a little bit. What if we could reduce the variation by a factor of 10? Make a quick change here and replot our gauge performance curve. If this is possible, we will correctly accept or reject more of the parts. We're moving from incorrect to correct when we do this. Let me break from the role play for just a moment. The gauge performance curve I am showing is an add in that we made for our 2022 Discovery Americas presentation. The add in is available on the community. Back to you, Jerry.
That is really an interesting chart, Jason. I don't think we do anything like that. What you're saying is that the gauge errors contaminate the measurements, but all I have is the imperfect measurement. Your gauge performance curve is plotted versus true part values. I wish I knew those true part values, then I could know exactly which parts to keep and which to throw away. Is there a way I can know the true part value?
We could know that directly if we had that ever elusive perfect gauge, which we don't. We really can't ever get to the level of knowing the true value of an individual part, but we can estimate the true part distribution given our knowledge of the gauge characteristics and the measured part distribution.
How would you do that?
Well, if we assume that gauge errors are normally distributed, and for the moment, let's ignore any bias or linearity problems that you might have, and that we have the measured part distribution. If we have that, we can back out the variance of the true part distribution using a simple equation. That simple equation is just the difference between the measured part variance and the gauge variance. The plot on the right is shown for a situation where the measured variance is 25, the gauge variance is 16, and if we subtract those two, the true part variance is nine. We're beginning to get the parameters for that distribution because we know the results of our gauge study and we understand the variance associated with our gauge.
Now, we would, from this, build a normal distribution that centered on the measured distribution mean with the new standard deviation. A gain, the result of which is going to look like the plot on the right. In the plot, the blue bars represent your measured part distribution. The areas above and below the spec are are shadeded in pink. Notice that the measured part distribution is much wider than the true part distribution. The measured distribution is what you get when you run the true part distribution through your imperfect gauge.
Okay, that makes sense, at least a simple case. How do you relate this to what it's costing my company?
Well, we can use this information in a numeric simulation to characterize the mistakes that our gauge is going to make. When we do that, we can generate a part inspection table like this.
Let me study this table for a minute. My eye is immediately drawn to the center, the green box that says 95.4 %. Am I interpreting this right? 95.4 % of my total production is truly good and we're shipping it.
Correct.
That's a good thing. Now, looking at the first and last columns, if I add 18 and 25, let's see, that's about 0.043 % of my production parts are truly low parts. A nother 0.041 % on the last column are truly high. This is bad. It says my process is making bad parts that must be thrown away or reworked. I see another problem. If I look at that center column and I add those all together, I get 99.9 % of my production parts that truly are good. It says that the gauge is identifying 2.3 % of those as too low and 2.3 % is too high as well. Now, the customer doesn't care about that. They're still getting good parts, b ut I certainly do. I'm making good product and I'm throwing it away. Worst yet, look at that center row, those red squares. There's another 0.036, 18 and 18, of truly bad parts, parts that are too low or too high that are being accepted by this measurement gauge. This is serious. I do not want to ship bad parts to my customer if I can help it.
That's right. This is cool. You're beginning to see the cost of having an imperfect gauge.
This is really interesting. It shows that if I don't do something about my imperfect gauge, I'll risk accepting bad parts and throwing away good parts, both of which are bad for my business. On the other hand, I think we've got a way to handle this, Jason.
Okay, what's that, Jerry?
Well, we use something called guard bands. These are our bands that are set inside the specification limits. If we set them far enough inside the spec limits, we can reduce and essentially eliminate shipping bad parts. Doesn't that fix at least part of our problem?
At least, guard bands are definitely a good way to reduce the percentage of bad parts that make it through your inspection process. A lot of companies use them. Have you considered the fact that improving quality using guard bands comes at the expense of throwing away good parts?
Honestly, that has occurred to us, but we haven't tried to quantify that damage.
Well, let's extend this example out a little bit more and let's just assume that we bring those specifications in by one unit of measure. We'll call these guard band limits. Our lower guard band limit would be 41 and our upper guard band limit would be 59. We're going to use this as our inspection screening values instead of the original upper and lower specs. Now, we can do the same numerical simulation and update the results. Let's just take a look at the differences between the tables. Can you see how the percentages have changed?
We went from shipping roughly 0.04 % of parts that were truly bad to only shipping 0.03 % of bad parts. That looks successful. Maybe we could even squeeze our guard bands in further and improve that. Especially given our high production volume. We're talking real bucks here.
It is. Also notice how many truly good parts are now being screened out. Every time you screen out and throw away good part, it is costing your company money.
Well, you're right about that, Jason. Is there a way to look at this monetarily? What if we assume that a bad part in the simulation results in a bad car? Can we input the cost of scrapping the car and see how that affects the bottom line?
Absolutely, we can do that. I'll need to get a little information from you, though. First, how much does it cost to make the car?
Yeah, let's say for the sake of this demonstration, $35,000.
Okay, great. That means for each rejected car, it costs your company $35,000. You might manufacture in rework costs here, but let's say, for example, we just throw the car away. Now we need production quantity.
I don't know. Let's just choose a million cars.
Okay, great. Now, how much do you charge for a truly good car that makes it to a dealership?
The dealerships buy... Let's just say they buy these cars from us for $40,000.
Okay. If I understand this right, your profit per car is that 40K minus the 35 K and your profit has been $5,000 per car.
Right.
Last thing, do you know the cost associated with selling a bad car?
That's a little tougher. There are the obvious costs of repairs to the bad car or potential cost of return. Those are relatively easy to calculate, but there's also damage to our reputation. Our customers demand quality, and if we start putting bad product out the door, it can quickly get out of hand and result in lost future sales. That's a lot more difficult to calculate. I know you need the number. For the sake of argument, let's just say that totals to $50,000 per bad car that makes it out into the market.
Excellent. Let's take a look at the profits and losses. Same simulation. Just review, make sure that we're looking at the correct values. You told me that manufacturing cost per car is $35,000. You then sell that to a dealer for $40,000. Our profit is $5,000. Cost of selling a bad car is $50,000. We're going to look at this across a 1 million car production run. Have I captured e verything, right?
I think that looks good.
All right. If we look at the net profits and losses, you stand to make about 346 billion from the 1 million cars you make.
That sounds good.
Not bad. The total profit from the truly good cars that are shipped is about 371 billion. The loss due to making truly bad cars that are caught in your inspection is 199 million, which is the sum of 98 million plus 101 million.
Okay.
The law... I let you digest for a second?
Yeah, I'm following.
Okay. The laws due to shipping truly bad cars, this is the one you are really concerned about, is 137 million, which is 68 million plus 69 million. Finally, the loss from scrapping truly good cars, this is what's costing your business, is $25 billion. That's quite a lot. That's the sum of $12.4 million and 12.4 million.
That's fascinating and also a little depressing that we're losing that much money. If you change things, let's say you change the guard band settings, will the total net profit change?
That's right. That's definitely true. You could see that change.
In that case, then could there be an optimum? I can imagine widening the guard bands or narrowing them and looking at the net profit, would there be an optimum for that net profit peaks out?
Yes, you can definitely explore that trade- off in a lot of different ways. You could answer questions like, how would improving my gauge by a factor of 10, like we showed with the gauge performance curve, improve my profitability? Or how much can I afford to spend on fixing or replacing a gauge? If we know what the costs to the company are for our measurement system, then we can justify the cost of fixing or replacing gauge. Also, just to your point, what if I adjusted my guard bands? We can definitely answer that question. A nother common one is what if I improve my process capability? I just tighten the variation in my process, what does that do to my profits and losses?
I could trade that off against the cost of improving that process capability. Interesting. Well, I must say, Jason, I'm impressed. This has been a good use of time, but I think I owe it to my company to muddy the waters just a little bit. This is all great for normal distributions and simple gauge errors and those kinds of things. Those calculations that you've shown are easy. But what if I have gauge linearity or bias problems? Or what if I have a skewed distribution, which is really pretty typical in my company. We rarely run into the nice bell shape curve. Getting a true part distribution out of the measured part distribution becomes a lot more difficult than just using that simple formula you showed earlier. Can you even can do that?
Absolutely. We are writing an add- in that will make you able to define the shape of any measured part distribution. W e can do the same exercise with measured part distributions that are normal or log- normal, uniform, Weibull, or even a custom distribution. It's an add- in we're working on. It's a work in progress.
All right, that's fantastic. I'm ready to buy in. When will that be available?
We have the basics of the add in worked out, but we need some time to make it more user friendly. We'll be working on that in the coming few months. Probably before midyear, we'll have that wrapped up. When it's done, we'll post it on the JMP website in our community file exchange.
A few months, really? I'll forget all this by then.
That's okay. We recognize that. Once our ad in is ready for prime time, we'll announce a series of open to the public seminars where we will go into detail about what you've seen here, as well as other aspects like relating these concepts to Donald Wheeler's EMP methodology, which is another personality in the Measurement Systems Analysis platform in JMP. Here's a quick peek at the topics for the up and coming talks.
We'll spend more time elaborating on how gauge studies that are using the AI AG classification that we talked about earlier, we'll talk about how that can lead to unrealistic gauge assessments. We'll also explore how Wheeler's Evaluating the Measurement Process, the EMP method, can provide us more realistic gauge classification. We're going to present the problem with AIG and present the solution using Wheeler's methods. We'll also show how EMP method can advise us on how to use our gauge. How do we use it in the production process? One example that we'll be covering is objectively setting guard bends. The remaining topics, hey, we'll spend a little bit more time interpreting gauge performance curves, talk about how to blend performance with part variation to determine cost associated with imperfect gauges.
Really, that is what we're talking about today, but we feel like we need to extend that a little bit so that we all understand how that works. Final two topics, how can Wheeler's calculations be factored into this gauge cost conversation, and how to understand gauge cost, again, to the point of non- normal part distributions.
That's perfect. Can you make sure that I'm on that invitation list? I want to make sure that everyone in my quality department attends your seminars.
Sure thing, Jerry. Anything else I can do for you today?
Yes. Get back to work on that ad- in. The sooner it's available, the better.
Will do. All right, this concludes our presentation. I'll say that as we were doing research for this talk, we uncovered many concepts that are important to understanding how to use measurement systems. We feel like one of these concepts deserve more time than we had in our talk today. We look forward to continuing the conversation with you in the coming months. Any closing thoughts, Jarry?
Just that if you've kind folks that are attending today, if you're interested in attending those upcoming seminars, please let your local JMP support person know or your support team know, and they'll make sure that you're included on that invitation list.
Excellent. With that, thank you, everyone, for attending.
Thanks, all.