Stop Missing Out: Reliability Tools Aren't Just for Reliability Engineers (2026-US-PO-2905)

Hi, thanks for taking a look at this presentation. Today, I want to talk a little bit about democratizing reliability and that word and what it means and really help us all to understand that we have a lot more use for reliability tools than we otherwise might think. I'm going to zoom in here a little bit just to introduce this. Maybe this is projection just from my experience, but maybe this is also something that I've observed, but I think a lot of people tend to think of the reliability menu in JMP as something that's just for catastrophic failures. I'm sorry, let me zoom back in here. I forgot you can't do the laser at the same time, but reliability tools are just for catastrophic failures and things like product testing labs, or really, if I don't have reliability in my title, then it's just not my responsibility.

Frankly, there are a lot of different menu options in here that can be a little bit daunting, but I think if we just shift our thinking a little bit, and we realize that we already have data that exactly fits many of these tools, then we can start applying these tools a lot more.

Today, I want us to recognize that reliability problems are everywhere. I'll talk about how to recognize those. The key is recognizing that your data relates to time and frequency. I'll talk a little bit about the traditional examples so we can wrap our heads around it, but I want to actually talk more about less traditional examples just so that we can start thinking about using these tools even if we don't have reliability in our title.

That being said, I've got a couple of case studies that we'll go through, but first we're going to talk about recognizing that reliability data. Let me just go there, and so there is this question: how do I know when I have reliability data? Two ways to think about this. The first way is recognizing that your data has some time component, whether it's time to an event, like time to complete a process or time to finish a polishing operation, something like that, or time between events. That might be something like, we might use the word frequency as well. Time between equipment breakdowns. That's reliability data and ordered in obvious to maybe less obvious here, I've got some examples of what this means.

The blue ones here are things that we'll have case studies for. Starting with the obvious, when a lot of us think of reliability, we think of testing samples of some product to failure, recording the number of cycles, or each time I was able to push the start button and the machine didn't blow up, recording the number of times I was able to do that before failing. Maybe something else that's fairly obvious or should be is finding shelf expiration times for either pharmaceuticals, food, or knowing when the milk is going to expire. That's a reliability problem.

Maybe we start to get a little less obvious when we say, how long should my calibration cycles be on a measurement device that I'm using in production? That's a reliability problem, or maybe if we're starting to get into the actually less obvious would be time between a process going out of statistical control, so the frequency of a process going out of statistical control or requiring maintenance or just having some general error. Maybe also we're doing a DOE or designing an experiment that builds a model to reduce the cycle time of some process, maybe a robotic arm.

Because we're talking about cycle time, that can be analyzed with reliability tools. Maybe we're modeling employee retention and want to see factors that contributed to an employee exit. Turns out that's reliability as well, or maybe we recognize a problem in our business, and we change something in our business because we're losing a lot of customers at a rate; there's that keyword rate or frequency, and we want to see if we've made a change to that. That's the first way.

The second way would be if our data doesn't fit this normal distribution, this nice bell-shaped pile that we expect to see quite often, but instead it follows one of these other distributions. In fact, these distributions often show up when we're dealing with things that relate to time or risk. Using the word risk here, but maybe frequency of something happening. If the frequency of something happens and that changes over time, that would probably fit a Weibull distribution. By the way, I hesitated to put the Weibull distribution here. Weibull distributions can take all sorts of shapes, so don't think that it has to look exactly like this one in order to be Weibull.

Maybe think of it more in terms of what's going on. A classic example from reliability is cracks in a substrate propagate, and then all of a sudden the failure rate changes at some point because the cracks become too big, and then the whole product or the whole substrate fails, but maybe employee turnover is a little less obvious way to think about this. This would be things get annoying, and then all of a sudden the employee says they've had enough, and that causes employee turnover or retirement. It doesn't have to be a bad thing for employee turnover either.

If we switch and think about log-normal, this is where the failure risk accumulates, or the frequency just accumulates over time. This might be in traditional reliability, something like oxides attacking a material and just breaking it down slowly. That rate doesn't change; it just accumulates, and then the product fails. This could be customer churn from numerous small frustrations, or we talk about exponential, and this would be when the risk or the frequency is just very, very constant, and we could have the product just fails for no other reason than it just fails. Less obvious than a component failing in an electronics product, this could be time between cybersecurity events or time between calls in a call center.

There are a lot of ways that we can think about this. If we recognize that we have data like that, then we can start making some better decisions with our data. I've got a couple of case studies, and the first one here is talking about our Old Faithful Furnace. We can pretend that we work at some manufacturing company. We'll use Red Triangle because JMP has red triangles, but we can pretend that we work there and everybody knows that Furnace 2, Old Faithful, Old Blue, is more reliable than furnace number one. People use it all the time by preference, except recently we've maybe noticed that it seems to be having more problems than either it should or that we're used to.

Luckily, we've got some data for this process, and I want to check to see if something has changed. If we have that data, we can use one of these menu items called Recurrence Analysis. That gives us the ability to fit a model and check with numbers to see if something has changed, so yes, this low p-value here says, yes, something has definitely changed. Of course, we can actually see that visually as well. We can see this furnace number two used to be much better, but now it seems to be having problems at a higher rate than the Furnace 1 that nobody wanted to use.

In fact, we can use that model to predict that if we don't intervene and make a change, eventually Furnace 2 is going to be worse performing than Furnace 1. Having the ability to use this data is really, really cool, and it's not technically a traditional reliability problem that we might think of. The second example here would be, maybe we're running an experiment. We've got a 5-factor DOE, and we run it as a full response surface model on some machine. We want to make that machine faster; in this case, a pick-and-place machine, just trying to make the pick-and-place machine move quicker.

When you do a designed experiment, when you do that DOE, and then you analyze it and fit the model to it, by default, the personality that JMP is going to select is Standard Least Squares. In this case, gives really, really tight confidence intervals that didn't actually predict super well, and it got people thinking, well, did we do this right? Did we do something wrong? Just a little shift in the thinking and realizing that we're dealing with time, and that time to complete something is reliability, allows us to pull up that Fit Model platform and choose Parametric Survival as the personality instead, which allows us to use that log-normal distribution.

Once we do that, then the model starts to predict better with more believable prediction margins. Very specifically, I want to show you, it's not a huge difference. In fact, the top five factors are the same top five factors and in the same order. We know that they have a little bit different of a parameter, but that's not really where the magic is. In fact, even the last, whatever, these eight or nine factors are the same and in the same order.

Where the magic happens from using this reliability data and the log-normal distribution is if we look at the magnitude of the middle effects. The Standard Least Squares model assumes that normality. Those effects are much larger, and they're influencing the prediction of the model versus using the parametric survival. Using that reliability information gives us a much more accurate model. It's really nice. Then also in green over here, the reliability model also allows us to predict when, say, 50% of our product will have been completed at this time, or 100% or 90%. We can get a much better understanding of the process itself by doing that.

Last case study. We can pretend that we're working at Red Triangle magazines or periodicals, and we just realized we're losing customer subscriptions and made a change to our process, and we want to know with numbers, did that change actually make the change that we saw over time? We can think about it this way, sorry. Sales was generating, before this problem, about eight new magazine subscriptions per day. The data collection showed that we were actually losing about eight magazine subscriptions per day using this Reliability Growth platform.

We can see what the problem was, and that is that we're losing as much as we're gaining, and so we're not growing as a business. They made their change on December 1st, and then they kept collecting data. They noticed a change, but they wanted that statistical certainty of did the change, was it effective? They want the causative certainty to say, was the change we made to our business the thing that caused this difference, or was it something else?

Reliability Growth platform has a couple of really cool tools, one of which is Automatic Change Point Detection. That change point, this says, so the tool has said I think that the change happened on the 18th of December, which is wonderful because it was slightly after December 1st. Had that said that the change point was in November 18th, then maybe we would have thought that our change wasn't the change that made this difference. Plus we can take a look and see after this change, the frequency of people canceling their subscription dropped pretty drastically. Now we're at about one incident per day instead of eight incidents per day.

It's a really cool tool, with some nice visuals. Maybe to conclude a little bit here, zoom in here. Basically just want to say to wrap this up, if we recognize that we've got time to event or time between event, frequency type data, we can often get better answers than we otherwise would when we use these reliability methods. There are a lot of those situations that we might not otherwise recognize. This middle conclusion right here says, using the reliability tools in JMP isn't as daunting as you might have thought.

I recognize that I didn't really talk about that much, and so I've got some resources here. Each of these Mastering JMP videos is about 41 or 42 minutes long. Even if you watch both, it'll take you less than an hour and a half to start being able to use these tools for yourself. Plus I put a list of a whole bunch of videos and other resources that we have available.

Let me zoom out one more time, and I'm going to show you one other thing. That is, if you want to play with any of these case studies, I have included them, and all you have to do is activate the contents and hit Open, and then a little Workflow Builder will show up, assuming you have JMP installed, and then you can push the Play button and then choose any of these case studies, so maybe you want to figure things out with this platform. How did it work? How was it launched? I can relaunch the analysis from that red triangle. I can see how it was populated, and I can maybe make some changes with it as well, do some fun things with the data, and play around. Thank you for watching. I hope you have lots and lots of success thinking about and using reliability tools.