Decision-Making with Prediction Intervals in JMP Profiler: A Case Study from Cencora-PharmaLex

Hello, everyone. My name is Hailey. If I would like to talk a little bit about my background. I have a PhD in biostatistics and work as a manager of a statistics and data science at PharmaLex Cencora. I'm working in the statistical analysis for both clinical and non-clinical research, helping to generate reliable insight for decision-making. I use JMP extensively for data analysis in supporting various projects. My goal is to apply advanced statistical methods to improve research quality and efficacy.

In this presentation, it's about improving decision-making with prediction intervals in the JMP profiler, and I will show a case study from Cencora PharmaLex. This poster shows how JMP's profiler tool use prediction intervals to improve decision-making. Prediction interval help manage uncertainty, making an outcome clearly. This is useful in critical situation where accuracy and risk management matter. Adding prediction intervals make a decision more reliable and models more precise.

In this presentation, I divided into two parts. The first I will talk about the simulation and the next I will talk about the case study from Cencora PharmaLex. Cencora PharmaLex is a leading provider of specialized service in the pharmaceutical and life science industry. In this case study, we focus on projects where the company utilizes a job profiler tool to enhance decision-making process for the client. By incorporating prediction intervals, Cencora PharmaLex was able to quantify the uncertainty in their model predictions, leading to more informed and reliable decision.

First, I want to talk about the simulation. This study evaluate prediction interval in a DOE with five continuous factor testing different models and scenarios for accuracy and reliability. The experimental design includes an incomplete model with five continuous factors and selected interactions.

Three scenarios we defined, first, low design with just 12 rounds until the high design with the 30 rounds and each scenario repeated 100 times. The response was simulated using a specific model and evaluated at key point in the factor space that we should put here minus 1, 0.5 and 0. Linear and random effects models were analyzed using the JMP and compare with the R-based model with LM and Stan to assess the performance.

As you can see, the first result here for the prediction values. The random effect model shows the difference between the JMP and the Stan in terms of prediction values, particularly in the less complex design. However, in the linear model, we don't see any differences for the predicted value for both softwares. Increasing the number of the blocks and runs generally improve the prediction value for both models as you can see here in two of the graphs for a linear model and also for the mixed model.

Now if we want to talk just about the upper and the lower predicted prediction intervals, in the linear model as you see here by JMP and by LM function in R, the difference is between the lower and upper predictions intervals don't appear in either JMP or the LM function in R, and confirming the reliability of the both result.

However, we come to the mixed model in the last number of the blocks, and runs generally leads to the wider prediction interval in the profiler. As you can see, compare between the two graphs for the JMP and for the Stan indicate greater uncertainty in the prediction. As more data points and variability are introduced, the model capture a broader range of possible outcome making prediction less certain, while a larger design improves the model's ability to generalize the generalization.

They also highlight the complexity and predictability of the interactions within the system. This effect is particularly noticeable in scenarios with a high variability, where the prediction profiler predicts a wider spread of the potential values rather than a precise estimate. As you see the compare for the low design, we see a wider range for the prediction values.

I want to bring your caution here. The results of the JMP is overall optimistic. It means the probability of the success may be too high. JMP doesn't use the t-distribution, we use the normal distribution instead of the T for the prediction distribution in calculating the prediction interval.

The students allow for accounting for limited data by the degree of the freedom and use the RMSE sigma [inaudible 00:06:24] as error of the prediction instead of the complete form, accounting for the DOE location. When you see the number of the observations is much more higher than the number of the parameter, when the degree of freedom are high or high enough, I can say that the prediction interval is also wider in the linear mixed model in JMP instead of the Stan in R. Pay attention. When the degree of freedom is low, we should pay attention for the overoptimistic of JMP prediction interval.

Here we want to talk about a case study in the Cencora PharmaLex, how we use the prediction interval for the client to show them what is the accuracy in the project and reliability in the project. The profiler shows a prediction line for response variable based on the level of the factors. Here we have eight different variables that we designed in the DOE.

Prediction interval represent the range which the true response is expected to fall within the certain level. For example, here we consider the 95 percentage. Easily we can change the red vertical line and show what is the differences here. We say to the client, "Look for the differences in the prediction values and the width of the prediction intervals and narrow for a prediction. Intervals indicate more precise predictions while a wider interval indicates more uncertainty." You can see here in the two graphs we change the values, we can move the vertical red line and show what is the differences to the client.

In the end, we can conclude the prediction intervals improve decision-making by showing a range of the possible outcomes and quantifying uncertainty. They help assess risks, improve accuracy, and support better choices. Especially in critical situations for some of the clients in the clinical and also the life science, their value extends to risk management and fields like pharmaceutical and life science. Thank you so much for having me. Here is my presentation.