Utilizing Extrapolation Control Feature in JMP® Pro 16 to Optimize Real-World Mining Process (2021-US-30MP-844)

[#1]

Level: Beginner

Xinjie Tong, Senior Research Statistician, Dow

The case study aims to assess 12 types of frothers in terms of their performance in mineral recovery in mining. In addition to frother type, the input variables also include particle size (coded value: 0, 0.35, 1) as a continuous variable and water type (Type A, Type B, Type C) as a categorical variable. The response variable is mineral recovery efficiency.

Using extrapolation control feature in the prediction profiler, a possible extrapolation was observed for Water Type B when particle size = 1 based on regularized T square metric. This extrapolation was caused by an imbalance in the DOE where the measurement for Water Type B was only taken with particle size = 0.35. The extrapolation control was turned on in the effort of optimizing mineral recovery efficiency to achieve reliable optimal solutions. 

The extrapolation issue disappeared when the DOE was augmented to be more balanced. The extrapolation control feature provided immediate understanding on how augmenting an imbalanced DOE could largely enhance the usability of the predictive model. 

Auto-generated transcript...

Speaker	Transcript
Josh Staunton	hi.
Xinjie Tong	hi josh.
Josh Staunton	hi is zingy that.
Xinjie Tong	Yes, good.
Josh Staunton	Well, thank you for agreeing to do this I'm excited to hear your your presentation.
Xinjie Tong	Thank you, so how can we get started.
	or any other thing, before we start.
Josh Staunton	yeah there's just a quick checklist, I just wanted to go through So if you have any.
	Your I am your phone that that's still kind of turned on.
Xinjie Tong	Well yeah.
Josh Staunton	yeah there's a.
Xinjie Tong	Great reminder Thank you.
Josh Staunton	And just one quick I gotta get a verbal affirmation that you kind of know what this being recorded so like you understand this is being recorded for JMP discovery summit.
	conference and you'll, be it will be available publicly in the job user community to give permission for this recording and use.
	Yes, yeah okay good.
	formality they're great and.
	I like your background everything looks good and I guess the only thing does the do you have your do you know if your display is set in 1920 times 10 at.
Xinjie Tong	I actually I don't.
	know clearly.
Xinjie Tong	display settings real quick.
	I'm at we think see 119 20 times 1080.
Josh Staunton	Oh perfect.
Xinjie Tong	yeah good.
Josh Staunton	Yes, that's it perfect and you're going to be doing this is going to be on utilizing extrapolation control feature and JMP pro 16 optimized reward mining process 2021 us 30 MP for for.
	Great alright so I'm going to.
	Do you want to see if you can start presenting just get your slides going and.
	If everything looks good from your side, I will stop my video go on mute and just pass it over to you.
Xinjie Tong	So are we can try real quick either okay.
Josh Staunton	If you want to present and I could just make sure.
	We can see your slides.
Xinjie Tong	yeah um so I need to first sharing my screen.
	yeah.
Xinjie Tong	And you see my screen.
	yep okay so, so I will probably do some DEMO in I'm not probably.
	I need to do some DEMO So when I do, that I will just maybe do this, is that OK, and then go back to the slides.
Josh Staunton	Exactly yeah that's that that's fine.
	And you know we there won't be any editing done within the video will cut out part of it that you know.
	But so you know if you make a mistake if you.
	You know just try to keep.
	kind of rolling through it we're all humans here it's okay to make a mistake.
	If you think you know, but if you if you really think it was in a catastrophic thing you'd like to see.
	Now you can just.
	We can we can we can start it over it's not a not a huge deal but yeah but don't don't allow just a little mistake to kind of bother you just kind of keep.
	You he'll be.
	yeah yeah.
Xinjie Tong	And I think it's about 20 minutes right the presentation.
	Okay, is that.
	The requirements like.
Josh Staunton	You know I'm not sure that everyone's a little bit different, but you know I don't.
	yeah don't worry too much about the time, those are more kind of guidelines just try to you know.
	Okay, Dan and get get out what you your research and we'll be okay to see it.
Xinjie Tong	Okay sounds good, so if that's the case, I will maybe start start presenting.
Josh Staunton	yeah, so I will I'll start my video and then yeah he wanted to just take a minute or you can start whenever whenever you're ready.
Xinjie Tong	Oh, I don't need to do any production is that right or.
Josh Staunton	No, have you could just give it, you could say you know say your name, and you know what you're going to be going through and.
	kind of roll into it.
Xinjie Tong	Okay okay.
Josh Staunton	Okay, all right well I'm gonna turn my camera off and going you but good luck, thank you.
Xinjie Tong	Thank you.
	So, my name is Xinjie Tong and my presentation is about how we utilize the extrapolation control feature in JMP Pro 16 in a real world mining problem, where we were trying to optimize the mineral removal efficiency in the mining process.
	So I will introduce a case study about the mining process, and then we would explore a little bit more about extrapolation control metric also mentioned, how do we optimize with extrapolation in mind and how using DOE augmentation can potentially deal with the extrapolation problem.
	Here's a slide that introduced about extrapolation control metric that are currently used in JMP 16 Pro and all these information are coming from the video in JMP community.
	made by SAS, as you can find the for the reference at the bottom, so there are two major metrics that are being used in JMP.
	One is the leverage, which is used in linear platform, specifically when we fit least squares. There is another
	metric called regularized Hotelling's T square, which is now implemented in nonlinear platform, such as GenReg, neural partial least squares. So for leverage,
	it's a function of the design matrix of the training data, as well as the prediction point. So it's dependent on the model form. Also for a linear model, it's proportional to the prediction variance.
	But differently, for the regularized t square, it's an unsupervised version, so it's not a function of any form of the model.
	And here's the equation that I steal from that video.
	This is the example that's also coming from that video to show how the optimization can differ when you have extrapolation in mind.
	So here is an example where we have three input variables and, as we can see here the MaxPulse and RunPulse have a strong positive correlation.
	If we don't consider extrapolation control, the model will tell us the blue point is the optimum. However, it doesn't obey the relationship in the training data, so it's not very
	reliable. The model is extrapolating. But if we turn on the extrapolation control, it will suggest this red point, which is now more realistic. So it does demonstrate how extrapolation control can really help with decision making and avoid unrealistic prediction.
	So now, I will talk about the case study that I encountered during my employment at Dow. So this case study is about mining process and our objective was to optimize mining conditions for enhanced efficiency of mineral removal.
	We have 120 rows in this data set, and we have three input variables and one output variable.
	For the input variable, we have water type, which is a categorical variable with three levels, so I coded them to Type A, B and C.
	We also have 12 types of frothers that were used, and they are also categorical. We have a numerical variable, which is particle size and I coded them to 0, 1 and also .35.
	Here the output is the removal efficiency, which is a value between zero and one, and all the values here are coded to protect the actual information.
	So the problem with this data set is that there is actually some colinearity due to the design stage. So if we can look at this graphic builder, we have very easily visualized the data structure using graph builder, which is a platform in JMP.
	So here, I was using frother type as the wrapping, so each box refers to one type of the frother and then we're looking at particle size on the Y axis and the water type on the X axis.
	So here, the problem is when we have water Type B, we'll only have one level of particle size, which is at the middle level, so the colinearity exists when we try to estimate particle size
	and interacting with Type B. So it's not estimable in this case.
	Due to this imbalance or some dependency, we will have extrapolation problem, and this is where the extrapolation control feature can really help us prevent any risk due to this imperfect data set.
	So, then, I went ahead and built the model and using different platforms in JMP, and this is to compare the metrics and how they can potentially impact our decision making.
	Before I fit any model, I usually will start this evaluate design, which are...which is one of the capabilities in JMP. So this platform allows us to ask to understand,
	given a data set, what are the terms that can be estimated from the data set. And I will do a quick demo about this. So here is my data set. So I have three input variables and one output variable. What we can do is we go to the DOE,
	design diagnostics, and evaluate design. So here, the Y response is optional, so I will drag my three inputs into here, click OK.
	And now it's telling me that if I'm only considering main effects, it's able to estimate those and I can look at my diagnostics, such as your prediction variance within the entire space...design space and also some diagnostics.
	However, I do want to estimate the two-way interactions So if I click two-way interactions,
	this alert comes up, and it's telling me this this data set cannot fit all the two-way interaction terms and the terms that's not being able to estimate have been removed.
	And if we look here, we do notice that the interaction between water type and particular size has been deleted and that's due to the colinearity issue that I just mentioned.
	So if we check here, these diagnostics have been updated, based on this new set of model terms. So now I know that
	due to the colinearity, I cannot estimate all the terms, so I can only start with these terms which the model can can fit. Then I will just go back to my slides. So I will start building the model.
	So the first model I fitted was within the GenReg platform. The reason is that my response, which is a efficiency. It's a value between zero and one and I usually will think about beta distribution, which can only be realized in this GenReg.
	However, in our case, although it's within zero and one, but the distribution is more like normal, and there's no high frequency near zero or one, which make it problematic to use normal distribution.
	So what I did it was I tried both beta and normal in my model fitting within this GenReg. I also tried the full model and the various methods for model reduction. So
	I compared the AICc and BIC, and I realized that I could be using normal distribution and the Lasso reduction methodology gives a smaller AICc or BIC compared to the full model. That's probably the one that I will move forward with.
	But now that we can use normal distribution and I realized that we can actually fit the same model, but in a different platforms.
	So the next thing I did was a used fit least square, which is a linear platform in JMP and I fit the same model in there.
	But now what it allows me to do is using the leverage as the extrapolation control metric, whereas, in GenReg, it was using the regularized Hotelling's T square. So now, I can compare how does these two metrics really contrast or how similar they are.
	So now we're going to the metric comparison and some results. So here is my map, so I started with the same full model.
	And I fit the model within GenReg and fit least squares where they were using different metrics. And in addition to comparing the metrics, I also wanted to see if I used a full model versus reduced model by changing the model form, how does that change the performance of the metrics?
	So here is where I will do another demo, and we can also see how we can use extrapolation control feature in a profiler. So I will start with this GenReg, which is the same script.
	Just so just so you...for information, I will click model dialogue. So this is where the model was set up. So efficiency for Y, these terms that were defined in the evaluate design and using beta or normal.
	And then click run. So I saved two models. The first one is the normal distribution with model reduction. The second one is the normal distribution with full model. So if we first start with a full model, what we can do is, we click here turn on the profiler.
	And in profiler,
	what's new here is, we can now click extrapolation control and we can turn the warning off.
	So now there's no details, so if I would like to see the details, I need to quick extrapolation details
	and it shows us the threshold in the metric. So if we can move along the particle size and the metric is changing, but they are within the threshold, so there is no warning.
	But if we go to Type B where that colinearity exists, we do see this possible extrapolation warning at very high particle size or at very little particle size.
	One other thing we can do is instead of turning on the warning, we can directly turn on the extrapolation control. And what it does is it only shows the range where there is no extrapolation.
	So, and now I want to mention that the threshold here is 16.3, and what if we fit a reduced model? Does that change the threshold and also the extrapolation region? So now I can go to my reduced model, and I can turn on my profiler which I already turned on, and here is...
	if I click extrapolation details and turn it on, we'll see it has the same threshold and the same range of extrapolation. So here this regularized T square is unsupervised and it doesn't change when we use a full model versus reduced model.
	And if we, instead of using GenReg, let's try the fit lease squares, which is linear platform and the leverage can be used for model extrapolation. So here is my full model. I can click factor profiling, turn on the profiler.
	Similar here, we can click extrapolation control and turn the warning on and also extrapolation details.
	So here the threshold is different because it was calculated based on different equation. So similar as before, Type B...Type A and C has no extrapolation but for Type B, it's actually...it all has the extrapolation warming.
	And if I set this to .35, this is the only point that there's no extrapolation. To show that more clearly, I can turn on my extrapolation control and then you would see that one that had B water,
	it's only able to predict this single level of particle size, which was observed in the experiment. So this is telling us that,
	given the data set and using leverage as the metric, we cannot predict any other particle size when we use water...water Type B.
	So this compared to the GenReg, where the regularized T square was used, is more strict, because in the other scenario, it was able to predict a narrower range. And again, this is because that one is unsupervised and this one is more directed towards the model form.
	So here we have threshold of .5, and I did try the reduced model when I was using leverage, and the threshold was also changed because it's dependent on the model form. So if you have a smaller model, you will have a smaller threshold. And this is the comparison table.
	In this specific case, we do see leverage is more strict than regularized T Square.
	So now, I was interested in how that we can optimize with this extrapolation control feature, because that's the objective our of our project.
	So here's the final output I got. So this is drawing all the input variables and the blue point is the one that I optimized without any extrapolation control. And it's suggesting this point, which was not observed in the data.
	However, if we do consider extrapolation control, it suggests something that's more similar to that training data which is now, we have more confidence...confidence in it.
	Another thing we can really do is to do some visualization with and without extrapolation control. So I actually want to demo this also very quickly,
	because I believe this is the future that I love the most is, for example, if we decided to...
	Sorry, I will just close this for clear visualization.
	So if we can...if we decided to use this profiler...this model, we can go ahead, and making sure we turn on the extrapolation control, and then what we can click is output random table.
	And what this does is it generates all the potential conditions that the model can predict without any extrapolation.
	So now, I have 5,000 runs and I can click here to very easily see the distribution or the histograms of each parameter.
	If I click Type A, we can see the model can predict the whole range between zero and one for particle size.
	Similar for Type C, but when we have Type B, it's only able to predict a narrow range of particle size due to that extra...
	sorry, extrapolation. So with this one, we can really optimize and simulate with extrapolation control in mind.
	And also, we can from this file, we can use the graphic builder
	and generate this plot, where we can see if we don't have extrapolation control, we have all the ranges with the color being the efficiency, the predicted efficiency.
	But with extrapolation control, we cannot predict all these ranges and that's a very good visualization based on that simulation function.
	And now, my last question would be, what if I really want to estimate these spaces? In order to do that, we have to fix the data or augment the data so that the model can predict the whole range. So now we can do data augmentation, which is also part of the JMP's capability.
	So augmented design, what we can do is, we can add a new block variable so that the new runs can be separated...can be grouped into a block two.
	Also, we can define the model terms that we want estimate. Now, I want to estimate all the two-way interactions.
	And here is a visualization of the augmented runs, so the red points are all the augmented runs and now since we are assigning the data into
	high and low of the particle size when water type is Type B, it's able to estimate all the terms and there's no colinearity problem anymore. And also in the profiler, we do see that when we have Type B, it's able to predict the whole range.
	So a few takeaways. There can be many sources of extrapolation, such as when you have dependency among input variables, or you have any other term that you cannot estimate but you want to
	be included in your model. In any predictive modeling world is also the machine learning. We have to take account the extrapolation control because it's the sanity check
	for the predictive world, and thankful for JMP 16 Pro, this feature make it very simple for us to use. And also it's very easy to demonstrate how the structure of the data set can impact the usability of the model. So
	therefore, I made this one, and I want to share with the broader community to really demonstrate the...how helpful this feature is. And that's all my presentation. Here is the reference and some disclaimer.
Josh Staunton	Great.
Xinjie Tong	Is that okay.
Josh Staunton	Thank you, that was that was awesome I love I.
	Love seeing the predictor in their profile.
	profile.
	Now great job that was that was he went off without a hitch.
Xinjie Tong	Thank you, thank you, I hope, it's not too.
	Well i'd say it's for beginners so there's no much complexity in there.
Josh Staunton	That it was like for me, you know that it's a little bit will advance, but no that was not that was great idea, he started talking about augmenting designs it's definitely the next level so.
	That.
	Yes, that's that's pretty much it we will, as I said, cut this up kind of Edit out, you know the front and the back, and then it should not sure when they expect to have it up on the website, but it will be up there shortly and.
	Thank you again for putting in the time and effort to to share this with.
	With our Community.
Xinjie Tong	yeah, of course.
	yeah sounds good.
Josh Staunton	Alright well take care, and hopefully we'll meet you at one of these events sometime soon.
Xinjie Tong	yeah.
	Thank you, looking forward to it.
	Thank you for the opportunity.
Josh Staunton	Oh yes, thank you very much, all right bye bye.
Xinjie Tong	bye have a good day.
	You too.