Retrieving the data from an industrial process historian, such as an AspenTech IP.21 historian, is only the first step in the analysis of a batch process. (We previously covered those steps at the 2025 JMP Discovery Conference in Austin.) In this paper, we cover the key steps needed to gain valuable insight into your process by using real data from a batch process.
We start by giving you four tools to prepare the data for analysis. Each of these tools allows you to visualize your process based upon the key steps in your production procedure. We follow that with four visualizations that highlight key features of your process. These visualizations can be used for process insight, process standardization, and ideation for process improvement. Next, we demonstrate the ease of developing models that connect your key outputs (e.g., QC measures) to your process variables. Finally, we conclude with some thoughts on how to use JMP Pro platforms to add even more insight.

Hi, I'm Tom Bidwell, and I'm a statistician for ALTA Performance Materials. Today, I'd like to share a few tips and tricks for analyzing data from an industrial batch process.
Just a few comments about who ALTA is. ALTA Performance Materials is a global company of about a billion dollars in revenue and a thousand employees, and we're located in 68 different countries. You can see all the markets we serve in the bottom right. What we do is manufacture thermoset resins that we sell to our customers, who then take those resins and fabricate the products that they sell in the marketplace.
Some examples of those products in each market: in building and construction, we're talking about solid surface countertops and sinks, tub showers, those kinds of products. In infrastructure, composite utility poles. In marine and recreational, sailboats, powerboats, jet skis. In transportation, in both the automotive and the heavy-duty truck markets, you might be familiar with the composite bed in a Honda Ridgeline as an example. Then in the corrosion market, we provide solutions to our customers to be able to fabricate composite tanks in highly corrosive environments.
The actual industrial resin that we're going to talk about today is a PG-maleate resin. The reason I can talk about it is because it was first created in the 1930s, so it's pretty old technology. Basically, we mix propylene glycol and maleic anhydride to create a half-ester. During that process, an exotherm occurs and heat is given off in the reactor. After that reaction is done, we heat to the esterification temperature, drive off the water, and create the polymer. That is an endothermic reaction where we have to add heat to make it happen. The process data you're going to see today and in this talk comes from this particular resin as we manufacture it.
We're going to focus on this middle section: preparing your data for analysis, some charting and visualizations, and some modeling concepts. Last year in Austin, Texas, I gave a talk on how we collect the data from an Aspen Historian. Then at the end, I'll give a few comments about future modeling efforts.
Speaking of that talk, I showed how I created an Aspen JMP add-in that allows us to pull the data from Aspen and concatenate the data in one table into these overlays that you see. What you see here in Aspen is, this is, say, a month's worth of data in the reactor, and each one of those times the temperature goes up and down, we're making a different product or a different polymer. The add-in's job is to extract the various time segments for a specific product.
A chemist might want to look at the last 10 batches of product A. What they do is use the tool, and what they get out on the right are these profiles. Really, the focus of today's talk is, now that we're at this point in the analysis process, how do we analyze the data? What do we do first? What do we do next? That's what I want to show you.
We've got three tables: the process data, the summary statistics of that data, and some QC results that are all dynamically linked together. We'll just take a brief tour through some of these tables. In terms of process tags, we have reactor temperature, weight, the glycol charge, the acid charge, the hot oil temperature, the amount of water that accumulates, and some column temperatures.
In the summary statistics table, what we have are summary statistics for each batch. We've got 40 batches, things like the charge ratio, the end date so we know when the batch finished. This would be… All these process tags are normalized, so that's what the Norm stands for. This would be the cycle time, the amount of glycol charge, the acid charge. Then we have things like the time by each step in the process.
Then similarly for reactor temperature, we have the min, the mean, and the max within each step of the process. Then finally, we've got a table with just two QC properties, Y1 and Y2. These are coded based on how many standard deviations each result is away from the mean.
Let's get into preparing the data for the analysis. The first thing that is probably the most important step is creating a stage column for each minute or each time step in the process. The key idea here is it's a batch process, and it follows a series of steps or stages in sequential order. That happens for every batch of the same product. What you can do is use your production procedure and the process variables to identify those stages in each row of your table. We use an If function to do that, to code in each row. What you'll end up with is what you see here at the bottom. The new stage variable in each row tells you what step of the batch you're in: charging glycol, then you can see right here we switch to charging acid, and then it repeats.
The way that works is, in the If statement, if time is zero, you are charging glycol. That's the first step. All the rest of these statements are Else If conditional statements. If none of them are true, then we're just going to write the same value over and over again, in this case charge glycol, and that's why it repeats in this table.
Once we encounter this step right here, if the previous minute is charging glycol, and we see the acid charge begin, then we start writing charge acid and so forth all the way down through the entire production procedure. My trick for doing this is we're going to create a new stage variable in our table. We're going to call it Stage 2. It's right here. That variable is just going to start off with, if time is zero, put in glycol, and then just repeat that over and over and over again. We want to be able to go through the process of modifying this formula. We do that with a simple graph in Graph Builder. We just make a graph of reactor temperature versus time overlaid by that stage. Pull up the formula, oops, wrong one, the formula and the graph next to each other.
You'll see how this works. As I add a row to the table, what I'm going to say is, if the previous row equals, I'm going to put in this value, glycol, and if the acid charge begins—this is my acid charge accumulator—then I want to write in the table, say, acid. All I'm going to do is hit Apply, and you can see that the color change happened in the graph. You can look at that and determine, based on your production procedure, is that correct? If not, you'd modify your formula. Then you just continue down adding each row of each step of the process until you're done.
As an example of what I might end up with, it's something like this. You have reactor temperature versus time, the color coding, charge glycol is over here, then charge acid in red, then we have our exotherm, then we heat up, then we esterify, and then we drop the batch into the next vessel.
I've got a local data filter for each of the batches, and then I have a column switcher for each of the key process tags. Just by doing this, and I've also added some reference points, this line represents the batch that did the shortest exotherm, the shortest time to heat up, and the shortest cycle time. You can very easily identify if your process is deviating from the best possible. We can go through and pick some different batches and look for some differences that occur. You could see there's a little bit of a change there. Right there, you don't have a big shoulder on the exotherm.
Very quickly, just by looking at this graph, you start getting process understanding without any modeling or any technique. There's a good one. We lost reactor temperature for some reason. You can see the temperature dropped. It didn't stay flat like it normally does. I could come across here and look at the hot oil, and we see that the hot oil temperature dropped, leading to a drop in the reactor temperature. All sorts of process understanding by just making this graph, just by coding the stages.
Another thing you can do, and this you have to do this second, you have to first code the stages, you can create new time columns that will normalize the temperature profile at the same point in the process. For example, the time from the start of the acid charge or the time from the start of heat-up. What you're going to do is put an If statement together to find the time when the batch changes from one step to the next.
In this case, if the stage is charge glycol and the lag of the next value, so minus one means you're looking one row ahead, if the next row is charge acid, pick up that time, else write, missing values, and do it by batch. Taking the maximum, basically, of that series records that constant value per batch, and you can subtract it from the column of times. This creates a time offset column.
An example of that graph looks like this. Now we have all the batches normalized to time zero when we start adding the acid charge, and we can start looking at the process variation around this point and asking questions. We'll get more to in a second. But again, this is all because we've coded the stages. The third idea is to calculate the batch summary statistics, and that's simply just using Analyze, Tabulate and then save it to a data table. This is my table, and this table is wide. I'll show you a trick for doing this. You've got times, temperatures, hot oil temperature, the amount of water accumulated, the column temperatures. If you're going to do this, I'll start over. If I put the batch ID in there, if you want to create some high-level batch summary statistics, you'd put them in first.
Things like, what was the maximum time? Maybe what was the maximum temperature in the batch? You could do that. Now, if you want to create summary statistics by stage, you would move to the right of this column. It will change the summary statistic to a range. Then I want to do the same thing with temperature, but I want to use different summary statistics. I'm going to, again, go outside the table, put min, mean, and max in there.
Then for something like, let's say the hot oil, I want to copy the same setup for hot oil. Here's the trick. When you add your stage variable, do it last and start from the right, because what's going to happen is the table is going to expand to the right, just like that. That's a lot easier than trying to say, "Oh, I want to do the water, and I have to drag it all the way across this table and find it out on the right." That's my little trick for doing this. Then the last step, make it into a data table so you have your summary statistics saved, and you can link it to your process data.
That next trick is one of my favorite things to do recently, and that is creating batch images. What you want to do is create a graph that illustrates the key features of your batch profile. Here's my graph that I'm going to use. I'm going to use this one right here. I've used the parallel Y axes, and this is time versus each of the water accumulation, the reactor weight, the hot oil temp, and the reactor temperature.
Once you have this, and you're happy with it, and by the way, I have a local data filter for each of the batches here, you want to save that script to your data table or to a script window. You just need to have the script available to you. Then the next thing you want to do—so that's step two—save it to your data table, you need to create a new column in your statistics table for the images to go. You need to make that an expression column. I'm going to do this through a script, which is really step four: make a script that loops through all the batches, creates the image for that batch, and saves it to each row of your statistics table. These scripts are in my appendix in my slides, so you can go find them. I also have a script to view the batch profiles. Last is to make sure that you label the new image column as a label column.
Let me bring up my script. Here's my script to add the images. I'm going to run the first couple lines here, which basically just open up the tables. We're going to be using this table right here. Now, what we're going to do is, I'm going to delete my profile column, so it's in here. I've already got the images saved. The first thing I'll do is delete it and we'll start over.
I'm going to create the new column and label it. It should show up right there with empty. If we look at the type of data, Column Info, it's an expression column. That's exactly how we want it. I'm going to get the number of batches in that table. If we look here, if I hover over it in the bottom of the window, it shows me there's 40 batches. The last piece is to have this For loop that loops from the first batch to the last one.
We pick up the batch ID from this batch ID column right here. For each row, we get the batch ID, and that's important, I'll show you here in a second. Then we'll paste in that graphing script. Here's the beginning of it: Graph Builder. The batch ID is important because what we're going to do is, in our local data filter, replace this where it says ID1 with the variable batch ID so we can loop through each of the batches.
We're going to tell JMP we don't want to see the pictures as we're making them. We take Object gb, we get it as a picture, and then we just add it to each row of the table. I'm just going to run this loop. As I do this, you should see the images show up in the data table. Here they go. It'll just take a second. It's grabbing each batch, making the batch profile, and adding it to the table.
Again, this script is in my appendix, so you can copy it and use it if you want. I do have the script as well to look at the batch profiles. If I run it, it'll allow you to pick an ID, and as a picture, you can look at each of these pictures that are in your table so you can see what they look like in case you ever needed to see them.
Now, we'll use this technique in a few minutes in a couple of the next graphing ideas. That moves us to visualization of the process data. The first one are the batch overlays, and you've already seen one of these. This upper right picture you saw before as a batch overlay. We're overlaying each profile at a specific time point. The idea here is, making these graphs allow you to look at your process variation and generate all sorts of improvement ideas right from the visualization.
Questions you might ask are: is there any unnecessary process variation that you can address right away? Did the differences you see in your process, did they occur due to a previous stage, or did they occur due to the current stage that you're in? Those are questions you'd want to address. Also, how does this variation in your process affect what you see going on later in the process? Finally, you might have some odd profiles like this one that we saw earlier, and you might be able to address that right away. All of this just from the batch overlay. Again, this is the one with the acid charge. Here's a good example.
If I highlight some of these profiles that have a high temperature at this point in the process when we start charging acid versus the ones that have a low temperature, what you see is that the curve shifts out to the right. So, you're costing yourself cycle time. This variation is costing you variation in the time dimension. Those are the kinds of things you can do with these graphs.
Here's the one for the esterification. Does this batch… Yes, it had a longer cycle time. Anytime you see this temperature deviate, you see longer cycle times.
The next idea is to look at what I refer to as mean profiles. What we're going to do is graph your key property, in this case for us, reactor temperature, by a QC property. We're going to overlay the QC properties and start looking to see, can we find any relationship visually between the QC property, and what's happening in the profiles? It gives you process insight, it might help you understand your model terms once you start modeling, and it might generate ideas for you. Let me show you how to make that one. We'll start with a graph, and I will start over.
In this case, I'm going to take the offset column for the start of heat-up, that's my X variable, and put reactor temperature here. I'm going to turn off, and we're going to use a spline. Just let it make it as sensitive as it can be. Then we're going to take our QC property and make it the overlay.
You can see on the right, they've got five different levels. I want to make this only three levels. I'm going to go to Custom Levels by right-clicking on the legend. It says when you hover over these bars, Alt-Shift-click to delete. I'm going to delete that one, and I'll delete this one. What that does is it gives me a group that's basically the QC results below one sigma, the QC results within one sigma of the mean, and the QC results above one sigma. Say okay. There we go.
I'm just going to expand this a little bit. What you see here in this part of the curve, green represents high values of Y1, red are values in the middle, and blue are values on the lower end. Because you see that ranking, you might ask yourself, "Hey, maybe that part of the process is critical to my QC property." Similarly, out at the end, I see green here, that's high values of Y1, red are in the middle, and blue are on the low end.
Again, maybe this plays a role. My guess would be this is more important because it's closer to the end of the process, but you can allow your models to tell you the answer to that question. Similarly, this loss of temperature seems to possibly play a role.
Those are mean profiles. This is the idea of you can fit these curves with FDE, this kind of thing. But this is a way to get some visualization to get process understanding before you start modeling.
The key comment that I want to make here is, each of these factors may be important, but they may not be causal. They may not be root causes. You always want to review whatever you find here with your subject-matter experts. They can help drive that particular discussion.
The next chart is something very simple. It's just a Y-by-X chart in Graph Builder, except we turn on the Column Switcher and put all of our summary statistics in there so we can switch out the X variable and look for correlations between the QC property and each of the process variables.
If I pull that one up, we're able to cycle through these very easily. You can see a correlation there with total cycle time correlating with Y1, other than this particular batch. Here's where the graph images come in handy. As soon as I hover over a point, up comes the image, and you can start trying to understand why was the cycle time long? What happened? In this one, again, we saw the hot oil fluctuating, we lost reactor temperature, it caused longer cycle time. Whereas when we come down here on the other end, we can see this is a more normal behavior of what we expect to see, and we get good, lower QC results. Then the one up here looks pretty normal. We don't have any explanation for—at least in these variables—why we had a high Y1 result. Tells us we want to look somewhere else.
Again, very quickly, we can just cycle through these. There's one, this is a good example. This is the total esterification time, which correlates with cycle time. Naturally, it also correlates with Y1. It's a good example of, you can have multiple variables correlating with your QC property, and you don't really know which one's the root cause. Again, the key here is dynamic linking of all your tables. Visualize your correlations before you start analyzing in a statistical method.
The last visualization is a model-driven multivariate control chart, and what I use this for is what I call batch conformance, trying to evaluate whether the batch profiles follow a typical average profile, and if they deviate too far away from that average. All we do is we feed in all of our summary statistics into the control chart and allow it to identify batches that look different. The really amazing thing about this is in about three clicks, you can understand which batches are different and why they're different. Let me show you that.
Here's our model-driven multivariate control chart. We're going to pick on the same point. You see it hovers. When I hover over it, it brings up a Pareto chart of all the summary values that went into it. Now I have that chart. I'm going to just sort it so it looks like it did in the thumbnail. If the bar is red, it tells you that particular batch in this summary statistic, so the max temperature when we're charging glycol, this batch was different or out of control. Sure enough, when I hover over it, you can see that batch was statistically out of control relative to the others.
Now, I'm going to click on this batch. It opens up the control chart. Again, when I hover over the point, this is where the magic happens. We can look at the batch profile, and we can see, yes, here's the reactor temperature, the maximum value when you charge glycol is the beginning value. It starts low. Most batches start like this. They start at a higher temperature. This is because typically we run products back to back to back, the reactor is hot, so the starting temperature is high. But if the reactor was down, it's naturally going to start at a lower temperature. But the method is quickly picking up the fact that the profile is different in this particular dimension.
Again, the batch images make this so much easier to explain to somebody. You can look and see where it's different, then you can visualize it, and you can explain it. Three clicks, it's amazing.
Okay, let's move into modeling. The most obvious platform to start with is Predictor Screening. Most are familiar, but if you're not familiar with Predictor Screening, you basically feed it all of your process variables, and in this case, our QC property, and it uses a bootstrap forest to identify potential predictors of your response from your data set.
The way it does it is, it takes your full data set, it creates a random sample called a bootstrap sample, then it fits the machine learning model, and it does that hundreds or thousands of times, and then aggregates it together to identify the most important factors. If I do that, I'll just click on the button, it'll fit the Predictor Screening model.
In terms of what did I actually do, let's relaunch the analysis real quick. I've got QC Y1 as the response. I've put in a number of my summary statistics: the times, the temperatures, and some of the water accumulation variables. Then I clicked OK. From here, it's identified the temperature at drop might be important, maybe something about charging acid might be important, something about the heat-up time could be important. So from here, you would just select which variables you want to move forward with, copy them, go to something simple like Fit Model, paste them in, and start your modeling process.
I'm just going to fit a simple model with this. You would work on removing factors that weren't important. You wouldn't do it this fast. But as you do this, you can find a model that seems reasonable and start trying to understand your prediction profile or, does it make sense as to why the QC property is changing as a function of these variables?
In terms of the model fitting process, that's what I just described. You use Predictor Screening to identify the variables. You take the top so many 10 or 20 factors. You could fit a stepwise regression like I did. You could fit a neural network. You could use any JMP platform at this point just used for your modeling technique. Again, most important, when you get done, review with domain experts to help separate out latent or non-causal factors from potential root causes and then move forward to identify what improvements you want to make.
As an example, here's a neural network that I fit with the same data. You can see all sorts of different shapes that we could start looking into for each of the different factors. Some of them a lot smaller, some a lot bigger. We saw this before: the average temperature during esterification is playing a role. That's because as the temperature decreases, we lose heat. We saw that before in the mean profile, that the Y1 value went up.
In terms of future work, I see FDE in my future now that I have JMP Pro, and we could look at some of those profiles and see how the shape affects the QC properties.
I also think SEM could play a role here. We're following a sequential process, so we could take some of our input variables to essentially predict the half-ester characteristics, then some of the mid-term variables as we polymerize, and then the polymer characteristics to link to our QC properties. I think this would be a good network diagram to work towards in SEM.
In summary, preparing batch data, I think, requires you to code your batch stages into the table so we can analyze the data in a meaningful way. Tools such as the Stage column and the Time offset columns are really useful. I hope I demonstrated that for you. The summary statistics by stage can also be used as input X's for a lot of different platforms, whatever your favorite modeling technique is.
Visualizations like the batch overlays and the mean profiles by other factors like your QC properties can help really drive insight even before you begin modeling. As a side note, batch conformance through the model-driven multivariate control chart is really aided through the use of the batch images.
Modeling the QC properties versus the process variables was very efficiently done through process screening followed by Fit Model or whatever your favorite platform is. Thank you.
Presenter
Skill level
- Beginner
- Intermediate
- Advanced