Industrial time-series data is growing rapidly in both volume and complexity, driven by widespread use of systems such as the PI System and AspenTech IP.21. JMP’s Data Historian integration enables analysts to connect directly to these systems, bringing high-frequency, multistream data into JMP for interactive exploration and analysis.
This presentation demonstrates practical workflows for selecting, importing, and structuring time-series data in JMP. Attendees learn how to work with both PI Asset Framework hierarchies and AspenTech tags, including techniques for organizing related data streams using stacking and other structuring tools.
We also preview upcoming capabilities that improve scalability when working with large numbers of data streams, including more efficient handling of high-volume imports and new approaches to stream selection. These enhancements include improved PI tag-based workflows and server-assisted filtering.
Many of these enhancements were driven by feedback from Early Adopter customers. This session provides both practical guidance for current workflows and a forward-looking view of how JMP continues to evolve to support large-scale industrial data environments.

Hi, I'm Michael Hecht, and I'm here to talk about data historians, what they are, how they interact with JMP, and how you can then use JMP to gain insight into your data.
The first question is, what is a data historian? Imagine some large industrial manufacturing plant or something similar that has lots of data. You can imagine this mythical refinery having sensors all over it to measure flow rates and temperatures and pressures, and that data gets fed into somewhere, and that somewhere, a lot of times, is a data historian.
What's a good definition for a data historian? Here's one: a specialized database that collects all those sensor readings, compresses them, timestamps them, stores them for later analysis. When I give a definition, I'm told I should also give a humorous one, so here goes. I think this is pretty good because "a meticulous digital pack rat" sounds like another good definition for a data historian. The key with respect to JMP is this section right here: for later analysis. We want to be able to read this data into JMP and maybe actually explain what happened at 2:17 AM sometime in the past.
Where in JMP do you find the controls to let you access these data historians? If you go to the File menu, you'll see Connect To, and that submenu has different access methods for databases. Data Connectors is our new flagship method that's very flexible and extensible, and users can make data connectors and share them. ODBC is a more legacy type of data access method, but we support it in JMP. Then you see Historian, which is where access to our data historians live.
JMP supports two different kinds, the AspenTech IP.21 and the PI Server. PI Server support was added some time ago in JMP 17. It was greatly enhanced in JMP 18. In JMP 19, the most recent shipping version of JMP, we added support for AspenTech. Let's just get right in there and see what it looks like to import some data from an AspenTech server.
Switch over to JMP. I'll go to that File, Connect To, Data Historian, and I'll choose AspenTech. I'm presented with this connection dialog that lets me provide authentication credentials. I'm connecting to our demo AspenTech server, which is on AWS. I click OK, and this import screen appears. It has connected to the server and fetched from the server a list of these tags, which are basically the endpoints of where data from different sensors is going.
I can sort them. They're sorted right now by tag. I can click Description and sort them by description, and you see a lot of things that look like it might be for some refinery or some process like that. There's a bunch of other things here that has to do with baking cookies, it looks like, for some reason. But we'll just stick with these. I'm going to select several of these draw temp tags, and it defaults to the last 24 hours of data, so I'll just click Import, and up appears a data table with a column for each of the tags I selected, and you see each observation is timestamped. Now, we don't have values for every tag because at some timestamps, these sensors were not reporting any data, but we have missing values there, and JMP is fine with that.
Let's just use Graph Builder to see if we can visualize this data. I'll put timestamp on the X-axis and these tags on the Y-axis. JMP has many tools for helping you gain insight into these processes. I'm using Graph Builder because that's what I feel qualified to do here. But you see what happened with these three sensors over the last 24 hours. That's great. Let's close that down, and we'll get rid of that as well and go back to slides.
Coming in the next release of JMP, we have some enhancements to this AspenTech importer, and I'll touch on a few of them here. I'm not going to go into any great detail, but we have increased the kinds of tags that the importer can handle. In addition to adding the discrete tags, which is AspenTech's term for character data, we're also expanding the types of analog or numeric tags that you can import.
We're also importing what is called non-archiving tags, which is basically just constants like settings on a sensor. Those now get imported as table variables. We added an option that lets you name the imported columns by their tag's description instead of its ID.
Let's move on to PI Server. PI Server is similar to AspenTech in that it has tags that you import, but then they add an optional layer on top of that, which they call the Asset Framework. That's just a system where you can, you being the database administrator for a PI Server, can define a hierarchy in which different data tags reside.
In this example, we have a hypothetical data server, and we want to monitor the power in it, and that might need to be divided amongst server racks and different power meters on each rack, and what have you.
The Asset Framework lets you build a structure that mirrors the physical arrangement of your facility. That's layered on top of the raw tags. It is optional, but the hierarchical structure that provides can be very useful to JMP. In this example, we are importing data from these power meters. It looks something like this, and you can see it has three inputs, A, B, and C, and a bunch of different things it can measure. Let's just see what this looks like.
In this case, we'll go to the other choice, the PI Server, and this server that we're connecting to is actually one that we host internally at SAS as a demo test server. I'll just click OK, and we are connected to it. I will switch over to a different database here. We want the SAS PI database. You can see there's this hierarchy at the top level, there's several things, but also there's multiple data centers in different cities, and I can drill down into one of those and look at Atlanta has six different server racks, and each one of those has some number of power meters, and each one of those has different things that are measured.
I'm going to select a few of these and click this Add button to add them to this list that I want to build up of the attributes I'd like to import. Let's import those from Power Meter 1, and we'll import the same three from Power Meter 2. Let's go over to Server Rack 2 in Atlanta and do the same thing for Rack 2's first and second power meters. Okay, so now add, and we have 12 different attributes that we're importing here.
Now, this server doesn't generate data continuously. It has a snapshot of stored data from the past. We have to set our time range to the times when it actually has data. We'll go back to April 1st of 2021 for one day's worth of data and click Import. As before, we get this data table, and it's flat, as you can see, similar to the AspenTech in that every tag produced a separate column. Those columns are all named IA, IB, IC, but they are prefixed with the levels of the hierarchy that make them unique. You see that it didn't put Atlanta Data Center on here because that's common to all of them, so it added that as a table variable up here.
But we can pull this into Graph Builder like we did before. We'll put timestamp on the X and all 12 of these on the Y. Get rid of my selection there. You can see that it's a bit jumbled, everything on top of each other. Hard to really make anything out. Let me go back to slides. We will get rid of this, and then let's look at what we did here. As you saw, that import, I said it was flat, meaning the hierarchy that the tags came from was lost. We really want to take advantage of that hierarchy because JMP, it really aids in the insights you can gain through JMP.
As you saw, everything overlaid like that was difficult to separate the data out. What we want to do is use this option that was added in JMP 18 to stack the import, just like we would with the Data Table Stack command, which is in the Tables menu. We want to combine multiple columns together, and we want to do that while we are importing. Let's go back to our import dialog, and all I need to do is check the little stack button at the bottom, and I'll do the import again.
It's the same data, but you see the shape of the table is very different. Now, we only have three columns with data in it, IA, IB, IC, and they're separated by these stacking columns that identified what server rack and power meter each one came from. Like I said, it's the same data. If we look at all the rows that have the same timestamp, there's four of them for these three columns. That's 12 values. They're the same 12 values that were in this top row here, just rearranged differently. But that arrangement is important because JMP can use that when we are doing analysis process.
Bring up the Graph Builder, we'll do the same thing again. Now I just have these three to choose from. I'll put them on there, and you can see that the data from all the different IAs are grouped together. Let's separate that out by putting the power meter stacking column as Group X, and we'll put server rack as Group Y. Now we can see what's going on this one data center for these first two server racks and their first two power meters pretty clearly. You can see something's going on here with Meter 1 in both cases. It's much higher than the rest.
Let's get rid of that, and we'll go back to slides, because now I want to talk about what's coming in a future version of JMP for PI Server. You may have seen that it was a bit cumbersome for me to add all those attributes from the different hierarchy levels, so we've added a feature to make it easy to add things with the same name like that from a higher level of the hierarchy with this Add With Filter feature.
I also mentioned that the Asset Framework feature of PI Server is actually optional, and not all installations have it, and those that don't have it were not able to use this feature with JMP. But now, in this upcoming version of JMP, we provide direct access to the tag data without needing Asset Framework at all. In JMP 19, we had issues with being able to import large amounts of data, and we've really improved that a lot in the future version.
Let's take a look at some of those new features that are coming. I mentioned the attribute name filter, and it's right here above this list of tags we want to add, and it lets me provide a PI search expression. Then, when I click on some high level of the attribute hierarchy, I can choose to add all the tags which match that expression, and what I'm matching is this attribute name over here in this last column. I want to match tags that start with I, and space, and then has one of the characters A, B, or C, which I can specify in this square bracket list here.
Then, when I add Atlanta Data Center, you see I go from 12 attributes to 63 attributes, and all the server racks and all the power meters for each one are automatically added. In fact, I can just select all of the data centers around the country and click Add. Now it will import all 315 from around the country. I'm not going to do that here because it takes a while. On my system, it took about a minute, but I did it and I saved it right here. Let's take a look at that. This is what you get when that's imported. It looks like it did before, except there's a lot more rows, almost 260,000. You see that we have an extra column for Data Center, which we didn't before because now we have multiple values for that.
I've added to this table a Graph Builder script here that I call Daily Report. I'll just click that, and you can see at a glance for one data center, all of their server racks and what they were doing for each power meter. I put the data center in a local data filter so we can just walk through each data center around the country looking for strange anomalies and try to gain insight here. That is the attribute name filter. But I also mentioned that we can read the raw tag data without using the Asset Framework, and that is on the second tab here, the second tab at the top of the dialog, PI Tags.
When I click that, I get a different user interface for selecting the tags. Instead of listing them all out like we did for AspenTech, we want to make this extensible to very large numbers of tags, and our customers tell us that their PI Servers have tens of thousands of tags. So instead of getting them all, we have another PI expression here, and this expression is sent to the server and says, "Send me all your tags, but only the ones that match this expression."
What's being matched in this case is the tag name, and I know that I want tags that have some number of characters at the beginning, which I'll represent with the star or asterisk there, and then a period, and then I, space, and again A, B, or C, which I'll represent as the question mark wildcard, which means a single character.
Now when I click Search, we get back from the PI Server all the tags that match that expression. It looks very familiar here. I'm going to select all of them and add them to what we want to import. 312 tags. But you notice we don't have any Asset Framework hierarchy working for us here. But you may see that the names of these tags have information embedded in them that replicates that hierarchy. Like this first tag, its full name is Atlanta Data Center.Server Rack1.ION 6200 Power Meter 1.I A.
Wouldn't it be great if we could leverage that information and build those stacking columns like we did for AF? Well, you can by using this feature that we added here: Split Tag Names to Create Stacking Columns. This lets you specify a delimiter, and here the default is the period, which is exactly what we need to separate these guys out. I could do the import again, but I'm not going to because on my machine, it took 45 seconds. We'll just go to where I've already done it and open up this data table.
It looks very similar to the AF version, except that our stacking columns don't have specific names because we don't have the Asset Framework to provide those to us. Instead, we have to just use generic label names, but I can go in here and rename these. We'll call this one Data Center, and we'll call this one Server Rack, and this one will be Power Meter. Again, we have our Daily Report script, which I can run. Again, we can step through the different data centers around the country.
Let's go back to slides. I also wanted to talk about this improvement we made for large data imports. Now, those last two examples I showed where we were importing over 300 tags, it didn't work in JMP 19 because of the way we were doing the fetches. We tried to do a single fetch to capture all the attributes. It turns out that PI Server has a limit, an upper limit on the maximum number of data points it can return in a single fetch. Usually, it's 150,000.
Also, here's a little graph that tries to represent that different sensors might be reporting their data at different frequencies, and something that has a high frequency of reporting might hit that limit sooner and therefore appear to be truncated. What we've done in this future version of JMP is to break the request up into multiple requests, multiple fetches, so that none of them exceed those limits. Then we combine them all together in one data table.
In summary, you can use JMP to import data directly from these data historians, and these are the two that we support. PI Server, in particular, has hierarchical metadata, and that's very useful when you have imported the data into JMP, and you should definitely take advantage of it. In a future version of JMP, we'll be adding more types of tags to choose from for IP.21. We've added the attribute filtering for the PI Server to make it easy to build a list of hierarchical attributes. We've added a complete new feature supporting tags from a PI Server. We've improved the way that JMP does the import itself so that it's more scalable.
Like I said, that's all coming soon in a future version of JMP. A lot of these features that we added to this part of JMP came to us from our users, from you, people who are in our Early Adopter Program. If you want to help shape what comes next in this area of JMP, then that's a really great way to do it. Thank you very much.
Presenter
Skill level
- Beginner
- Intermediate
- Advanced