Helper Functions for Exploratory Data Analysis

Welcome to this Discovery Summit presentation. My name is Holger Specht. I work in the analytics department at ams OSRAM, and I use JMP for exploratory data analysis, really 24/7. In this presentation, I want to show you some helper tools which you can use for exploratory data analysis.

This picture really shows how JMP can be used to interface data directly with the human brain. Here I have easy example where I can show you how powerful it can be. It's a very simple children game, so you just have to find the way from the entrance to the exit. When you get stuck in a dead end, you turn back and find your way through the maze.

That will take some minutes for a kid, and I think it will also take some minutes for an experienced analytics guy. But there are some cool tricks that can speed up this process, and I want to show that.

I developed a tiny script that works like a paint bucket in a graphics tool. When I use that and click here on one point, then JMP will find neighboring points and color them, so here in black, and it will continue to find those points till it found all the points which belong to this wall, and that we will see in a second.

Okay, now it's finished. Now you see that there are two walls, a black wall and an orange wall, and now you can go from the entrance to the exit within seconds. The human brain now really finds the way here really quickly from the entrance to the exit, and you just have to stay between the black and the orange wall. Also nice, so if you have two black walls or two orange walls next to you, you know that it's much faster to go back to the entrance and then to start again.

How can that be done in JMP? Easy trick. You need the script and two extra steps. The extra steps I want to show you. The first step is you have to download the Image to Table Add-in from the Marketplace. It's programmed by Jed Campbell, and here you just have to click on Download and install it.

Then you can open this add-in. The add-in, there you can select an image like the maze, and then you can import it. Here the maze is still a graphic, but when you click on Import, JMP will convert this image to a data table and also show a graph that is related to the data table.

Now we have this link between the graph and the data table. When I select data points here in the graph and go to the data table, they are automatically also selected in the data table, and vice versa I can also select data points here, and they will be selected here.

Now the final step is this paint bucket. In a graphics program, the program knows the X and Y coordinates, and it knows for a point what are the neighbors. But JMP doesn't know that. It just knows these rows, and it knows, okay, there is an X and Y coordinate, but it doesn't know which row is a neighbor of another row.

We need this intermediate step to tell JMP where are the neighbors. That can be done with another add-in. That is the QuickSelect toolbar and one of the buttons in the QuickSelect toolbar does exactly this job. If you want to get the QuickSelect toolbar, you have to go to the Community, go to the Add-ins, and in the Add-ins, there is this QuickSelect toolbar, and you can download it here by clicking here. Then you just have to activate it by right-clicking here in the toolbar area and selecting QuickSelect.

This is the button that we need, so the Neighbors. We just have to select the coordinates, and we want to have the direct and the diagonal neighbors. Let's click on Okay. JMP takes a while to analyze the coordinates X and Y and check for neighbors, then it will merge the data back to the data table, and in some seconds it will finish. Okay, and here it is.

We have now a column with the direct and diagonal neighbors. As an example, let me make it larger. Let's go to row number 5, and we see that row number 4 and 6 are neighbors. This is clear from the X and Y coordinates, but there are also in the 300 some other neighbors of this row.

With this additional information, we can go back now to this graph and use the paint bucket tool. This time, maybe we select the top wall and now JMP again, searches for neighboring points which belong to the same wall and colors it. At the end, we will have the two walls with the different colors. Quite powerful. Compared to the children approach that you really do that by hand and find every dead end.

This is really much faster. I think the time that I needed now to explain the tool was not much longer than the time that the kid would need to find the exit. With every new maze, there is a huge payback then because then you can really solve such a maze in seconds. Quite powerful tool here.

Let's have a look also at the other functions here. For example, when we zoom in to this arrow and mark the arrow, so we could label this arrow by clicking here and typing arrow. Then we look at the data table. There's a new column with zeros and ones, and all the rows that belong to the arrow are then ones, and the others are zero. This is quite easy.

There are also two other ways to label data points. I want to show this label function. Let's also write here arrow. Here in this case, column with property multiple response is added to the data table. Here we see that these points belong to the arrow. Because it's multiple response, we can now collect additional information here. Let's zoom out here and maybe change the information that we see.

That was the color with the vector, the column with the vector with all the neighbors. We can count these neighbors, and here is the number of the neighbors. I want to use this as nominal value and maybe put it on the color, and maybe also put it on the red.

Here you see now the different types of points. There are points with eight neighbors, and these are the points within the wall. They have really eight neighbors, but there are other points which just have five neighbors. Let's open again the legend and select those points. I also want to label those. Let's use again this tool, and this time, we say 5N for five neighbors. Now we have the two labels. We see here that in the arrow, there are also some points with five neighbors. Just select the arrow. Here we have then both labels. It's a part of the arrow and the point has five neighbors. This is much better than the way before because now we really can collect all these informations.

Now let's go to another dimension. For example, with these color coordinates. Let's use the red channel and the green channel and look at the individual points. Here, it seems that there's a two-fold distribution. Let's mark one of the distributions, and label it.

Now we can go back to our graph, and instead of the number of neighbors, we remove that again. We take our new column with the labels, and since JMP 16, the Graph Builder can understand this multiple response property. When I use it here in the Graph Builder, you see here now that for every label, a separate graph is generated. Now I can select data points, and the corresponding data points will be then also selected in the other graphs. Here we see that some of these points are also in this subgroup and in this subgroup.

To facilitate this a little bit, I use another button. This is a very nice button here. This is similar to this black button, but what it does is whatever I select, it will be set to one. This is quite powerful. Now I can use this column to separate between selected and unselected points. Let's use this button or this column here as a color.

Now I can select data points and the selected data points in the other plot are then highlighted with this separate color. Now, for example, I can select here all the data points with main color. Now the data points will also be selected in the 5N in this arrow group. The last icon that I want to present you is this invert button. When you click on the invert button, then the inverse of the selection is then selected. The graph really likes our analysis. It seems that we found a hidden secret in the data set.

When you found such a secret, you want to share it with your colleagues. The easy button to do that is this PowerPoint, Send to PowerPoint button. You just have to click there. If there is a PowerPoint presentation open on your system, then the graph will be automatically added to your PowerPoint presentation. Even better, when you press Control and click on the graph, then you can come back to your data table. In this data table, a new script group is generated. Then you have here a script to recreate exactly the graph that was sent to PowerPoint. If you have a presentation from half a year ago, and if you want to find out how the graph was generated, this will save you really half an hour.

Let's come back here to the main presentation. We discussed the QuickSelect toolbar, and I showed you some functions already of the Graph Builder toolbar. Actually, it's just an extension of the report toolbar. These three icons you might know from the report toolbar. After installing the Graph Builder toolbar, you will have more items available in this report toolbar.

To install it, you just have to go to the Marketplace and download this toolbar and then come back to JMP. If you want to add this toolbar, just right-click here and select the report toolbar. At first, it will show up at the top, and you can right-click here again and change the location to left. Then this toolbar will be here.

I already showed you the Send to PowerPoint, and I used several times this open and close command. Let's go again to a plot like this. When I click here, I can open the control, and when I'm finished, I can close the control. Here, I think in this case, we just have a single point here. It's a little bit boring. But then we take again these neighbors. Then sometimes we want to have this legend accessible to change the colors, and then the legend doesn't help so much. Then we can just hide it, close the menu, and then share the plot. Then maybe some minutes later, we want to change the color, then we just have to click here, have access to the legend again and close it again.

There are many more icons, and due to time constraints, I don't want to show them all in detail here in this presentation, but come to the Discovery Summit in Berlin, and then we will discuss there. There is some last add-in that I want to show you, and this is the Normalize toolbar. Again, like the Graph Builder toolbar, you just have to go to the Marketplace and download it from there. Then an icon will show up here, and it's also in the Columns menu, Normalize.

Let's open this data set. It's weather data versus time. Here we have a plot versus the time of several measurement stations, and we want to normalize it a little bit. We could select here some data points and use our new QuickSelect toolbar and say reference. We have here now a reference area, and we want to normalize every single curve by the values in this reference area. To do so, we can use here this average here, and we need the reference as a reference. We pick this column, say, okay, we want to use the entries with a 1 in this column as a reference, and we want to normalize by station. After setting all the settings, we can apply and close this menu. Open here this menu and use this new column on the Y-axis. Now we normalized every curve to the values here, and we see in spring, there's an intersection, and then in autumn there is another intersection.

With this example, I want to close here and highlight again how useful JMP can be to access data with the human brain. Thank you.