This presentation walks through the techniques and Python code found in the jmpex package's code, which is the code used to bridge between JSL and R using Python. It gives in-depth coverage of parameters passing between the three environments: JSL, R, Python. It also showcases the power of the JMP and Python integration, its design, techniques and hidden corners. Attendees are empowered to use these techniques in builing JMP and Python applications and interfaces. There is additional discussion on the packaging of Python scripts for native installation (pip/jpip) and how that ties into add-ins and the JMP Marketplace.

I'm Paul Nelson. I'm here to present about digging into JMP, R, and Python, and what we've done. In JMP 19, we had to restore the R integration. It went away in 18, and we've resurfaced it, brought it back for JMP 19 using the Python integration as the base. It acts as a bridge between JSL, Python, and R, and it's built on top of rpy2 in the Python environment. By doing it this way, it also provides us a Python API for your Python programs to use that is very similar to the interface that we present with JSL.
We created a package called jmpex, and it's distributed in our sample scripts Python directory. It's there, you can use it in your own code as well. Some of the important notes, the things that have changed since the previous R that's been in JMP, since JMP 14 support, is running UI code from within R or from Python is not supported. The windowing loops from Python and from JMP clash, and you'll wind up crashing JMP. There's no way around it at this point. This includes if R tries to pop up in an installation mirror, if you're trying to install packages, and you get a pop-up for an install window, install package. Please install your packages outside of JMP, outside of it in an R script so that they're there and ready for you to use when you run JMP.
Also, if you need to plot things from R, that needs to be done also without the UI. You set the device, the graphics device, to say a PNG file to a filename, run the plot, then turn the device back off, and then you can open a window with JSL that loads that file. The example of this is in the Scripting Index R Get Graphics. That command itself is deprecated, but we left the scripting index example there with the workaround on how you actually accomplish that.
The first step is getting the R environment installed and configured. Go to the cran.r-project.org site to download the version of R that's appropriate for your machine, whether it's Apple Silicon, or Intel, or Windows. Make sure it's a 64-bit distribution and install R. You also need to set up your R_HOME environment variable. R can tell you this by launching R after it's installed and doing R.home(). It'll tell you the path to what it believes its home environment variable should be.
That needs to be set because rpy2, the Python package, when it installs, it needs that R_HOME environment variable to know where your R instance is. Setting that up on Mac, you're editing your .profile, or .zshrc, or .bashrc files to add the home environment variable. Or on Windows, you open up the settings and find the environment variables and add the appropriate R_HOME environment variable. Installing jmpex.
With the R scripts, we wanted to make it as seamless as possible, so calling R Init will automatically check for jmpex and attempt to install jmpex, rpy2, and all of the appropriate dependencies that first time R Init is called. Since jmpex is a Python package, you can also install it from the Python environment with jmputils jpip. Here's the script on this page is the same sample script that you'll find in the scripting index, where import jmp, import os, set up the path, do the jmp install of our jmpex.zip package that's in the Sample Scripts directory.
How it's supposed to work. We see the R Init, and the output says we couldn't find jmpex. Running the package install, it goes through the list of the packages it needs to install, and then your environment will be properly set up. The scripting index, it is your friend. We have done a lot of work to try and present both examples for JSL, and the Python environment, and the R scripting in the scripting index. Then cover a little bit later, we'll actually look at some code. This is the other difference is the handling of R variables that have a dollar sign in them. Rpy2 does not handle that the same way, so we have to access them a little differently.
I'm going to go to the scripting index and show some of what we have and where you can find things, and then we'll talk about how they handle the dollar sign variable scripts. Let me pull the… This is JMP 19.2, and here if you go under functions in the scripting index, scripting index is Help, Scripting Index. That brings up the window. If you go to functions, we find all the R functions are easily accessible, connect, control, these are the familiar ones that have been present since JMP 14. Execute a script, the execute, you pass in a list of variables, the list of variables that come back, your R script, the Get Graphics version.
Here's the workaround for the, since you can't run UI from the R, is we take the execute, pass it the path, build the path up inside the R script, set the PNG for the output path, plot our graphics, turn the device off, and so we can run this. Here's our image that we opened in a JMP window. The JMP window we can handle just fine. It's R or Python creating a window that doesn't behave with JMP.
Then the Python, this is the JSL Python commands, they're all also here in the scripting index. Excuse me. Then there's an entire Python category in the scripting index, and this is what you can access from the Python script editor and the Python code that you can run within JMP. Jmpex, jmputils is how you can load packages from Python, jpip, if you wanted to install the jmpex directly, there's a scripting index sample right here, and this is the same as what was pasted in the other. Run the script, and if you don't already have it installed, it will install the jmpex package. Here is content on what the jmpex provides from Python.
You can get the version. The version tells us the rpy2 version. It tells us the version here is the version of this, our jmpex package. The Init, we set it up. Here, this is how you get the version of R itself. On mine, we're running version 4.5. This returns in the same manner that the previous JSL code returned. We've tried to maximize that our return values from this are the same as what you got previously. These functions here mimic what the JSL versions are. You can do this operate either from Python or JSL, but the real heart of it are two new functions, obj2r, this takes a Python object, turns it into an R object in the R environment, an R object to Python object.
The Python integration marshals between JSL and Python, but this is the code that's necessary to marshal between Python and R. If you're running from JSL, the R send do all of this under the cover, but they call directly this same Python package. I'll go back to continue the slides. I didn't cover the… We'll go back, we'll cover the variable, the R variable. We'll drag that over there to the R dollar, handling the dollar sign variables.
Here we've got a Python script where we're going to pull in, we'll ask R for its iris data frame, and R pulled the iris. This is the way R and rpy2 handle it, is the dollar sign variable is handled with the function rx2. We can get that. We can run that. This returns the R object that contains Sepal.Length. You can also do it by submitting an R code block and getting the value, calculate the mean of that value. Now, we can do that with the jmpex as well by doing a submit of the same bit of code through our submit. You see that the return value is exactly the same as the hard way of going right straight through with the rpy2 code.
There's a lot of setup that rpy2 requires that the jmpex provides for you without having to go into all of that. The R object, we can take what comes back and we can see. Before, we had the Sepal.Length. It tells us it's a rpy2 object, a FloatVector in R. We can ask for it as a Python object. Now what we have back is a NumPy array. We can do that and we can print that. Now this is we're working in Python rather than working over in R. You can bring stuff back and forth. The r2obj and obj2r help you with that without having to do lots of internal conversions.
Then you can create a Pandas DataFrame directly from taking the data frame object from R as a Pandas DataFrame that you can manipulate with Pandas in Python and all of that work, trying to make that flow more seamless and work for you. Let me see. The scripting index, we've handled the scripting index.
The last part of this is the jmpex package is there in sample code. You can look into it, look how we implemented this functionality, incorporate it into your own, or modify it and update it so that you can give us fixes if you find things that don't work for you, you can take a look at it. It's structured as a real Python package. The jmpex.zip sitting in the script index Python directory has a jmpex directory. Within that is the R.py, a license file, and the pyproject.toml tells Python how to install it, what dependencies are, and all of the pieces. This is the way an installable Python package is bundled up. When you install it, it winds up in your site-packages directory.
The pyproject.toml file contains the project, its name, the version number that we saw with the version description of it, the requirements of what the project needs to be able to be installed on your machine, and the dependencies that it will try and pull in to make sure this works on your machine, all with just the install of the jmpex package. As we were talking, the jmpex, the R.py file is the heart of all of this marshaling between R and Python. Two methods are the real heart of the entire process, the obj2r and r2obj. Because things like lists can contain other objects, these functions have to work recursively if we hit a list so that we can walk our way through it to find out all of the types, to convert all of the types when it goes between R and Python or Python and R.
We'll take a look at the code. The docs for rpy2 are at this link. Like I said, the jmpex package was designed to support and mimic what we had before in the JSL support, plus provide that interface for your Python programs and your Python scripting. We'll take a look, dive into that code directly. You have access to all of that because it's in the Sample Scripts directory. Let's go in there.
Here is the R.py file that's within the jmpex. I've unpacked the zip file into a temporary directory, and we've got the jmpex directory, we've got R.py. The initial setup is we import jmp, we set the imports to the log so that we can see our output on JMP's scripting log. There's a platform, we set up some of the imports for Pandas, NumPy, and we set. There is a default for the environment. It works really well on Mac because that's where it usually all goes. This is set for my location on Windows.
Again, this is if the system doesn't detect an R_HOME, it will default to these locations. Then Windows, another bit to keep warning messages out, needs to be set the API mode. Now we're actually into the rpy2 Python, how rpy2 interacts with R. There's a bunch of imports that we need to set things up, including converters for R to NumPy and back, and Pandas and NumPy, and some logical types. We do set platform up, even though we can't really use the graphics system other than to plot directly to a file. Then this is the Python class that supports the R connection. I've set a version.
The IsConnected simply takes a variable whether or not we've set it. The Init, this is where in one of the scripts, and we can go back to that, or we can just look, is where we set jr equals to R parentheses from our import. It calls the initialization when we constructed the R object. Here we set connected is true, we set a system locale, we try and check for make sure rpy2 is on the system, or we throw an error. The code for submit is pretty straightforward. We take the object, we set the script to it, get the result, and return that, and errors go to the JMP log.
The submit_file. Similarly, we take the file path, we send it along to R, and return the results. Like I said, the r2obj and obj2r are the key parts to this whole thing. Here is where we have to take from what's passed in, check what kind of variable is it. Is it a NumPy array? Is it one-dimensional? Is it two-dimensional? Is it a scalar vector, a scalar? Then we have to create the appropriate type in R and send that data along. Similarly, here's two-dimensional, and then more than that, we have to transpose and set that up there. If they are single variable scalars, then we have to do another set of conversions to set it appropriately.
This is a lot of work that took effort to map all these things out so that it works seamlessly, and you just have it in the jmpex rather than having to put all this boilerplate code in your own code to interact between Python and R. You'll see here in the case of the list, we have to walk through the list, enumerate through them, and call again into obj2r. We're walking recursively down the list if we have types that need to be handled.
The DataFrame, if you've got a Pandas DataFrame, then we can put that through, mark the columns, put that through to the R. Then if it's a JMP data table, you can pass the JMP data table object in. It does a conversion and sends it over to an R data frame. Similarly, a column, you can take a JMP column and send that over, either as numeric or character vector, into the R side of the environment. r2obj returns all of that data from R. Again, we have to ask, what is it on the R side, and convert that to the appropriate Python object. When it's back as a Python object, then that is easily transferable to JMP's JSL if you're working in JSL.
There's a corresponding in JMP C++ py_to_obj and obj_to_py that has to work through all of these conversions as well. I encourage you, if you're interested, unzip the file, take a look at the code and tinker with it, see what you want to do with it, and incorporate the techniques. It's powerful what can be done between the Python and JMP. Let's see, we'll go back to the slides, and that's it. Thank you for your time.
Presenter
Skill level
- Beginner
- Intermediate
- Advanced