JMP is Pythonic! Enhanced Python Integration in JMP 18

[#1]

JMP 18 has a new way to integrate with Python. The JMP 18 installation comes with an independent Python environment designed to be used with JMP. In addition, JMP now has a native Python editor and Python packages specific to JMP. This JMP Python environment has enhanced connectivity and interaction with JMP, which means using Python with JMP has never been easier.

In this series of blog posts, I introduce you to the new capabilities in the Python integration with JMP. This post walks through the new Python Script Editor in JMP. In upcoming posts, I cover other new features of the integration, as well as package management.

JMP Python Script Editor

In previous versions of JMP, the only way to execute Python code was to make a connection to an externally installed Python environment, and then use a JMP Scripting Language (JSL) wrapper function to submit the Python code to that environment. Now you can edit and submit a Python script directly in JMP! Simply open a Python script in JMP or create a new one from the File menu (File > New > Python Script).

The new Python script editor has syntax highlighting and allows you to code in Python in a more natural way. Clicking the Run Script button executes the Python code, and messages logged in Python are listed in the JMP Log. 

jmp and jmputils Python packages

Along with the new JMP Python environment come two new “JMP aware” Python packages: jmp and jmputils.

With the jmp package you can:

• Run JSL from Python using jmp.run_jsl(‘’’jsl script’’’).
• Open, close, and save data tables.
• Get information on a table open in JMP.
• Create JMP data tables, add/delete/modify JMP data table columns, and add/delete JMP data table rows.
• Access data in a JMP data table directly in-memory at the column or cell level using zero-based indexed array operators on data tables. This indexing also supports Python slice operators on both column and row indices.
• Use a JMP data table and a JMP data table column as a sequence for Python iteration.

With the jmputils package you can:

• Use the jmputils.create_jmp(dir_path) to create a command line JMP pip (jpip) script in the specified directory. This script can then be used to install packages in the JMP Python environment. Note that installing from the command line is sometimes necessary for certain python packages.
• Use the jmputils.jpip( ) function to install python packages in the JMP Python environment.

All of these functions are described in more detail in the help documentation[LINK], and there is also a new Python category in the JMP Scripting Index that lists all of these packages and functions.

Python JSL Functions

As with previous versions of JMP, you can run Python from JSL with the Python Execute( ), Python Submit( ) and Python Submit File( ) functions. You can also transfer data to and from JMP and Python with the Python Send( ) and Python Get( ) functions.

Changes to the Python JSL functions include:

• Python Connect( ) remains to support creation of a scriptable object in JMP to the local Python.
• Python Send(data_table) now creates a jmp.Datatable object in Python which is a live reference to JMP data table accessible in-memory from Python. Previously, a copy of the data_table was created as a pandas.DataFrame by exporting the table as a CSV file to disk, and having pandas read the CSV file, which could be very slow for large data tables and could cause data degradation due to binary->text->binary conversion.
• Python Get(object_name):
• If the object_name is a variable name that references an open JMP data table in Python, it returns a JSL scriptable object reference to the JMP data table referenced by the Python variable name.
• If the object_name is a Python pandas data frame, it will write the pandas data frame to a CSV file and then open that file as new JMP data table. This approach is not recommended, however, given the new ability to create and connect JMP data tables in-memory from Python.

Deprecated JSL Python functions that are no longer needed:

• Python Get Graphics( )
• Python Control( )
• Python Init( )
• Python Is Connected( )
• Python Term( )

New JSL Python functions include:

• Python Install Packages( ) allows you to install python packages in the JMP Python environment.
• Python Create JPIP( ) creates a command line script that wraps the JMP jpip and that can be used to install python packages from the command line in the operating system. The script is written to a director chosen with a directory picker dialog.

[Ceg1]

Hi,

is there a way to connect to user-installed python instance in JMP 18, as it was possible in JMP17? 

[Paul_Nelson]

No. 

That was too fraught with problems.  For some users it just worked, but far too many customers had trouble in just getting it to work.  Almost as important, a Python script running in JMP had no guarantee that it would run properly on another user's machine.  Likely wouldn't if there was any custom Python install location.  As a user, treat the JMP supplied Python as your own JMP specific Python virtual environment. 

We have put significant effort into ensuring the JMP Python is isolated from other Python installation(s) present on your system.  Nothing is preventing you from using your existing pipeline, and you can install packages into the JMP environment. You may be able to bring that workflow within JMP.  If anything, the additional capabilities of JMP 18 should make it easier for you import/export data between JMP and an external Python pipeline if desired.

We believe the benefit of a Tested and Verified Python installation that works on JMP first launch, without configuration, a better customer experience.  Having a known Python version and location lets us focus on delivering features and capabilities rather than fighting to solve issues due to unpredictable customer Python installations.  The stability, and the embedded Python allows us to provide a much deeper integration. The new ability to create, modify, and access JMP data tables in memory from Python is one such feature.

I do recognize that this does cause a transition from JMP 14-17.  JMP 18 is a fundamental shift in capabilities. Some of the initial design decisions did not hold up to the test of time.  I have tried to minimize any incompatibilities, but some changes were necessary.  Where changes were necessary, it was done in 18 to ensure JMP 18 provides a stable foundation moving forward.

[hogi]

Are there plans for Workflow Builder to accept Python as alternative to JSL code?

[Paul_Nelson]

No, not in the immediate future.  But it would be a good new feature request.

There are lots of things that can be done. The challenge, is focusing first on the ones that bring the most benefit to everybody.  The primary focus right now is on foundational issues, especially data access and interoperability between Python and JSL.

There still remains a lot to be done to make data table access fully featured from Python, such as setting expression columns, rows state columns, column formatting, ...  Making sure the marshaling of data between JSL and Python encompasses as many JMP data types as possible.  

And as to be expected with any large overhaul, there are a few bugs and small oversights.  These need to be addressed as well. While a JMP add-in that utilizes Python code works just fine, the Add-in builder does not yet know about Python files so you have to pack up the JMP add-in file  ( zip archive ) by hand.  Also, while a Python file within a project will run just fine, it does not yet know how to find other files also contained within the project file such as data tables, other data files, or additional Python scripts.  If you run across other limitations that should work, please report those to technical support.   New cool capabilities that would improve JMP's power for your work, please submit as feature requests.

[hogi]

Hi @Paul_Nelson ,
I submitted a wish to provide Python support in workflow builder.
A big win-win:

- seamless poing pong of JSL and Python code

- a nice enhancement for users who already use Workflow Builder.

Workflow Builder + Python 

[Eric_Hill]

It would be helpful to provide a link in this post to documentation of the two "JMP-aware Python packages".

Thanks!

Eric

[Eric_Hill]

Oh, I see there is a placeholder in the article for a link to such documentation, it just needs to be replaced with the actual link.

[BimodalJackal79]

If I can't use my own python interpreter, is there a solution to give the code a scheduled task to execute that python code at regular intervals to process the data to generate the chart automatically.