One year ago, JMP users who also used Python made it clear they thought our Python interface was less than optimal. Ouch! We thought the JMP and Python integration we introduced more than five years ago was hitting the mark. Realizing it was only satisficing (in other words, that it was good enough but not the best), we decided to address the challenge head on.
JMP Principal Systems Developer Paul Nelson @Paul_Nelson spearheaded the effort to create a consistent, full-featured, integrated development environment for writing and executing Python scripts within JMP in JMP 18.
I recently spoke with Paul about the upgraded JMP and Python integration. Paul emphasized that the goal was to provide a significantly more productive environment for Python developers. Some key improvements include:
A consistent environment that will work immediately when installed, without configuration. Python supports unlimited customer Python environments. Adding new JMP and Python capabilities that could not be tested in unlimited customer Python environments was untenable. The solution? JMP 18 includes an embedded version of Python that is the basis for a fully tested environment that will work immediately upon installation.
Direct memory access to Python data tables in a live environment. In previous versions, Python users accessed a copy of a JMP data table. Now, JMP 18 provides direct access to JMP data tables from the Python environment, so Python users can access, create, and modify JMP tables from Python scripts within JMP.
A new Python Script Editor that uses Python commenting and code coloring, as well as an embedded log. The previous integration used JMP Scripting Language (JSL). JSL was familiar to JSL developers but was often uninviting to JMP users with Python experience but no JSL experience. In JMP 18, users can open and run Python scripts directly in the JMP script editor and run JSL from their Python scripts. The JMP Scripting Index documents and gives samples of the new functionality. Having a consistent environment for Python scripts also provides a foundation for writing add-ins using Python as well as JSL.
Hear more from Paul:
Interview with Paul Nelson.mp4
Video Player is loading.
Current Time 0:00
/
Duration 6:08
Loaded: 0%
0:00
Stream Type LIVE
Remaining Time -6:08
1x
Chapters
descriptions off, selected
captions settings, opens captions settings dialog
captions off, selected
en (Main), selected
This is a modal window.
Beginning of dialog window. Escape will cancel and close the window.
End of dialog window.
This is a modal window. This modal can be closed by pressing the Escape key or activating the close button.
By the way, not being a Python programmer, I hadn’t heard the rumor that Python originator, Guido van Rossum, was reading scripts from “Monty Python's Flying Circus” when he started working on the new language. He wanted a short, unique, intriguing name, so he decided to call it Python. Sounds plausible!
Comment / Question on bringing data back to JMP from Python.
Currently we can do this :
dt = Python Get( df )
where 'df' is a Python dataframe.
I have noticed that what the above command actually does is write the dataframe out to a CSV in the $TEMP folder, then invoke a JSL to read it in, then delete that temporary CSV. For large files there's obviously an overhead to both writing the file and scanning it / reading it in.
It would be very nice if this could be an in-memory transfer direct from Python to JMP
In JMP 14-17 both the Python Send ( dt ) and Python Get( df ) operated by saving CSV through a temporary file. This is one of the major changes in JMP 18. With JMP 18, the Python Send ( dt ) now creates a jmp.DataTable object in Python. This is not a copy it is a live in memory reference to thedata table. It is able to be read, and modified directly from Python. From JSL if you do a Python Get ( dt ), the data table you will get the in-memory data table reference. Left for compatibility concerns was the original Python Get( df ); method of getting a pandas dataframe into a JMP data table as a copy.
You now have the power to take a data frame in Python and simply iterate on the data frame columns and create in-memory, the equivalent JMP data table. See jmp.DataTable. under the new Python category in JMP 18's scripting Index. There is also a Python subdirectory to Sample Scripts. In there you will find a JMP2pandas.py script that shows two ways to create in-memory a pandas.DataFrame from a jmp.DataTable object, and one example of creating the jmp.DataTable object from the pandas.DataFrame ( in-memory )
# create JMP DataTable from pandas DataFrame
dt2 = jmp.DataTable('BC2',df.shape[0])
for j in range( df.shape[1] ):
dt2.new_column(dt[j].name, dt[j].dtype)
dt2[j] = list(df.iloc[:,j])
Where df is a pandas dataframe, and dt2 is the new JMP DataTable. ( note dt is an also a JMP jmp.DataTable object, the type parameter is a jmp.DataType enumeration value. ). The example file opened 'Big Class.jmp' as dt, creates a pandas.DataFrame, the creates dt2 from the data frame.
Formula columns do not support Python code in JMP 18. It is something that is being investigated, and would be a great feature. There was an enormous amount of infrastructure that needed to be added to make JMP 18 robust, work on startup, support a Python aware script editor, enhanced error reporting, add scripting index, and opening up a bridge from Python to JMP. Many things that had been lacking for some time, have now been enhanced to make what be believe to be an inviting and productive Python environment within JMP.
Now that the heavy lifting has been done, we are not calling it a day. Much more to come.
Follow up note on Expression columns. While the jmp.DataTable.Column object does not support reading or writing the column from Python, you can create the column from Python and work around the issue using jmp.run_jsl()
...
# with a data table dt, having columns 'list' and 'np.array'
# add a new expression column with a formula using run_jsl()
# create an expression column from Python
dt.new_column('expression', jmp.DataType.Expression)
jmp.run_jsl('''
dt = Current Data Table();
:expression << formula(:list + :np.array);
''')
...
Getting an Image from a column is a bit more involved, but is possible using the run_jsl(), but that is a good topic for a blog post all of its own.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.