JMP 18 and its embedded Python functionality provide access to many open-source tools, which opens the door to interacting with a number of data sources, including SAS! Let's explore how the open-source Python package SASPy can be leveraged in the JMP environment. In this tutorial, we:
If you're only interested in the punchline, see the attached files for full example scripts. But if you'd like a refresher on using Python with JMP, here are some blog posts that can help:
Getting started with Python integration in JMP® 18
Creating JSL functions implemented in Python
Maintained by SAS, the open-source SASPy enables you to interface with SAS using Python. SASPy supports SAS Viya connections, as well as local and remote connections using Integrated Object Model (IOM). Following a connection, one can use the robust API to access lists of SAS libraries; get library information, list tables, and table information; get individual column information; download tables; convert to and from pandas DataFrames; run SAS code; access the SAS log; utilize SAS statistical and machine learning procedures; and...maybe that's enough for now.
Let's start in JSL-land. For the tutorial, we need numpy, pandas, and SASPy. Run the following JSL script to install the needed packages.
Names Default To Here( 1 );
// install numpy and pandas packages
Python Install Packages( "numpy pandas saspy" );
Verify the packages install correctly by looking at the log. You'll only need to install them once (assuming it was successful) since the installed packages persist in JMP between restarts.
To connect, you'll need to know some information about your SAS instance. Namely what type of connection you'll be using (Viya vs. IOM), the location of the SAS server, and any credentials you need to access it.
SASPy is highly configurable, so refer to the SASPy documentation for your specific connection scenario. We cover connecting using two methods:
Start by opening a JMP Python script window. Also open the embedded log by right-clicking and selecting "Open Log." The script below uses a Python dictionary containing connection parameters. Replacing the parameters with ones specific to your SAS instance will allow you to connect. Replace the variable in SASSession with the dictionary corresponding to your connection type, open up the embedded log using the right-click menu, and give it a run.
Note on security: While the technique below can help to verify/debug your SAS connection, saving your credentials in a separate file where you can tightly control permissions is probably a better option. Speak to your system administrator for support if you are unsure about your organization's security policies. If the steps below allow you to connect, go ahead and cheer, but then please move your credentials out of the script using the steps in the "Connecting using a configuration file" section.
Another note: We can execute Python from JSL and vice versa, so when running code, make sure to check the corner for the presence of the blue and yellow Python logo to determine if you are about to run Python or not.
And one last note: JMP passes control to the Python script while running, so it is typical that the GUI is unresponsive while the code executes.
import saspy
viya = {
'url' : 'https://notrealsas.jmp.com:1234', # must contain http or https
'user': 'username',
'pw' : 'password',
'verify' : True, # verify certificates
}
iom = {
# location of java on your machine
'java': r'C:\Program Files (x86)\Common Files\Oracle\Java\javapath\javaw.exe',
}
iom_remote = {
'java': r'C:\Program Files (x86)\Common Files\Oracle\Java\javapath\javaw.exe',
'iomhost': 'notrealsas.jmp.com',
'iomport': 1234,
'omruser': 'username',
'omrpw': 'password',
}
# connect to SAS
# replace the second argument with your connection dict, mind the **
sas = saspy.SASsession(prompt=False, **viya)
print(sas)
A successful connection will show something similar to this in the embedded log:
SAS Connection established. Subprocess id is 10040
Access Method = IOM
SAS Config name = default
SAS Config file = C:\Users\you\AppData\Roaming\JMP\JMP\Python\Python311\site-packages\saspy\sascfg.py
WORK Path = C:\Users\you\AppData\Local\Temp\SAS Temporary Files\_TD16996_SAS-PF4H167P_\Prc2\
SAS Version = 9.04.01M8P01182023
SASPy Version = 5.10.0
Teach me SAS = False
Batch = False
Results = Pandas
SAS Session Encoding = wlatin1
Python Encoding value = cp1252
SAS process Pid value = 16996
If you can't connect, carefully read the log for clues, also verify your username and password (if applicable). Opening the embedded log before running your code will show print statements as well as any exceptions. Other common reasons that the connection might fail are:
Now let's move the connection information to another place. Again, contact your system administrator to adhere to the security practices in your organization. Download the example config file (sascfg_personal.py) attached to this post and replace the values with your credentials. Take note of the SAS_config_names in this file to ensure your connection option is on that list. Then connect using the file location:
import saspy
# replace with path to your config file
config_file = r'/some/path/to/your/config/sascfg_personal.py'
# replace the cfgname argument with the connection in your config file
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
print(sas)
The output should look the same as when connecting previously.
One way to import SAS data sets is to download the file directly. JMP is able to import the data set file type (sas7bdat), which helps preserve data formats and labels. The Python script shown below downloads a file from SAS and uses JSL within the Python script to open the file.
import saspy
import jmp
import tempfile
config_file = r"C:\Users\user\Documents\sascfg_personal.py"
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
print("Connected to SAS", sas)
table_name = 'cars'
library_name = 'sashelp'
print('Download ' + library_name + '.' + table_name)
data = sas.sasdata(table_name, library_name)
metadata = data.contents()
remote_file = metadata['Enginehost'].query('Label1 == "Filename"')['cValue1'].item()
# save in temp
tmp_dir = tempfile.gettempdir()
print('Copying ' + remote_file + ' to ' + tmp_dir + '...')
sas.download(tmp_dir, remote_file)
print('Opening table...')
jmp.run_jsl('''
dt1 = Open(
"%s/cars.sas7bdat",
Use Labels for Var Names( 1 )
);
''' % (tmp_dir) )
You should see a data table open containing data from the Cars data set.
SASpy allows data sets to be converted to/from a pandas DataFrame which can be useful when used as part of a data pipeline doing transformations, cleaning, creating data, etc. JMP itself is also able to import pandas DataFrames as a JMP table, which adds even more possibilities.
In the Python example below, we open the SASHELP.class data set as a DataFrame, open a JMP table, and populate the JMP table with the values from the DataFrame using the defined utility function dataFrameToJmpTable()
import saspy
import jmp
config_file = r"C:\Users\user\Documents\sascfg_personal.py"
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
def dataFrameToJmpTable(dataFrame, table):
# Populate a jmp data table with a pandas dataframe
# Credit: Paul Nelson
cols = dataFrame.columns
values = dataFrame.values
for (i,k) in enumerate(cols):
print(i, k, dataFrame[k].dtype)
l = values[:,i].tolist() # turn pandas (numpy) column to a list
# create new columns based on the data type of the pandas column
value = values[0,:][i] # get example value from the i'th column
print("Example value: ", value)
if dataFrame[k].dtype == object and isinstance(value, str):
table.new_column(k, jmp.DataType.Character)
table[k] = l
elif dataFrame[k].dtype == 'float64': # numpy.number:
table.new_column(k, jmp.DataType.Numeric)
table[k] = l
elif dataFrame[k].dtype == 'int64': # or 'int32'
table.new_column(k, jmp.DataType.Numeric, dlen=4)
table[k] = l
else:
print('Unhandled column type: {dataFrame[k].dtype} . Defaulting to string.')
table.new_column(k, jmp.DataType.Character)
table[k] = l
sasData = sas.sasdata('Class', 'SASHELP',
results='pandas',
# dsopts={'where':'Age > 12',}
)
dataFrame = sasData.to_df()
values = dataFrame.values
num_rows = len(values)
dt = jmp.DataTable(name='Class', rows=num_rows)
dataFrameToJmpTable(dataFrame, dt)
If you are successful, you should see the open Class data table.
Similar to downloading data sets, we can use the DataFrame interface provided by SASPy to upload data back to SAS or upload a JMP file. Here’s an example of uploading a JMP data table file to SAS and then importing it using the SAS procedure PROC IMPORT:
import saspy
import jmp
config_file = r"C:\Users\user\Documents\sascfg_personal.py"
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
table_name = 'animals.jmp'
remote_file = sas.workpath + table_name
local_file = jmp.SAMPLE_DATA + table_name
results = sas.upload(local_file, remote_file, overwite=True)
print(results)
# import uploaded file as a data set, replace data set if existing
command = '''
PROC IMPORT OUT= WORK.animals
DATAFILE="{jmp_file}"
DBMS=JMP REPLACE;
RUN;
'''.format(jmp_file=remote_file)
results = sas.submit(command)
print(results)
Check the embedded log or download the file to verify that it worked.
Below is an upload strategy using DataFrames. We first convert the JMP table to a DataFrame, then upload it using dataframe2sasdata() :
import saspy
import numpy as np
import pandas as pd
config_file = r"C:\Users\user\Documents\sascfg_personal.py"
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
def jmpTableToDataFrame(dt) :
# Credit to Paul Nelson again
# Creating a Pandas dataframe from a JMP DataTable
df = pd.DataFrame()
for idx in range( len(dt) ):
if dt[idx].dtype == jmp.DataType.Numeric:
# creates numeric column directly using Python's Buffer Protocol
col = np.array( dt[idx] )
df[ dt[idx].name ] = col.tolist() # make it a list for pandas.
elif dt[idx].dtype == jmp.DataType.Character:
# create a list by iterating through values
col = list()
for i in range ( dt.nrows ):
col.append(dt[idx][i])
# Build character column from list
df[ dt[idx].name ] = col
else:
print("Not adding type:", dt[idx].dtype )
return df
data_table = jmp.open(jmp.SAMPLE_DATA + "Fitness.jmp")
df = jmpTableToDataFrame(data_table)
result = sas.dataframe2sasdata(df, 'Fitness', 'WORK')
print('WORK Tables:', sas.list_tables('WORK'))
Check the embedded log to see if the Fitness table shows up in the list of WORK tables.
In my experience, using entire files for the upload/download strategy tends to be the faster of the two strategies. It also limits data conversion or representation issues. On the other hand, DataFrames are widely accepted by other Python modules. I recommend using the file strategy unless you have a need to do additional Python work on the data.
In the last section we used the sas.submit() function to execute SAS code to import a file, but we can do this with any SAS code. Check out the attached saspy_cowboy_hat.jsl for an example demonstrating using SASPy within a JSL script and executing SAS code to create and view a classic SAS graph. Credit: @bryan_boone and @ChrisHemedinger.
Scripts, add-ins, and pipelines using SASpy in JMP 18 will continue to be supported in JMP 19, meaning you can continue to use and build upon any created infrastructure when working with SAS. Additionally, JMP 19 will deliver a number of features related to SAS integration, further augmenting the scripts and techniques mentioned in this post. Some familiar features, such as the SAS export dialog and various JSL SAS integration functions will return. They are all built upon the data connector infrastructure to allow for easily configuring, sharing, and utilizing SAS connections across your organization.
Do you have JMP 18 currently and want to try out future SAS integration features before JMP 19 goes into production? A great way to provide focused feedback and direction is to apply to the JMP 19 Early Adopter (EA) program. Visit jmp.com/earlyadopter for more information.
By incorporating the robust open-source tools that Python provides into JMP, many opportunities for accessing and interacting with data become available. The full SASPy demo script can be found attached to this post.
Also, a very big thank you to Tom Weber for supporting SASPy. As an open-source package, please consider contributing improvements or suggestions to SASPy at github.
Happy scripting everyone!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.