JMP 18 and its embedded Python functionality provide access to many open-source tools, which opens the door to interacting with a number of data sources, including SAS! Let's explore how the open-source Python package SASPy can be leveraged in the JMP environment. In this tutorial, we:
- Install SASPy.
- Connect to SAS using a Python script.
- Download and open data sets from SAS.
- Upload tables to SAS.
- Execute SAS code.
If you're only interested in the punchline, see the attached files for full example scripts. But if you'd like a refresher on using Python with JMP, here are some blog posts that can help:
Getting started with Python integration in JMP® 18
Creating JSL functions implemented in Python
SASPy: Connecting JMP with SAS
Maintained by SAS, the open-source SASPy enables you to interface with SAS using Python. SASPy supports SAS Viya connections, as well as local and remote connections using Integrated Object Model (IOM). Following a connection, one can use the robust API to access lists of SAS libraries; get library information, list tables, and table information; get individual column information; download tables; convert to and from pandas DataFrames; run SAS code; access the SAS log; utilize SAS statistical and machine learning procedures; and...maybe that's enough for now.
Installing SASPy with JSL
Let's start in JSL-land. For the tutorial, we need numpy, pandas, and SASPy. Run the following JSL script to install the needed packages.
Names Default To Here( 1 );
Python Install Packages( "numpy pandas saspy" );
Verify the packages install correctly by looking at the log. You'll only need to install them once (assuming it was successful) since the installed packages persist in JMP between restarts.
Connect to SAS using Python
To connect, you'll need to know some information about your SAS instance. Namely what type of connection you'll be using (Viya vs. IOM), the location of the SAS server, and any credentials you need to access it.
SASPy is highly configurable, so refer to the SASPy documentation for your specific connection scenario. We cover connecting using two methods:
- Viya, which connects using http to a Viya instance, no Java installation needed. Will not work with SAS 9.4.
- IOM, which requires Java installed on your machine. Can access local instances on Windows or remote instances on any platform.
Connect using a dictionary
Start by opening a JMP Python script window. Also open the embedded log by right-clicking and selecting "Open Log." The script below uses a Python dictionary containing connection parameters. Replacing the parameters with ones specific to your SAS instance will allow you to connect. Replace the variable in SASSession with the dictionary corresponding to your connection type, open up the embedded log using the right-click menu, and give it a run.
Note on security: While the technique below can help to verify/debug your SAS connection, saving your credentials in a separate file where you can tightly control permissions is probably a better option. Speak to your system administrator for support if you are unsure about your organization's security policies. If the steps below allow you to connect, go ahead and cheer, but then please move your credentials out of the script using the steps in the "Connecting using a configuration file" section.
Another note: We can execute Python from JSL and vice versa, so when running code, make sure to check the corner for the presence of the blue and yellow Python logo to determine if you are about to run Python or not.
And one last note: JMP passes control to the Python script while running, so it is typical that the GUI is unresponsive while the code executes.
import saspy
viya = {
'url' : 'https://notrealsas.jmp.com:1234',
'user': 'username',
'pw' : 'password',
'verify' : True,
}
iom = {
'java': r'C:\Program Files (x86)\Common Files\Oracle\Java\javapath\javaw.exe',
}
iom_remote = {
'java': r'C:\Program Files (x86)\Common Files\Oracle\Java\javapath\javaw.exe',
'iomhost': 'notrealsas.jmp.com',
'iomport': 1234,
'omruser': 'username',
'omrpw': 'password',
}
sas = saspy.SASsession(prompt=False, **viya)
print(sas)
A successful connection will show something similar to this in the embedded log:
SAS Connection established. Subprocess id is 10040
Access Method = IOM
SAS Config name = default
SAS Config file = C:\Users\you\AppData\Roaming\JMP\JMP\Python\Python311\site-packages\saspy\sascfg.py
WORK Path = C:\Users\you\AppData\Local\Temp\SAS Temporary Files\_TD16996_SAS-PF4H167P_\Prc2\
SAS Version = 9.04.01M8P01182023
SASPy Version = 5.10.0
Teach me SAS = False
Batch = False
Results = Pandas
SAS Session Encoding = wlatin1
Python Encoding value = cp1252
SAS process Pid value = 16996
If you can't connect, carefully read the log for clues, also verify your username and password (if applicable). Opening the embedded log before running your code will show print statements as well as any exceptions. Other common reasons that the connection might fail are:
- Viya
- The URL address of SAS is inaccessible or misspelled.
- Failing to format the 'url' parameter as http(s)://<domain>:<port>, for example, http://server.jmp.com:5432
- Marking the 'verify' field as "true" when the SAS instance is using self-signed certificates (warning: connecting with verify=false is a possible security risk).
- IOM
- Not having Java installed (you may need to restart JMP or your computer after installing Java).
- Incorrect path to the Java executable. Try looking at the environment value for JAVA_HOME (Mac) or searching your path for the java folder (Windows).
Connect using a configuration file
Now let's move the connection information to another place. Again, contact your system administrator to adhere to the security practices in your organization. Download the example config file (sascfg_personal.py) attached to this post and replace the values with your credentials. Take note of the SAS_config_names in this file to ensure your connection option is on that list. Then connect using the file location:
import saspy
config_file = r'/some/path/to/your/config/sascfg_personal.py'
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
print(sas)
The output should look the same as when connecting previously.
Opening a data set
Open a data set by downloading a file
One way to import SAS data sets is to download the file directly. JMP is able to import the data set file type (sas7bdat), which helps preserve data formats and labels. The Python script shown below downloads a file from SAS and uses JSL within the Python script to open the file.
import saspy
import jmp
import tempfile
config_file = r"C:\Users\user\Documents\sascfg_personal.py"
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
print("Connected to SAS", sas)
table_name = 'cars'
library_name = 'sashelp'
print('Download ' + library_name + '.' + table_name)
data = sas.sasdata(table_name, library_name)
metadata = data.contents()
remote_file = metadata['Enginehost'].query('Label1 == "Filename"')['cValue1'].item()
tmp_dir = tempfile.gettempdir()
print('Copying ' + remote_file + ' to ' + tmp_dir + '...')
sas.download(tmp_dir, remote_file)
print('Opening table...')
jmp.run_jsl('''
dt1 = Open(
"%s/cars.sas7bdat",
Use Labels for Var Names( 1 )
);
''' % (tmp_dir) )
You should see a data table open containing data from the Cars data set.
Open a data set using a pandas DataFrame
SASpy allows data sets to be converted to/from a pandas DataFrame which can be useful when used as part of a data pipeline doing transformations, cleaning, creating data, etc. JMP itself is also able to import pandas DataFrames as a JMP table, which adds even more possibilities.
In the Python example below, we open the SASHELP.class data set as a DataFrame, open a JMP table, and populate the JMP table with the values from the DataFrame using the defined utility function dataFrameToJmpTable()
import saspy
import jmp
config_file = r"C:\Users\user\Documents\sascfg_personal.py"
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
def dataFrameToJmpTable(dataFrame, table):
cols = dataFrame.columns
values = dataFrame.values
for (i,k) in enumerate(cols):
print(i, k, dataFrame[k].dtype)
l = values[:,i].tolist()
value = values[0,:][i]
print("Example value: ", value)
if dataFrame[k].dtype == object and isinstance(value, str):
table.new_column(k, jmp.DataType.Character)
table[k] = l
elif dataFrame[k].dtype == 'float64':
table.new_column(k, jmp.DataType.Numeric)
table[k] = l
elif dataFrame[k].dtype == 'int64':
table.new_column(k, jmp.DataType.Numeric, dlen=4)
table[k] = l
else:
print('Unhandled column type: {dataFrame[k].dtype} . Defaulting to string.')
table.new_column(k, jmp.DataType.Character)
table[k] = l
sasData = sas.sasdata('Class', 'SASHELP',
results='pandas',
)
dataFrame = sasData.to_df()
values = dataFrame.values
num_rows = len(values)
dt = jmp.DataTable(name='Class', rows=num_rows)
dataFrameToJmpTable(dataFrame, dt)
If you are successful, you should see the open Class data table.
Uploading files to SAS
Similar to downloading data sets, we can use the DataFrame interface provided by SASPy to upload data back to SAS or upload a JMP file. Here’s an example of uploading a JMP data table file to SAS and then importing it using the SAS procedure PROC IMPORT:
import saspy
import jmp
config_file = r"C:\Users\user\Documents\sascfg_personal.py"
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
table_name = 'animals.jmp'
remote_file = sas.workpath + table_name
local_file = jmp.SAMPLE_DATA + table_name
results = sas.upload(local_file, remote_file, overwite=True)
print(results)
command = '''
PROC IMPORT OUT= WORK.animals
DATAFILE="{jmp_file}"
DBMS=JMP REPLACE;
RUN;
'''.format(jmp_file=remote_file)
results = sas.submit(command)
print(results)
Check the embedded log or download the file to verify that it worked.
Below is an upload strategy using DataFrames. We first convert the JMP table to a DataFrame, then upload it using dataframe2sasdata() :
import saspy
import numpy as np
import pandas as pd
config_file = r"C:\Users\user\Documents\sascfg_personal.py"
sas = saspy.SASsession(prompt=False, cfgname='iom', cfgfile=config_file)
def jmpTableToDataFrame(dt) :
df = pd.DataFrame()
for idx in range( len(dt) ):
if dt[idx].dtype == jmp.DataType.Numeric:
col = np.array( dt[idx] )
df[ dt[idx].name ] = col.tolist()
elif dt[idx].dtype == jmp.DataType.Character:
col = list()
for i in range ( dt.nrows ):
col.append(dt[idx][i])
df[ dt[idx].name ] = col
else:
print("Not adding type:", dt[idx].dtype )
return df
data_table = jmp.open(jmp.SAMPLE_DATA + "Fitness.jmp")
df = jmpTableToDataFrame(data_table)
result = sas.dataframe2sasdata(df, 'Fitness', 'WORK')
print('WORK Tables:', sas.list_tables('WORK'))
Check the embedded log to see if the Fitness table shows up in the list of WORK tables.
In my experience, using entire files for the upload/download strategy tends to be the faster of the two strategies. It also limits data conversion or representation issues. On the other hand, DataFrames are widely accepted by other Python modules. I recommend using the file strategy unless you have a need to do additional Python work on the data.
Executing SAS code
In the last section we used the sas.submit() function to execute SAS code to import a file, but we can do this with any SAS code. Check out the attached saspy_cowboy_hat.jsl for an example demonstrating using SASPy within a JSL script and executing SAS code to create and view a classic SAS graph. Credit: @bryan_boone and @ChrisHemedinger.
Looking ahead to JMP 19
Scripts, add-ins, and pipelines using SASpy in JMP 18 will continue to be supported in JMP 19, meaning you can continue to use and build upon any created infrastructure when working with SAS. Additionally, JMP 19 will deliver a number of features related to SAS integration, further augmenting the scripts and techniques mentioned in this post. Some familiar features, such as the SAS export dialog and various JSL SAS integration functions will return. They are all built upon the data connector infrastructure to allow for easily configuring, sharing, and utilizing SAS connections across your organization.
Do you have JMP 18 currently and want to try out future SAS integration features before JMP 19 goes into production? A great way to provide focused feedback and direction is to apply to the JMP 19 Early Adopter (EA) program. Visit jmp.com/earlyadopter for more information.
By incorporating the robust open-source tools that Python provides into JMP, many opportunities for accessing and interacting with data become available. The full SASPy demo script can be found attached to this post.
Also, a very big thank you to Tom Weber for supporting SASPy. As an open-source package, please consider contributing improvements or suggestions to SASPy at github.
Happy scripting everyone!
Resources
SASPy Configuration
SASPy Python API
SASPy Example Jupiter notebooks
pandas Documentation
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.