Issue Sending Pandas DataFrame from Python to JMP 18 Data Table

[#1]

Context:
I am using JMP 18 with Python integration to merge multiple datasets. The merge and processing in Python works correctly using pandas, but when I try to send the final pandas DataFrame back to JMP using Python Get(), the resulting JMP Data Table is empty. I want to create a JMP table from my pandas DataFrame in Python and access it in JSL.

Example of What I’m Trying:

// Send JMP tables to Python
Python Send(jmp_table1);
Python Send(jmp_table2);

// Python processing
Python Submit("
import pandas as pd

def jmp_to_pandas(jmp_table):
    cols = list(jmp_table)
    data_dict = {}
    for col in cols:
        data_dict[col.name] = list(col)
    return pd.DataFrame(data_dict)

# Convert JMP tables to pandas DataFrames
df1 = convert_jmp_to_pandas(jmp_table1)
df2 = convert_jmp_to_pandas(jmp_table2)

# Merge tables
merged_df = pd.merge_asof(df2, df1, left_on='Time', right_on='Time', direction='nearest')
");

// Attempt to bring merged DataFrame back to JMP
dt_merged = Python Get(merged_df);
dt_merged << Set Name("Merged_Data");
dt_merged << Show Window;

Problem Observed:

• dt_merged is created but contains no data. [but the data frame is generated as I can save it to csv and use it]

• Using global in Python does not seem to fix the issue.

• Python Get() does not appear to recognize the pandas DataFrame correctly.

What I Want:

• Convert a pandas DataFrame into a JMP table from Python.

• Access it in JSL with proper name, show the window, and manipulate columns.

• Avoid creating empty tables and maintain all rows/columns.

Potential Solutions I’m Considering:

• Exporting to CSV from Python and then importing to JMP - tried and works but it's something I don't want. 

• Ensuring the DataFrame is declared global before Python Get(). - didn't work

Environment:

• JMP 18

• Python 3.x (embedded)

• Using inside a JSL script

Any guidance or working examples of sending pandas DataFrames back to JMP from Python would be greatly appreciated!

[#2]

JMP 18, org not migrating to JMP19 , I am not sure whether its a test version roll-out  or a developed application. I suspect later one. 

[#3]

Maybe wait for JMP19.1 to get some missing features back: Undocumented Secrets 

besides that:

one of the reasons to switch to JMP19: from_dataframe()
Going further with Python in JMP 19 
another reason: wonderful improvements in the data grid like Column Tags and Filter Views.
Organizing and filtering data in the JMP 19 data table 

Do you have the chance to install JMP19 "manually" on a test system ?

[#4]

Yes , I can do that. I have not done it. Another reason I am not using that is the script compatibility as other users would continue using JMP18. I am already facing quite alot of issues with table referencing and joins. 

[#5]

---

@zetaVagabond1 wrote:

 I am already facing quite alot of issues with table referencing and joins. 

---

Issues  with JMP18  vs. JMP17?

[#6]

JMP uses a Jan 1, 1904 epoch for time 0 in seconds ( Mac Excel epoch date)

Python uses the standard UNIX Jan 1, 1970 epoch date for time 0 in seconds.

JMP 19 properly marshals date time values across JSL / Python boundaries.

JMP 19 has a  dt = jmp.from_dataframe(df) that will create a date table directly from the data frame in memory.  

In JMP 18 the only in-memory choice is to build the columns yourself, looping on the pandas columns and paying attention to date/time conversion.  

Alternatively,  you could go through CSV.  That's the way it was done in JMP 14-17.  But be aware you can get small binary differences in numerics when going binary -> CSV -> binary.  There are samples in JMP 18's SAMPLE_SCRIPTS/Python directory that show how to go the CSV route.    Basically, tell pandas to save as csv, then tell JMP to open the CSV.

I really suggest you upgrade to 19.  Dramatic improvements to data table handling have been made in 19.

[#7]

Rory Vaden: How to multiply your time?
https://www.youtube.com/watch?v=y2X7c9TUQJ8 

Here: Wait today for your IT department to switch to JMP19 - maybe invest some effort to convince them.
Then you can write and use some great code tomorrow.

The alternative is to start today writing code that is already outdated today.

Python with JMP19:
- easy to code
- slim
- much faster