topic Re: In this situation, should I use Python or JSL for the main controlling script? in Discussions

In this situation, should I use Python or JSL for the main controlling script?

lala — Sun, 23 Feb 2025 10:44:37 GMT

Now JMP can seamlessly connect with Python, and JSL and Python can call each other.

I have a task:

1. **Receive and extract data (py):** Every minute, use Python code to receive data from an API every 3 seconds. Extract structured data from the received data in real-time and store it in a memory array A. Immediately extract the data into another memory array B. Then, immediately clear memory array A to prepare for the next batch of data (this processing time is sufficient and will not cause blocking). At the end of the minute, use Python code to create a new JMP table (let's call it Table A) and write the structured data from memory array B into this JMP table. Finally, clear memory array B.

2. **Concatenate and stock selection (JSL):** Use JSL to append the data from the newly created JMP Table A to an existing JMP Table B. Perform stock selection calculations using technical indicators in JSL on the combined Table B. Then, close JMP Table A.

This process repeats continuously.

Which of the following two approaches is better for this situation, with a focus on stable operation and reduced system resource consumption?

A. Within a single, overarching Python script, continuously run the data receiving/extraction (py) and the concatenation/stock selection (JSL) portions, using timers.

B. Within a single, overarching JSL script, continuously run the data receiving/extraction (py) and the concatenation/stock selection (JSL) portions, using timers.

Thanks!

Re: In this situation, should I use Python or JSL for the main controlling script?

lala — Sun, 23 Feb 2025 10:49:13 GMT

DeepSeek R1 's

Re: In this situation, should I use Python or JSL for the main controlling script?

lala — Sun, 23 Feb 2025 10:54:58 GMT

grok3

Re: In this situation, should I use Python or JSL for the main controlling script?

Craige_Hales — Mon, 24 Feb 2025 16:02:58 GMT

Write two separate programs and leave both running. You could use JMP to launch Python via RunProgram, or Python to launch JMP via whatever Python uses. But start with a .py and a .jsl file and run them manually until it is debugged.

Make the Python code place completed work in a directory with timestamp filenames. Make it sound an alarm if the work is piling up (because the other worker has failed.)

Make the JSL code wake up every 10 seconds to use filesInDirectory to look for new work. Make it sound an alarm if it has been >2 minutes since any work was found (because the other worker has failed.)

In both Python and JMP cases, the programs are designed to loop forever. You should not make a controller that keeps relaunching them. All the communications is done through the file system, creating and deleting files.

It is possible there might be a race between the file writer and file reader; if JMP is reading incomplete output, just wait until there are two files available and work on the next-to-last. That will be a minute behind of course.

Re: In this situation, should I use Python or JSL for the main controlling script?

john_madden — Mon, 24 Feb 2025 18:37:54 GMT

Could I append a (possibly lazy, sorry) question on the same general topic?

I have a large codebase in JSL (10,000+ lines) that does a whole variety of sequential pre-processing, transforming, parsing, and re-organnizing a large automatic, monthly data download I get from my institutional RDBMS's that consists each time of a dozen files in various formats (xml, csv, xls, etc.).

I am gradually moving toward retirement and I don't want my code and the very valuable resulting clean, useful JMP database to sink into oblivion. However, my best colleagues to take over the project are all Python programmers and don't really feel emthusiastic about learning JSL.

My idea is to factor my JSL code into logical modules of modest size (say ~500 lines of code), each of which performs a well-defined step in the overall process with clearly defined inputs and outputs. THEN, I would wrap each module into some sort of Python wrapper that my colleagues could work with, avoiding having to learn the nitty-gritty details of the JSL. I might also include some Python hooks in the JSL that might allow them to extract basic troubleshooting information when the JSL runs into problems.

Here's my question: what is the best video, white paper, manual chapter, discussion thread, book, seminar, etc. on this site that could help me (a complete Python newbie) figure out what kinds of hooks and wrappers might be the things I am looking for … with examples of Python code that controls and "sniffs" existing JSL code in this way.

Maybe unanswerable question, but any suggestions would be welcome.

Re: In this situation, should I use Python or JSL for the main controlling script?

Craige_Hales — Mon, 24 Feb 2025 19:38:41 GMT

@Paul_Nelson

Re: In this situation, should I use Python or JSL for the main controlling script?

jthi — Mon, 24 Feb 2025 19:45:41 GMT

These might give some ideas

New in JMP® 18: Python jmp.DataTable and pandas.DataFrame
JMP and Python: The Next Step
Scripting Guide > Python
Check things from community written by @Paul_Nelson

And maybe this but start with the ones mentioned earlier

Re: In this situation, should I use Python or JSL for the main controlling script?

Paul_Nelson — Mon, 24 Feb 2025 20:08:44 GMT

Breaking up the JSL into modular well defined and documented components is the biggest and best thing you can do.

The Python scripting can do a jmp.run_jsl('' script code ''') so if your routines been made into JSL functions, they can be called from Python. If you want to go deeper, you'll need to learn Python to create Python based objects that behave in a way a Python programmer would expect. If you can make your routines small enough and well documented, the it can be their job to translate from JSL to Python, as long as the algorithm has been well documented.

Also, the Python support in JMP is advancing with every release, so workarounds necessary in 18 may be handled natively in JMP 19, or 20,.. The current focus in JMP 18, and 19 has been data interoperability. There isn't yet as the Python equivalent of JSL function Python Execute( ), where lists of inputs, outputs and code are sent across the boundary. The best that can be emulated would be doing something like.

import jmp jmp.run_jsl(''' ... Include("Path_to_file"); v1 = Python Get("py_var1"); v2 = Python Get("py_var2"); result = my_jsl_fn(v1,v2); Python Send(result); ... ''')

There is some support in 19 to interoperate from Python by being able to directly get/set global namespace and here namespace variables directly from Python, so the above Python Get() may not strictly be necessary in JMP 19 and beyond.

That is good topic for us to look at, better ability expose JSL functions to the Python environment.

Re: In this situation, should I use Python or JSL for the main controlling script?

Paul_Nelson — Mon, 24 Feb 2025 20:34:10 GMT

I agree with @Craige_Hales on the need to test each path. When large data and high performance come into play there is no substitute for testing in your environment. No real shortcut to say one is better than the other without testing and making measurements. Even then performance tuning tips and tricks can be specific to each language.

There are some general principles.

Do not handle, move, duplicate the data any more than strictly necessary.
Pay attention to O( ) of your code, Unroll loops where possible avoid n^2 or worse algorithms.
Minimize filesystem access in critical areas and make sure memory doesn't grow beyond physical memory or you will be swapping to disk.
- Physical RAM vs Local SSD vs HDD vs Network drive.

In the quest for absolute performance you may need to test sections of your code to see which performs faster, JSL or Python. For example, it may be faster to process data as a numpy array or pandas dataframe and convert to a data table all at once vs appending to a large data table in JSL (I haven't tested this, so I don't know, I'm just making an example.) You need to test each major component of the pipeline when performance is critical.

Re: In this situation, should I use Python or JSL for the main controlling script?

john_madden — Mon, 24 Feb 2025 21:00:56 GMT

Thank you!! I've downloaded all of them and will dig down!!!

Re: In this situation, should I use Python or JSL for the main controlling script?

john_madden — Mon, 24 Feb 2025 21:07:51 GMT

Paul, thanks so much. Your comments tell me I am somewhere in the correct ballpark. I am SO appreciative of all you are doing for the JMPing Pythons (or Pythonic JMpers) out there. I'm positive there are a lot of big, established legacy JSL projects out there in this situation of not wanting to be rewritten, but needing to add mechanisms to "unzip their fly" as it were (forgive me) for Python folk to peek inside.

Gratefully, John

Re: In this situation, should I use Python or JSL for the main controlling script?

lala — Tue, 25 Feb 2025 03:00:16 GMT

Thank you all experts for your answers.

I conducted a comparison between Python and JSL for a complex indicator calculation.
The time consumption differs by nearly 10 times.
I ran the full matrix when using JSL.
Since I'm not familiar with Python, I could only ask various AIs to implement it in Python according to this JSL logic.
The result shows that the modification by grok3 is the best.