ChatGPT JSL Script

[#1]

I'm not very proficient at JSL scripting so, as a test, I asked ChatGPT-4 to write a JSL script for me. I used the "Corn, Wheat, Soybean Production" data table from the JMP sample data. My request was:

"I have a dataset to analyze corn, wheat and soybean production. It consists of 5 columns (Year, State, Commodity, Commodity acres planted, and total acres planted). Can you generate a JMP JSL script to analyze this data?"

In response, it generated the JSL script shown below. However, when I run the script, it doesn't execute and there are no error messages. I don't know enough about JSL to know why it's not working.

I'm just curious if any experienced JSL programmers would like to comment on the quality of the ChatGPT script and perhaps point out to me what is not working. Is this script total garbage or is there maybe one small error that, if fixed, would make it work? I did insert the pathnames manually so it's possible that I screwed that up. I am running JMP on a Mac so the pathnames are formatted accordingly.

Here is the script ChatGPT generated:

// Import the data from a JMP file
dt = Open( "$SAMPLE_DATA/crop_production.jmp" );

// Calculate summary statistics for each commodity
commodities = {"corn", "wheat", "soybean"};
For( i = 1, i <= N Items( commodities ), i++,
	dt_sum = dt << Summary(
		Group( :Commodity ),
		Weight( :Commodity acres planted ),
		Statistics( Sum, Mean, Min, Max, Std Dev ),
		Subsets( :Year, :State )
	);
	dt_sum << Select Where( :Commodity == commodities[i] );
	dt_sum << Save Selected Rows(
		"$SAMPLE_OUTPUT/" || commodities[i] || "_summary.jmp",
		"jmp"
	);
	dt_sum << Delete Rows;
);

// Create a scatterplot matrix of the data
dt << Scatterplot Matrix(
	Y( :Commodity acres planted, :Total acres planted ),
	Group By( :Commodity ),
	By( :Year, :State )
);

// Save the scatterplot matrix as an interactive HTML file
dt << Save Interactive HTML(
	"$SAMPLE_OUTPUT/scatterplot_matrix.html",
	Scatterplot Matrix
);

// Close the dataset
Close( dt, NoSave );

[#2]

Hello,

Just made some basic fixes to the file opening and the commodities list.

// Import the data from a JMP file
dt = open( "$SAMPLE_DATA/Corn Wheat Soybean Production.jmp" );

// Calculate summary statistics for each commodity
summarize( commodities = by( :Commodity ) ); // Creates list of unique items in : Commodity
For( i = 1, i <= N Items( commodities ), i++,
	dt_sum = dt << Summary(
		Group( :Commodity ),
		Weight( :Commodity acres planted ),
		Statistics( Sum, Mean, Min, Max, Std Dev ),
		Subsets( :Year, :State )
	);
	dt_sum << Select Where( :Commodity == commodities[i] );
	dt_sum << Save Selected Rows(
		"$SAMPLE_OUTPUT/" || commodities[i] || "_summary.jmp",
		"jmp"
	);
	dt_sum << Delete Rows;
);

// Create a scatterplot matrix of the data
dt << Scatterplot Matrix(
	Y( :Commodity acres planted, :Total acres planted ),
	Group By( :Commodity ),
	By( :Year, :State )
);

// Save the scatterplot matrix as an interactive HTML file
dt << Save Interactive HTML(
	"$SAMPLE_OUTPUT/scatterplot_matrix.html",
	Scatterplot Matrix
);

// Close the dataset
Close( dt, NoSave );

The output from the script were not useful. It was 3 subset data tables that could better be displayed in tabulate or through another platform. The visualization is by:State, which, depending on what analytics you're looking for, may not be useful.  Either way, with only 3 points per plot, it takes up a lot of space.  The interactive HTML didn't save either.  I didn't fix anything other than what I stated earlier.

--- My opinion below! ---

I've prompted ChatGPT-4 for some expression manipulations and found it to be lacking in understanding of how to use correct syntax, which, I as a learner, am still struggling to understand myself. ChatGPT-4 is not a useful tool to aid in JMP scripting at this time.

To respond to your prompt, ChatGPT works best with specifics.  For example:

"I have a dataset to analyze corn, wheat and soybean production. It consists of 5 columns (Year, State, Commodity, Commodity acres planted, and total acres planted). Can you generate a JMP JSL script to analyze this data? Statistics of interest are: stat1, stat2, ...., stat N"

Also, giving ChatGPT a persona may help improve the quality of output. For example:

"You are an expert statistician with 30 years experience in analysis of food usage metrics from various farms and health agencies across the world. A client has presented a dataset to you for analysis which contains data regarding corn, wheat and soybean production. The dataset consists of 5 columns (Year, State, Commodity, Commodity acres planted, and total acres planted). Generate a JMP JSL script to analyze this data. Statistics of interest are: stat1, stat2, ...., stat N. Create a tabulation table of these metrics. Create scatter plots that compare each state, per year, per commodity. Create X visualization containing Y1, Y2, and Y3 metrics as a heatmap, boxplot, etc"

[#3]

Hello, 

For what it's worth, the Save Interactive HTML message needs to be sent to a report rather than a data table object.

For a simple Scatter Plot using Big Class.jmp, with no By variable, on Windows it would be something like this:

dt = open( "$SAMPLE_DATA/Big Class.jmp" );
rep = dt << Scatterplot Matrix(Y( :height, :weight ));
rep << Save Interactive HTML("C:/temp/JMPUserCommunityGPT.html");

~John

[#4]

The way I would use ChatGPT+, Bing, etc. is to understand what analysis I can do, giving as much context as possible in the prompt, but without asking for a specific solution in code. This can give you an idea and some references of what other industries, academics or researchers have been doing.

As with any other model, training data will be essential to guarantee quality in their outputs. Open-source languages such as Python have a lot more content, communities, and libraries to train, predict, and improve via user feedback.

For proprietary software focused on point-and-click, such as JMP, there should be an effort to integrate a GPT-like assistant in their product. Something like the search bar in JMP 17, but with a Q&A interface.

Excel is planning to do such integration, which Microsoft calls Copilot.
https://m.youtube.com/watch?v=I-waFp6rLc0

For coding environments, GitHub Copilot has been there helping experts write, debug, and document their code.

AWS just released theirs, named Code Whisperer, but many others are coming out (e.g., https://www.phind.com/)

Exciting times!

[#5]

Echoing others, I've had mixed use of chatGPT for jsl code. That being said, I expected it to do much worse given jsl is much more niche than python, etc.

For beginners, the code definitely lacks necessary syntax to make things work seamlessly, but I've found it does give great insight into various capabilities/functions available to do a job. So, while the syntax isn't correct, I now have much better keywords to search deeper to find out the proper use.

For instance, chatGPT gives me the function I probably want, but I can then use the scripting index to learn more about how that function is used correctly.

[#6]

What are some specific tasks or actions that developers can perform using the ChatGPT JSL Script?

[#7]

I found this video useful for thinking about what generative AIs can do. What I got out of it: imagine towers on a 2D plane where a tower might represent Python syntax or C++ syntax or JSL syntax or Algorithm for computing Means or Algorithm for Sorting or etc. There are spaces between those towers that can be interpolated. The interpolated answers will usually not be better than the towers they are formed from, usually a bit worse.

He's also making a point about there not being an intelligent entity in the AI, just a body of collected knowledge that lets you collaborate with all the people that contributed.

minutes 15-25 if you don't want to watch for an hour...

[#8]

I find this discussion rather silly.  ChatGPT's "abilities" depend on the training data it had - I strongly suspect there is less jsl training data compared with Python, for example.  I've read various accounts where ChatGPT greatly facilitated writing code in Python or R, much more commonly used.  If JMP were to use AI and train it on jsl, then I would expect similar success - but it will require considerable training data specific to the task - LLMs are amazing in the scope of what they can do, but the technology behind them should be far more powerful if used for specific tasks where it is trained on relevant data.  Also, many JMP tasks are already automated in ways that make having ChatGPT help unnecessary - isn't that one of the main virtues of JMP, it's ability to do many things without the need to script things from scratch?  It is for me, anyway.

JMP already has the capability to generate a script from a series of commands you use for some analysis.  I guess the question is what types of tasks would using ChatGPT to create the script be better than what JMP already can do.  Indeed, there are tasks where scripts are required because JMP does not have a built in capability to do what I want - but I've used the Community to provide me with scripts in those few cases I've had.  I trust the JMP Community more than I trust ChatGPT!  But I don't want to give anyone the impression that I am downplaying ChatGPT - I've been very impressed by what it can do, if used carefully.  Writing jsl is not one of those things.

[#9]

This thread is very interesting.

I've used ChatGPT for a few things and was less than impressed.  I asked it to write some Oracle PL/SQL code.  The code looked great but didn't run.  After some tweaking I got it to run but the results were incorrect. 

I asked it a question about Bitbucket and got another excellent looking answer, but it was incorrect.

Caveat emptor!  Hallucinations rule!

[#10]

JMP needs to get ahead of this. There is great potential for a custom LLM for JSL given the general models have diluted training data. It would be highly useful to generate JSL script via prompt. JSL is a quirky langauge. Automation and learning from a chatbot would be ideal