Need help w/ Advanced JSL - Reading data from a Journal

[#1]

hello 

Need help w/ a JSL script that is so far created w/ the help of GPT4.0 but is not working. 

Script purpose: Take a single journal of analysis and extract data from it to a csv. 

In this first round I am focused on journals containing only categorical analysis. 

The journal may contain 1 or multiple categorical table for a categorical variable, followed by a shared chart and a frequency chart. I am only interested in extracting data from the tables. My actual implementation deals w/ a multinomial class of 5 classes however in the sample journal and script, I have reduced that to 2 classes (binary) for simplicity.

The categorical table contains 3 rows for a given parameter value : count, percent and pvalue. I am only interested in extracting percent values if and only if pvalue<= 0.05

I need to store the outputs to a csv w/ format: 

journal name, parameter name, parameter value1, percent class0, percent class1
journal name, parameter name, parameter value2, percent class0, percent class1

The script should understand number of categorical tables present in that journal and then extract all table information (only for pvalues<= 0.05) 

For anyone w/ limited time to look into this; 

the script runs w/o an error but does not extract data as expected. 

the main problem is at the end of the script (rows 155-175) . .Two functions are built to process based on Type(journal) . I think it gets the Type correctly but is unable to connect to the functions properly 

note: My phase 2 will consist of similar intent but w/ journals containing logistic regression results 

I have not posted that yet. I rather see this one resolved first.

Attached you will find Aircraft Incidents journal from the JMP sample data library and a sample script adapted to try for 2 class levels. 

Much appreciated 

Altug Bayram

[#2]

Is there a specific reason to extract these from Journal instead of the JMP report used to create the journal?

[#3]

Yes, the journal sizes are significantly smaller to communicate w/ my customers. My journal formats are 2 types, they either contain categorical or continuous parameter analyses. I hope extracting the data is not that difficult from a journal than compared to a report. 

[#4]

From the report you would basically just save the table and be almost done (some cleanup might be required depending on your final result)

Names Default To Here(1);

dt = Open("$SAMPLE_DATA/Aircraft Incidents.jmp");

fg = dt << Fit Group(
	cg1 = Categorical(
		ChiSquare Test Choices("Pearson Only"),
		X(:Broad Phase of Flight),
		Responses(:Fatal),
		Cell Chisq(1),
		Share Chart(1),
		Frequency Chart(1),
		Legend(0)
	),
	Categorical(
		ChiSquare Test Choices("Pearson Only"),
		X(:Weather Condition),
		Responses(:Fatal),
		Cell Chisq(1),
		Share Chart(1),
		Frequency Chart(1),
		Legend(0)
	),
	<<{Arrange in Rows(2)}
);

dt_cont = cg1 << Save Contingency Table();

[#5]

I am performing large scale analysis and have my script process broken down into multiple phases. I do have a lot of  historical  analysis results already produced, I am not looking into a solution (for the current problem) that analyzes and then tries to create a summary report (if possible) - simply because it is not practical to rerun them. Rather I like this summarization script (extracting data from a journal) to be decoupled and hence for the reasons explained working. 

My journals contain multiple reports and I have multiple journals. Overall it is a lot of data. Journal route is the most appropriate for my work. I need to do this via a script or else it would not be practical. At the moment being, I am only after a single journal of categorical table(s). Once I understand my current error and have a working script for a single journal, I should be able to expand it to go over all journals of categorical type. My other future phase is a similar need for a second journal type containing multiple multinomial logistic regressions per journal. 

Something wrong in the script as it does not communicate journal to the sub-functions. I was thinking and hoping that this error is still an easy one. I have gone through scripting index as well to see if anything else can help, no luck. 

[#6 · Accepted solution]

I'm not sure what your script is doing as you are not dealing with Table boxes (it would be very easy). The functions are also incorrectly defined which most likely explain your functions not being executed like you expect them to be.

I most likely wouldn't even bother messing with the display box the contingency table utilizes and I would parse it from text

Names Default To Here(1);

j = Current Journal();

agb = j["Fatal By Broad Phase of Flight",AlignmentGridBox(1)];
txt = agb << get text;
lines = Words(txt, "\!N");
Remove From(lines, 1, 3);

res = {{}};

For Each({line}, lines,
	res_line = {};
	vals = Words(line, "}\!t");
	Insert Into(res_line, Remove From(vals, 1)); // param name?
	Insert Into(res_line, Remove From(vals, 1)); // param value?
	Insert Into(res_line, Words(Remove From(vals, 1)[1], ",")); // class1
	Insert Into(res_line, Words(Remove From(vals, 1)[1], ",")); // class2
	
	Insert Into(res, Concat items(res_line, "\!t"));
);

dt = Open(Char To Blob(Concat Items(res, "\!N")), "text");

[#7]

hi Jarmo, 

First thanks for your support/help. I wanted to take sometime to implement your approach here to my script. I have been able to adapt it but I do have a few more questions .. I will select the above as the solution though. 

My follow up questions:

Recalling that my phase1 is a journal of pure categorical tables, number of tables / journal is variable. 

1. How to get number of categorical tables for a given journal (above case for instance, that would be 2)

2. How to get the names of such categorical tables (e.g. "Fatal By Broad Phase of Flight" and "Fatal By Weather Condition" )

3. My phase2 will be pure collection of multinomial logistic regressions. The script I have derived from your support did not directly execute but I do need to spend some time on it before bringing. I can also prepare a sample data. So let me put a placeholder on this one. 

4. What is the fastest way to get into more advanced scripting ? E.g. If I am not mistaken, in my example, properties box shed some light into potential JMP function names that may be related. I have been using using scripting index quite a bit but it has not been very effective in my case. I also recall there used to be a scripting documentation w/ the older versions, I am unsure where the last version of that exists. 

Thanks again. 

[#8]

1. Utilizing XPath is one option

obs = (j << XPath("//OutlineBox[contains(text(), 'Fatal By')]"));

2. When you have outline box references, you can use << Get title

obs << get title;

4. Utilizing Scripting Index, Scripting Guide from JMP Help (JMP Help also has JSL Syntax Reference). Also at the moment you shouldn't rely on LLM for JSL as they are very unreliable (as you can see from your script).

[#9]

I tried the obj syntax as is in the sample data and it did not work .
Name Unresolved: obj in access or evaluation of 'obj' , obj/*###*/

All my tables have a common string in the table name so I can search for that common string (like "Fatal by" in above example ) .

I assume when it works, obj will be a list so in that case I likely do not need to know number of tables in the journal

For LLMs, I agree w/ you. In the beginning the script structure is given very quickly just to find out many areas do not work. What is more frustrating is that they can get literally stupid at times... It would provide a code to which I would provide my feedback and even in some case tell it to make specific corrections .. next round, the LLM would provide identical wrong solution. I think I even saw this more than 2 times in a row. 

But many times and for simpler stuff, it can jump start me so I continue to use w/ caution of course.

[#10]

With JSL you should start interactively in JMP and let JMP create the script. Then if that isn't enough and you wish to do a bit more scripting, go briefly over Scripting Guide. Then utilize Scripting Index as your main resource (it also has links to Scripting Guide / JSL Syntax reference) and consult LLM from time to time BUT understand that LLM are generally very wrong with JSL.