JSL: Help accessing report information: Xpath or Tree Structure?

[#1]

Hello scripters,

  I'm having some difficulty trying to access some information in a report coming from the XGBoost add-in platform. I have a JSL program where I generate a table of different XGB models -- different depths, learning rates, iterations, etc. This code runs the XGBoost platform and then pulls information from the XGB report to store in a output data table.

  The difficulty I'm having is trying and generalize some of the code to pull information using the Outline box() and NumberColBox() commands. Attached is a sample report and a screenshot below.

  There are a total of 4 number column boxes I want to pull data from -- outlined in red in the screenshot. Normally, I would write some JSL like the following to define the outline box number and number column box number to then Get the data and have it stored in a list, which I can then assign to columns as needed in my output data table.

o_box = 1; 
t_ncol_box = 35;
n_ncol_box = 36; 
Tconfrates_ncol_box1 = 28; 
Tconfrates_ncol_box2 = 29;				
				
Tdata = w[Outline Box( o_box ), Number Col Box( t_ncol_box )] << Get;
NT = w[Outline Box( o_box ), Number Col Box( n_ncol_box )] << Get;
TConfR1 = w[Outline Box( o_box ), Number Col Box( Tconfrates_ncol_box1 )] << Get;
TConfR2 = w[Outline Box( o_box ), Number Col Box( Tconfrates_ncol_box2 )] << Get;

 This JSL works just fine with the following outputs:

Tdata = {0.92503341611747, 0.937048078393258, 0.0137071818977882, 0.0387471836917961,
0.012822908993725, 0}

NT = {6253}

TConfR1 = {1, 0}

TConfR2 = {0, 1}

  I know I can also get some of the data by referencing specifically the outline box and number column box like:

Tdata = w[Outline Box( "Fit Details" ), Number Col Box( "Training" )] << Get;

  But, because of how the report is organized, it doesn't include the number of rows used in the data set -- the next number column box with the value of 6253 in this example (there is no title). Plus, how would I reference the Confusion Matrix rates -- the pass/pass, pass/fail rates this way? A similar code for the confusion matrix rates 

TConfR1 = w[Outline Box( "Training" ), Number Col Box( "Pass" )] << Get;
TConfR2 = w[Outline Box( "Training" ), Number Col Box( "Fail" )] << Get;

will get the confusion matrix counts, not rates (granted I can calculate the rates from the counts, but accessing the information right away would be easier).

  But, when the code runs through the next iteration in the table (with different model parameters), number column box values change for the same data in the report. For example, t_ncol_box, is not 35 the second time around, it's now 43.

  So, unfortunately, it looks like the "standard" way of referencing the report and outline boxes won't really work. I have not come across this issue with other modeling platforms in JMP, like Boosted Trees, Neural Net, Bootstrap Forest, and others. I was thinking of accessing it via XPath, but I don't know it well enough to know how to correctly extract the values I'm interested in -- especially when the number column box value changes -- and the Number Column Box doesn't always have a title to work with -- for example the number of data points in the training set N, it's just the next iteration in the number column box from the training fit details (Entropy RSquare, RASE, etc.).

  I thought I can get around the "old?" or "traditional?" way with a logical statement that changed the value of the number column box variables if the model step == 1 or !=1, but after 16 models, the values changed again -- and again at 34. There must be a more efficient way to access the data without having to include logic statements for specific model step values like at i=16 or i=34. It's not so easy when the number col box value can change, and without understanding why.

  Any suggestions on how to do this with a method that is more flexible to how the report keeps changing the number col box value, and for ones without a title, I'd be very grateful. I'd like as generalizable of a solution as possible as the Y column I'm modeling could be a 2-level ordinal/nominal, 3-level ordinal/nominal, or continuous data type.

Thanks!,

DS

[#2 · Accepted solution]

You can get reference to the outline boxes easily with something like this

ob_cm = rep[OutlineBox("Confusion Matrices")];

then explore what is under those, in this case col span box might be a good reference

Show(cpb[1] << get, cpb[2] << get);

or take second table box

tb = rep[OutlineBox("Confusion Matrices"), Table Box(2)];
cpb << get as matrix;

Fit details can be accessed with similar idea but you can just use number col box name (like you already did)

tb_val = rep[OutlineBox("Fit Details"), Table Box(1), Number Col Box("Training")] << Get;
show(tb_val);

[#3]

Hi @jthi ,

  Thanks for your quick reply and suggestions. They helped and I was able to access the correct data with some slight modifications to your suggestions. I ended up going with this for my situation, but your suggestions were the right solution, thanks!

Tdata = w[Outline Box( "Fit Details" ), Number Col Box( "Training" )] << Get;

NT = w[Outline Box( "Fit Details" ), ColStackBox( 1 ), Number Col Box( 2 )] << Get;

TConfR1 = w[Outline Box( "Confusion Matrices" ), Table Box( 2 ), Number Col Box( y_levels[1] )] << Get;

TConfR2 = w[Outline Box( "Confusion Matrices" ), Table Box( 2 ), Number Col Box( y_levels[2] )] << Get;

Just needed to make sure I was accessing the right ColStackBox and Table Box under the respective Outline Box.

Thanks again for the help!,

DS

[#4 · Accepted solution]

I'm not sure how helpful this will be. But in case you or anyone else would like to see some xpath examples, here is a set of examples that I refer to when I am using xpath. You may already be aware but if not, the edit -> properties page will show you the tree structure, with interactivity when you hover over and click on areas of the report. So you can click on the area of interest (number col box, etc) and it will show you the related info in the properties pane. You can expand the 'box path' outline box in the properties pane, and it will show you the outline tree code or the xpath code depending on the drop down choice, and you can generalize from there. The right most icon in the box path area will copy the code.

You can run the whole thing, but skip to the last 20 lines or so and use an associative array on the output of the last bits of xpath.

I suspect you can use something similar to this and look for 'Training' or 'Predicted Rate' and then as Jarmo says, explore the results.

Anything commented out is not working. I haven't debugged this in years, and don't recall if that part is even valid code.

// This is a file containing some examples of xpath, which is useful for traversing display trees (outline boxes and other display boxes) and getting data from these reports. For example, getting mean and st dev from the summary stats section of a distribution report window, or an Rsquare from a fit y by x. both of these are shown below.

//Tip: go to Edit -> Properties from a report output. Expand the 'Box Path' outline box in the properties pane. Switch between Subscript and Xpath to see the different paths. and change the root to whatever is convenient.  Right most icon here is copy, so you can copy / paste the code.

//below focuses on xpath.

names default to here(1);


	open("$SAMPLE_DATA\Semiconductor Capability.jmp");
	myDist = Distribution(
		Stack( 1 ),
		Continuous Distribution(
			Column( :NPN1 ),
			Customize Summary Statistics(
				Robust Mean( 1 ),
				Robust Std Dev( 1 )
			),
			Capability Analysis( 1 )
		),
		Continuous Distribution(
			Column( :PNP1 ),
			Customize Summary Statistics(
				Robust Mean( 1 ),
				Robust Std Dev( 1 )
			),
			Capability Analysis( 1 )
		),
		Continuous Distribution(
			Column( :PNP2 ),
			Customize Summary Statistics(
				Robust Mean( 1 ),
				Robust Std Dev( 1 )
			),
			Capability Analysis( 1 )
		)
	);


reportObj=myDist<<xpath( "//OutlineBox[ text() = 'Summary Statistics']");
for(i = 1, i<=nitems(reportObj), i++, show(reportObj[i]<<get title););

/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/


strList = {};
reportObj=myDist<<xpath( "//OutlineBox[ text() = 'Summary Statistics']//StringColBox");
strList = reportObj[1] << get;
show(strList); 

/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/



//for(i = 1, i<=nitems(reportObj), i++, show(reportObj[i]<<get); strList = reportObj[i] << get;);
try(MnIndx = loc(strList, "Mean")[1], MnIndx = -1);
try(StdIndx = loc(strList, "Std Dev")[1], StdIndx = -1);

/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/

statList = {};
reportObj=myDist<<xpath( "//OutlineBox[ text() = 'Summary Statistics']//NumberColBox");
for(i = 1, i<=nitems(reportObj), i++, show(reportObj[i]<<get); statList = reportObj[i]<< get;);

statList = reportObj[1]<< get;


/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/


if(MnIndx > 0 & StdIndx > 0, 
	mean = statList[MnIndx];
	std = statList[StdIndx];
	,
	mean = -999;
	std = -999;
);
show(mean, std);

/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/


reportObj=myDist<<xpath( "//OutlineBox[ text() = 'Summary Statistics']/ancestor::OutlineBox[last()]");	//gets top ancestor
show(reportObj << get title());// {"Distributions"}
reportObj=myDist<<xpath( "//OutlineBox[ text() = 'NPN1']/ancestor::OutlineBox[last()]");	
show(reportObj << get title());// {"Distributions"}
reportObj=myDist<<xpath( "//OutlineBox[ text() = 'NPN1']//OutlineBox");
show(reportObj << get title());//  {"Quantiles", "Summary Statistics", "Capability Analysis", " Long Term Sigma"}; depending on your settings

/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/


reportObj=myDist<<xpath( "//OutlineBox[ text() = 'NPN1']//StringColBox"); // gets all the string col boxes under NPN1
for(i = 1, i<=nitems(reportObj), i++, show(reportObj[i]<<get);); // next several lines...
/*reportObj[i] << get = {"100.0%", "99.5%", "97.5%", "90.0%", "75.0%", "50.0%", "25.0%", "10.0%", "2.5%", "0.5%", "0.0%"};
reportObj[i] << get = {"maximum", "", "", "", "quartile", "median", "quartile", "", "", "", "minimum"};
reportObj[i] << get = {"Mean", "Std Dev", "N", "N Missing", "Range", "Robust Mean", "Robust Standard Deviation"};
reportObj[i] << get = {"Lower Spec Limit", "Target", "Upper Spec Limit"};
reportObj[i] << get = {"Below LSL", "Above USL", "Total Outside"};
reportObj[i] << get = {"", ""};
reportObj[i] << get = {"CP", "CPK", "CPM", "CPL", "CPU"};
reportObj[i] << get = {"Below LSL", "Above USL", "Total Outside"};
reportObj[i] << get = {"Z Bench", "Z LSL", "Z USL"};*/


/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/

	
//reportObj=myDist<<xpath( "//OutlineBox[ text() = 'Summary Statistics']"); // gets the 3 summary stat outline boxes in the report	
//for(i = 1, i<=nitems(reportObj), i++, show(reportObj[i]<<get title());); 

reportObj=myDist<<xpath( "//OutlineBox[ text() = 'Summary Statistics']//ListBox//TableBox//StringColBox");	// get all cases where there's string col box in a table box in a list box in the summary stats.
reportObj=myDist<<xpath( "//OutlineBox[ text() = 'Summary Statistics']//StringColBox");	 // same as above but less strict.
StringList = reportObj[1] << get();
for(i = 1, i <= nitems(StringList), i++,
	show(StringList[i]); // shows all the strings in the string col box
);

/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/

/*not working (and not sure if this is even remotely valid code)
list_testSS.Key=(myDist<<xpath("//*[contains( text(), 'Summary Statistics')]/following-sibling::*//StringColBox")) <<Get;
list_testSS.Key=(myDist<<xpath("//*[contains( text(), 'Mean')]")) <<Get;
///following-sibling::*//StringColBox

list_testSS.Key=(myDist<<xpath("//*[contains( text(), 'Summary Statistics')]/following-sibling::*//StringColBox")) <<Get;
list_testSS.Val=(myDist<<xpath("//*[contains( text(), 'Summary Statistics')]/following-sibling::*//NumberColBox")) <<Get;


list_levelString=(myDist<<xpath("//OutlineBox[ text() = 'Long Term Sigma']//StringColBoxHeader[text()='Capability']/..")) << get;
list_levelMeans=(myDist<<xpath("//OutlineBox[ text() = 'Long Term Sigma']//NumberColBoxHeader[text()='Index']/..")) << get;
*/


/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/

//////////
//FIT model rsq example



dt = open( "$SAMPLE_DATA\Big Class.jmp");

//fmObj = Fit Model(Y( column( yParamNames[i])), Effects( column( xParamNames[k])),
fmObj = Fit Model(Y(column("height")), Effects( column("weight")),
		 Personality(Standard Least Squares), 
			Run(
		 column( "height") << 
			{
				Plot Actual by Predicted( 0 ),
				Plot Residual by Predicted( 0 ), 
				Plot Effect Leverage( 0 ), 
				analysis of variance(0),
				lack of fit(0),
				parameter estimates(0),
				effect tests(0),
				effect details(0)
			}
	)
);

myRsqr = -9999;


//Find the table containing RSquare
fmOlbXpObj = fmObj << xpath( "//OutlineBox[ text() = 'Summary of Fit']//TableBox//StringColBox");

show(jmp version());
show(fmOlbXpObj);
show(fmOlbXpObj<<get());
show(fmOlbXpObj[1]);
show(fmOlbXpObj[1]<<get());
show(fmOlbXpObj[1][1]);

fmNames = fmOlbXpObj[1] << get(); // store the list of names from the fit model outlinebox xpath object (strings) into fmNames for use in associative array below

/*xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*/


fmList = fmOlbXpObj[1]<<get();
//fmOlbXpObj[1] is a list containing the text items in the stat table, the first of which is "RSquare". Next line gets the index (e.g. kk=1)
for(kk=1,kk<=nitems(fmList),kk++, if(lowercase(fmList[kk]) == "rsquare", tableIndex = kk; break()));
//and now get the actual numbers from the stat table
fmOlbXpObj2 = fmObj << xpath( "//OutlineBox[ text() = 'Summary of Fit']//TableBox//NumberColBox");
fmList2 = fmOlbXpObj2[1] << get();

show(fmOlbXpObj2);
show(fmOlbXpObj2<<get());
show(fmOlbXpObj2[1]);
show(fmOlbXpObj2[1]<<get());
show(fmOlbXpObj2[1][1]);

fmValues = fmOlbXpObj2[1] << get(); // store the list of values from the fit model outlinebox xpath object (numbers) into fmValues for use in associative array below


//and use the index kk to get the RSquare
myRsqr = fmList2[kk];
show(myRsqr);

// associative arrays

fitModelAA= associative array(fmNames, fmValues);
myRsqr_AA = fitMOdelAA["RSquare"];
show(myRsqr_AA);

[#5]

Hi @mikedriscoll ,

  Wow, very cool! I am familiar with both the Edit > Show Tree Structure and Show Properties options from the right-click of the gray triangle, but I was not aware of the Box Path and associated options there. That is VERY helpful, thank you. I will absolutely be using this in the future to get the XPath and Subscript info for reports. This definitely makes it a bit easier to generalize from just the Tree Structure Report, which seams to be missing some information. It would be nice if they had an option to bring up both Tree Structure and Properties information simultaneously. Thanks for the suggestion, and I definitely think this is another valid solution, so I'll mark it as well. Thanks also for the example XPath code. I think it will be helpful to study that and start to familiarize myself with how to use XPath correctly.

Thanks!,

DS

[#6]

Happy to help!

---

@SDF1 wrote:

It would be nice if they had an option to bring up both Tree Structure and Properties information simultaneously. 

---

This looks to be possible now. I don't see an 'edit -> show tree structure' in my version of JMP, but here is how I would do that.

1. From report output with Properties not shown, expand the window to the right so there is at least some "new" blank space beyond the default that JMP created when it created the report window.

2. ctrl+shift + right click in that "new" blank space, and click 'show tree structure' (I tried this with properties shown and got different results once, not sure)

3. From the original report window (not tree structure window), go to edit -> show properties