So I've been doing some work with neural networks, and it's a bit of a pain having to go through the whole process every time I want to change K-fold or the number of hidden layers. So, what I'd like to to is write a script that will take an open spreadsheet, take a range of values for K-fold and number of hidden layers, stopping when R-squared reaches a specified threshold. Is this possible, or is there a simpler way to do it?
There is no interactive feature to survey a range of fitting parameter values as you want to do. I think that a script is required.
See Help > Scripting Guide and change the index to show Objects. Then locate the Neural platform in the first list. Select this item and you will see the protocol for it in the second list. These messages serve to make the platform perform the analyses that you want.
Your loop could either run a fixed number of iterations and span fitting parameter values to under- and over-fitting the data or you could reference the metric you use in each iteration and stop when your criterion is met. See the same Scripting Index when showing Functions to see the For() and While() iteration functions, respectively.
Thanks - that's what I was thinking. One (hopefully) final question - how do you read the R-squared values that you get from the training/validation tables that the model generates?
Do you mean to ask, how do you extract the R square from the report with a script?
Correct - I want the script to stop when a certain threhold R-squared value is reached, so I need to get this value somehow, and it's not clear from the documentation how to do this.
The message to launch the Neural platform returns a reference to this object. Save it in a variable, e.g., nn. That object is the analysis layer. It understands messages about computation. You need to ask the analysis layer for the reference to its associated report layer. Save this in a variable, e.g., nnr. Now, you can subscript this reference to locate any point in the display tree.
Names Default to Here( 1 ); dt = Open( "$SAMPLE_DATA/Big Class.jmp" ); nn = dt << Neural( Y( :weight ), X( :age, :sex, :height ), Informative Missing( 0 ), Validation Method( "Holdback", 0.3333 ), Fit( NTanH( 3 ) ) ); nnr = nn << Report; train r sqr = nnr["Training"][NumberColBox(1)] << Get( 1 ); valid r sqr = nnr["Validation"][NumberColBox(1)] << Get( 1 );