Windows 10 > JMP 17.2 Standard > Neural Network > Trimming Variables Procedure?

[#1]

Dear JMP Community,

I am working with JMP 17.2 Standard on Windows Pro 10.

I am exploring the predictive value of Baseline biological measures to determine the future outcome of medical treatment using Predictor Screening and Neural Networks.

I would like to know if there is a recommended automated process, including JSL scripts, that would allow me to trim the model to the fewest variables meeting predefined performance criteria (e.g., NPV ≥ 75% and PPV ≥ 75%).

Of note, I wonder if there is anything equivalent to the Stepwise regression fitting for the Fit Model platform that could be applied to Neural Networks. 

Here is a brief description of the process I am currently using:

1. Screen > Predictor Screening: Assess all available baseline variables for predictive value
2. Select the top 25 entries (most influential) and run a simple Neural Network model
   1. 3 hidden layers, KFold validation with k = 3
3. In the NN report, assess the Variable Importance and trim the bottom ~10 - 20% (Empirically determined)
4. Repeat steps 2 and 3 until the Prediction Performance falls below the desired criteria
5. Recall the previous "best" model

I suspect that JMP 17.2 Standard may not be up to the task and that I may need to upgrade to JMP Pro. Still, I would appreciate your input about possible approaches to automate/speed up my process.

Thank you.

Best regards,

TS

[#2 · Accepted solution]

You can definitely write a script (or might be better as an add-in) for something like this and you won't need JMP Pro for it (JMP Pro does have few useful features for something like this though). And there are many different ways of how you could approach this while scripting (have one window which takes care of everything and dynamically updates / have multiple windows which pop-up as necessary / ...).

I think this might require JMP Pro to work properly but this add-in can give some ideas what can be done Neural Network Tuning Add-In 

[#3 · Accepted solution]

Hi @Thierry_S,

Happy new Year !

If you're using JMP standard version, the add-in for Neural Network tuning proposed by @jthi may not work. However, @shampton82 did a great script for JMP standard version to automate the creation, fitting and comparison of Neural Networks. See the discussion here : Best Neural Net model 

There might be something to try using the Python Integration in JMP, to have more flexibility designing and fitting a neural network. @scott_allen is working on an add-in using Torch, so you might have another option in a near future.

Hope this answer might help you,