Neural Network: Does same neural network setup produce different Rsquare values, if it run multiple times?

[#1]

Hello

I have used neural network of JMP to predict one response with 14 input variables. But the model with same setup produce different R-square value in multiple runs (plz see the attachment).

Is it obvious for neural network?

Or do I have to change same setting in the neural network to get same R-square value in multiple run?

I have few more questions:

1. it also produce inconsistent r-square value for steady increase in leaning rate

2. How to select optimal setup of neural network in JMP(number of nodes, leaning rate, etc.)?

I  appreciate for your time.

Thanking you.

Regards,

K.K.SAHOO

University of Georgia

706-351-1037

[#2 · Accepted solution]

Hello K.K.

What you are seeing is due to JMP starting the fit with different portions of the data.  You are required to use a validation set with NN and unless you set a random seed (Free add-in on JMP User Community) you will get a different or potentially different answer every time you run the analysis because of the random selection for the training and validation sets. If you are using JMP Pro you can also set the number of tours to a higher value ( > 10) and this will limit some of the random values you are seeing for R-square.  First and foremost though I recommend that you get the add-in and try it out especially if you want to get the same answer every time you run the NN on your data set. 

Hope this helps.

Best,

Bill

[#3]

Thank you Bill.

Appreciated.

Can I use the JMP generated neural network formulas to predict response for outside JMP using same inputs ( inputs not used to develop model)?

For example

-10,000 input sets of variables

-I have used one simulation tool to predict some output (Y) for only 2,000 points and develop neural network or any statistical model.

-Can I use these models to predict the Y for remaining 8,000 input sets of data?

Thank you.

Regards,

SAHOO

[#4]

Thank you Bill.

I have downloaded the random seed file (randomseedreset3.jmpaddin). but the file extension is in .jmpaddin and not able to add to the add-in.

It is asking for .jsl file extension.

How can I get the random seed file in correct extension?

or

how can I add this file as add-in?

thank you for your time.

Regards,

SAHOO

[#5]

Hello SAHOO,

I am not 100% sure as to what you are trying to do, but I will give it shot.

Translating JMP formulas to other programs can be tricky because a JMP formula is written in JSL (JMP Scripting Language)  Translating a Neural Net formula can be particularly tricky because they can be very complex.

One suggestion would be to run the NN analysis on your original 2000 points and then concatenate the other 8000 points to that table.  Your predicted values will automatically be calculated.  One thing to be aware of though is that you want to be sure that the low and high values for each input are part of the analysis for the original 2000 points.  If any of the 8000 points you add have input values outside their low and high range they would then be suspect because you would be extrapolating at that point.

As for the add-in, make sure you have JMP open and then double click the icon for the add-in.  You should be asked if you want to install it or cancel. Click install and it should load for you.

Let me know if this is what you were looking for.

Bill

[#6]

Thank you Bill.

I was able to export Neural network model from JMP as SAS file where formulas were written nicely and clearly understood for each nodes as well response (Y).

I can use these formulas to predict responses in other platform like Microsoft excel (Am I correct?).

Is it true that repeatability of neural network is not working in JMP? ( plz look at the explanation in your website. http://www.jmp.com/support/notes/20/518.html )

If I am going to use the random seed add-in, then how can I use for the neural network.

Regards,

SAHOO

[#7]

SAHOO,

The support note you reference for issues with setting the random seed was from 2007.  I am almost 100% certain that setting the random seed works in versions 10 through 12. 

As long as you can bring the formulas into Excel you can use them for your predicted responses.  I would recommend you check at least a few of your Excel predicted responses in JMP just to be sure your formulas transferred correctly.

Best,

Bill