You can find the syntax about the IF function here: if . Or if you are up to the challenge to learn the JMP Scripting Language, you can start here: Scripting Guide .
The If statement that I showed can be read as: if the row number is less than or equal to 700, assign a missing value, otherwise depending on whether the absolute value of difference is less than 3 standard errors, assign 1 or 0.
In another word, the formula calculates whether a prediction error is within 6 standard errors, or call it 6 sigma.
What I said that the model is doing a good job was subjective. But I am going to explain what I meant. If the prediction error has a Normal distribution, the 6 sigma brackets the middle piece of the distribution, which holds over 99% of the probability. We wish to see the prediction errors fall into this bucket with over 99% chance. Then we say that the model is doing its job.
Now for this particular data, the validation set is about 300 observations, and we see about 95% fall into this bucket. Bad news? Not necessarily. My previous statement about 99% chance was assuming there are many many more validation data can be tested, and we wish to see the 99%+ chance. For merely 300 observations, with 95%, it is a subjective judgement. If I see 80%, 70%, 60%, I will increasingly raise doubt.
I attach the file as requested.