topic Re: JMP script verification: discrepancy between results from script run and manual run in Discussions

JMP script verification: discrepancy between results from script run and manual run

TinaJon — Fri, 09 Jun 2023 18:08:49 GMT

Hi I am verifying a JMP script for a linear regression. the last verification was done 10 years ago using JMP9.0. I am now verifying the same script using JMP14.2.0

I found 99% of the results match between a manual run and a script run, however, there is a 0.001 difference between the R2 (R square) from two runs. It really puzzles me because the SSE and SST identical between the manual runs and the script runs, and because R2=1-SSE/SST, so R2 is suppose to be the same.

what is wrong? I thought it may be the rounding of my manual run, but tried different decimals and the data is still 0.001 different.

I cannot look at the JMP script because it is encrypted and protected.

Any thoughts what went wrong? Any....

your input is very helpful!

Re: JMP script verification: discrepancy between results from script run and manual run

txnelson — Fri, 14 Jan 2022 23:01:41 GMT

Try double clicking on the Summary of Fit table and changing the format to Fixed Dec and change the number of decimals to 5. Then compare the values.

Re: JMP script verification: discrepancy between results from script run and manual run

TinaJon — Wed, 19 Jan 2022 04:27:40 GMT

Thank you txnelson, for your kind input!

I circle back to my data and checked the 5-decimal R-square from manual run and that from the script run: NO, they are different even using the 5-decimal!

I am so (and even more) bewildered now, because:

1. I have multiple data sets (>20), some of them show same R-square from both the manual run and script run (which can verify that the script work well); but some of them show different R-square values from manual run and script run (the difference can be 0.967 versus 0.964); my understanding is if the script is a ROBUST script, isn't it supposed to work well for any data sets? why would there be bias (some data sets work fine for the script while other not)? Something is not right......

2. An example: manual run results (SSE: 0.16159206664151, SST: 4.4432420157201, R-square: 0.96363194575722); script run results (SSE: 0.16159206664151, SST: 4.4432420157201, R-square: 0.96660472225036). Please note that the SST and SSE are respectively identical in the manual run and script run, and because R-square=1-SSE/SST, so the R-square then is suppose to be identical, why on earth are they different? I computerized and confirmed the manual run has the correct R-square (~0.964), so does that mean the script has an issue? Should I check to see if my script is wrong?

I am so so puzzled now......

Your input is very welcome!

Re: JMP script verification: discrepancy between results from script run and manual run

txnelson — Wed, 19 Jan 2022 10:38:06 GMT

I suspect that JMP changed some calculation from a single precision to a double precision in one of the releases after JMP 9. The JMP support group is probably the only folks that have access to JMP 9 and JMP 14.2 to validate your findings, and to get to the bottom of the issue. I would contact them at

support@jmp.com

Re: JMP script verification: discrepancy between results from script run and manual run

Mark_Bailey — Wed, 19 Jan 2022 14:35:17 GMT

You have not shared your script yet. I have never seen a discrepancy before that was not understandable after a full review of the data table (data and meta-data) and the script. But I suggest that you contact JMP Technical Support, too.