I have not noticed more out of memory messages after upgrading to JMP 10. However, I have a "feeling" of JMP 9 crashing more frequently. Maybe 10 is more stable in that sense, ie. gives an error message rather than just quit.
I see both memory leaks and sluggishness in my JMP 10.0.2, especially when I am scripting or using certain platforms repeatedly. I haven't yet been able to figure out which platforms trigger the problem. If it gets bad I close my JMP session and restart it.
I rarely use large data sets so this is not a size-of-the-data-table issue.