cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Check out the JMP® Marketplace featured Capability Explorer add-in
Choose Language Hide Translation Bar
nikles
Level VI

Possible to increase floating point number precision?

Hi.  Is there a way to specify the floating point precision JSL uses when performing calculations?  That is, the number of significant digits used.  For instance, on my computer 1 minus 1e-15 equals 0.999... (15 "9"s).  But if I try 1 minus 1e-16, I just get 1.    

x = 1;

y = 1e-15;

z = x - y;

Show(z);      //z = 0.999999999999999 (aka 15 decimal places)

y = 1e-16;

z = x - y;

Show(z);      //z = 1

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions
txnelson
Super User

Re: Possible to increase floating point number precision?

Here is the complete script that I ran to give you the results I posted.

x = 1;

y = 1e-15;

z = x - y;

Show(z); //z = 0.999999999999999 (aka 15 decimal places)

y = 1e-16;

z = x - y;

Show(format(z, "Fixed Dec", 30, 20));

I ran the script in 11.21.1 and got the results you reported.  So the difference is the change to version 12.2.  

Jim

View solution in original post

5 REPLIES 5
txnelson
Super User

Re: Possible to increase floating point number precision?

What you are seeing is JMP attempting to display the data in a default friendly format.  The precision behind the value does not change when specifying a format to display the value in.  So if you specify to display the value of "z" with more precision it will do that.

Show(format(z, "Fixed Dec", 30, 20));    = "0.99999999999999989000";

Jim
nikles
Level VI

Re: Possible to increase floating point number precision?

Jim,

I'm sorry but that does not appear to work on my system.  I used the format command and the answer still came out as "1".  Obviously it works on your system, but not mine.  My company still has us using JMP 11.2.1, if that matters.  My computer is < 1 yr old though.

Question: the documentation for Format is a little confusing. 

s = Format(x, formatstring, <currencycode>, <ndecimal>, < <<Use Locale(b=1)>)

Based on that, the "20" in your command occupies the currency code field, and the "30" occupies the ndecimal field.  Naturally I care about the latter.  What is the purpose of the "20"?

nikles
Level VI

Re: Possible to increase floating point number precision?

Hi again Jim.  Also realized another important question.  In your post you indicated the result to the problem 1 minus 1e-16 was:

0.99999999999999989000

(Note the "8" in the 16th decimal place)


Was this a typo?

txnelson
Super User

Re: Possible to increase floating point number precision?

Here is the complete script that I ran to give you the results I posted.

x = 1;

y = 1e-15;

z = x - y;

Show(z); //z = 0.999999999999999 (aka 15 decimal places)

y = 1e-16;

z = x - y;

Show(format(z, "Fixed Dec", 30, 20));

I ran the script in 11.21.1 and got the results you reported.  So the difference is the change to version 12.2.  

Jim
Craige_Hales
Super User

Re: Possible to increase floating point number precision?

JMP uses the standard hardware floating point support.  IEEE floating point - Wikipedia, the free encyclopedia

There are about 15 significant digits, as you've noticed.

JMP does not have support for longer representations.  (Big Int, Big Num libraries are not in JMP at this time)

As txnelson​ points out, sometimes a default format for printing a number may not show all 15 or so possibly interesting digits.

Craige