Community Trekker

Joined:

Mar 4, 2013

Possible to increase floating point number precision?

Hi.  Is there a way to specify the floating point precision JSL uses when performing calculations?  That is, the number of significant digits used.  For instance, on my computer 1 minus 1e-15 equals 0.999... (15 "9"s).  But if I try 1 minus 1e-16, I just get 1.

x = 1;

y = 1e-15;

z = x - y;

Show(z);      //z = 0.999999999999999 (aka 15 decimal places)

y = 1e-16;

z = x - y;

Show(z);      //z = 1

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

Super User

Joined:

Jun 22, 2012

Solution

Re: Possible to increase floating point number precision?

Here is the complete script that I ran to give you the results I posted.

x = 1;

y = 1e-15;

z = x - y;

Show(z); //z = 0.999999999999999 (aka 15 decimal places)

y = 1e-16;

z = x - y;

Show(format(z, "Fixed Dec", 30, 20));

I ran the script in 11.21.1 and got the results you reported.  So the difference is the change to version 12.2.

Jim
5 REPLIES

Super User

Joined:

Jun 22, 2012

Re: Possible to increase floating point number precision?

What you are seeing is JMP attempting to display the data in a default friendly format.  The precision behind the value does not change when specifying a format to display the value in.  So if you specify to display the value of "z" with more precision it will do that.

Show(format(z, "Fixed Dec", 30, 20));    = "0.99999999999999989000";

Jim

Community Trekker

Joined:

Mar 4, 2013

Re: Possible to increase floating point number precision?

Jim,

I'm sorry but that does not appear to work on my system.  I used the format command and the answer still came out as "1".  Obviously it works on your system, but not mine.  My company still has us using JMP 11.2.1, if that matters.  My computer is < 1 yr old though.

Question: the documentation for Format is a little confusing.

s = Format(x, formatstring, <currencycode>, <ndecimal>, < <<Use Locale(b=1)>)

Based on that, the "20" in your command occupies the currency code field, and the "30" occupies the ndecimal field.  Naturally I care about the latter.  What is the purpose of the "20"?

Community Trekker

Joined:

Mar 4, 2013

Re: Possible to increase floating point number precision?

Hi again Jim.  Also realized another important question.  In your post you indicated the result to the problem 1 minus 1e-16 was:

0.99999999999999989000

(Note the "8" in the 16th decimal place)

Super User

Joined:

Jun 22, 2012

Solution

Re: Possible to increase floating point number precision?

Here is the complete script that I ran to give you the results I posted.

x = 1;

y = 1e-15;

z = x - y;

Show(z); //z = 0.999999999999999 (aka 15 decimal places)

y = 1e-16;

z = x - y;

Show(format(z, "Fixed Dec", 30, 20));

I ran the script in 11.21.1 and got the results you reported.  So the difference is the change to version 12.2.

Jim

Staff

Joined:

Mar 21, 2013

Re: Possible to increase floating point number precision?

JMP uses the standard hardware floating point support.  IEEE floating point - Wikipedia, the free encyclopedia

There are about 15 significant digits, as you've noticed.

JMP does not have support for longer representations.  (Big Int, Big Num libraries are not in JMP at this time)

As txnelson​ points out, sometimes a default format for printing a number may not show all 15 or so possibly interesting digits.

Craige