cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Choose Language Hide Translation Bar
lwx228
Level VIII

How to make a number with more than 16 bits count correctly in bytes?

In JMP, when the number of bits in a numeric format is greater than 16 bits, it is displayed as a scientific count.
The result calculated with "length(char())"、 "right(char(),)" is different from the result calculated with text format of the same number of digits.
How can we make this very large number compute with these functions the same result as the text in the text forma?

 

Thanks Experts!

2021-05-17_18-31-01.png

4 REPLIES 4
Craige_Hales
Super User

Re: How to make a number with more than 16 bits count correctly in bytes?

First, read this article on 64-bit floating point. There are always 15 significant digits, sometimes 16 depending on the number.

Then, if you really want to show the noise where the least significant digits are (starting in position 16), use a format.

You can show more digits, up to around 30, I think, which is only useful for aligning columns. There is NO actual data there, it is meaningless.You can show more digits, up to around 30, I think, which is only useful for aligning columns. There is NO actual data there, it is meaningless.

Craige
lwx228
Level VIII

Re: How to make a number with more than 16 bits count correctly in bytes?

I've already set it this way, but I still don't see 1111111111111111111111 when I format it numerically.

 

Thank Craige!

2021-05-17_19-58-58.png

lwx228
Level VIII

Re: How to make a number with more than 16 bits count correctly in bytes?

If this number is in the matrix, I still have no way to deal with it.
Craige_Hales
Super User

Re: How to make a number with more than 16 bits count correctly in bytes?

You'll need to use the format when you convert the number to a string, not just in the data table's column display:

 

char(12345678901234567890,40,20)
/*:

"12345678901234567168.0000000000000000000"

 

or

 

char(12345678901234567890,30,0)
/*:

"12345678901234567168"

 

You might also want to use the format function and be explicit about the format. I'm not sure what rules char() is using above.

Craige