topic How to make a number with more than 16 bits count correctly in bytes? in Discussions

How to make a number with more than 16 bits count correctly in bytes?

lwx228 — Fri, 09 Jun 2023 19:47:02 GMT

In JMP, when the number of bits in a numeric format is greater than 16 bits, it is displayed as a scientific count.
The result calculated with "length(char())"、 "right(char(),)" is different from the result calculated with text format of the same number of digits.
How can we make this very large number compute with these functions the same result as the text in the text forma？

Thanks Experts!

Re: How to make a number with more than 16 bits count correctly in bytes?

Craige_Hales — Mon, 17 May 2021 11:38:27 GMT

First, read this article on 64-bit floating point. There are always 15 significant digits, sometimes 16 depending on the number.

Then, if you really want to show the noise where the least significant digits are (starting in position 16), use a format.

You can show more digits, up to around 30, I think, which is only useful for aligning columns. There is NO actual data there, it is meaningless.

Re: How to make a number with more than 16 bits count correctly in bytes?

lwx228 — Mon, 17 May 2021 12:02:40 GMT

I've already set it this way, but I still don't see 1111111111111111111111 when I format it numerically.

Thank Craige!

Re: How to make a number with more than 16 bits count correctly in bytes?

lwx228 — Mon, 17 May 2021 12:04:32 GMT

If this number is in the matrix, I still have no way to deal with it.

Re: How to make a number with more than 16 bits count correctly in bytes?

Craige_Hales — Mon, 17 May 2021 12:20:54 GMT

You'll need to use the format when you convert the number to a string, not just in the data table's column display:

char(12345678901234567890,40,20)
/*:

"12345678901234567168.0000000000000000000"

char(12345678901234567890,30,0)
/*:

"12345678901234567168"

You might also want to use the format function and be explicit about the format. I'm not sure what rules char() is using above.