How to get rid of extra decimal points during floating point calculations?

shaira — Mon, 08 Apr 2019 02:03:23 GMT

Hi,

I am making a script which takes a matrix as input, gets the interval between the elements in the matrix, and outputs that interval. My problem is that JMP output has extra decimal places sometimes (see log output below). How do I ensure that if my input has N decimal places, output should also have N decimal places?

Script:

outmtx = [0, 0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4, 1.6, 1.8, 2,
		  2.2, 2.4, 2.6, 2.8, 3, 3.2, 3.4, 3.6, 3.8, 4,
		  4.2, 4.4, 4.6, 4.8, 5, 5.2, 5.4, 5.6, 5.8, 6,
		  6.2, 6.4, 6.6, 6.8, 7, 7.2, 7.4, 7.6, 7.8, 8,
		  8.2, 8.4, 8.6, 8.8, 9, 9.2, 9.4, 9.6, 9.8, 10];

mtxdiff = J(Nrows(outmtx)-1,1,0);		  

For (i=1, i<=Nrows(outmtx)-1, i++,  mtxdiff[i] = outmtx[i+1]-outmtx[i]);

Show(mtxdiff);

Log window output:

mtxdiff = [0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2,
 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 
0.199999999999999, 0.2, 0.2, 0.2, 0.2, 0.199999999999999, 0.2, 
0.2, 0.2, 0.2, 0.199999999999999, 0.2, 0.2, 0.2, 0.2, 0.199999999999999, 
0.2, 0.2, 0.199999999999999, 0.200000000000001, 0.199999999999999,
 0.200000000000001, 0.199999999999999, 0.199999999999999, 0.200000000000001,
 0.199999999999999, 0.200000000000001, 0.199999999999999];

Thanks,

Shaira

Re: How to get rid of extra decimal points during floating point calculations?

Mark_Bailey — Mon, 08 Apr 2019 08:58:44 GMT

See the Round() function. In this case, Round( mtxdiff, 1 ) should do what you want.

topic How to get rid of extra decimal points during floating point calculations? in Discussions

How to get rid of extra decimal points during floating point calculations?

Re: How to get rid of extra decimal points during floating point calculations?