cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
JMP is taking Discovery online, April 16 and 18. Register today and join us for interactive sessions featuring popular presentation topics, networking, and discussions with the experts.
Choose Language Hide Translation Bar
ravi_sasjmp
Level II

Limitation Of HexToNumber()

I'm using JMP10 and notice for Hex To Number( hextext), hextext is limited to 31-bits. Is there a way around this?

Hex To Number("7FFFFFFF") = 2147483647, Hex To Number("80000000") = -2147483648. 

I need to multiply two hex strings (each can be upto 4-bytes) and in the resulting number, depending on state of MSB bit, extract Bits[29:14] or Bits [27:12]. Can this be done in JSL?

1 ACCEPTED SOLUTION

Accepted Solutions
Craige_Hales
Super User

Re: Limitation Of HexToNumber()

JMP uses 64 bit floating point to represent the JSL numbers; the number of fraction bits available is only 52 or 53. If you multiply two 32 bit integers, you might need 64 bits to represent the answer. If the answer is greater than 52 or 53 bits, you'll lose precision on the low order bits...maybe as many as 11 or 12 bits can be lost.

Here's some JSL that might help.

 

// crude unsigned hex multiply, poorly tested, no idea what negative numbers are
// do your own testing to make sure this meets your needs
// expects upper case hex ABCDEF, not abcdef
hextonum = [
"0"=>0,"1"=>1,"2"=>2,"3"=>3,
"4"=>4,"5"=>5,"6"=>6,"7"=>7,
"8"=>8,"9"=>9,"A"=>10,"B"=>11,
"C"=>12,"D"=>13,"E"=>14,"F"=>15	
];
numtohex = Associative Array( hextonum << getvalues, hextonum << getkeys );

hexmultiply = Function( {a, b}, // "123AE" and "EF012" for example
    {// locals
    ia,  na=length(a),amat = J( na, 1, 0 ), ib,nb=length(b), bmat = J( nb, 1, 0 ),ir,nr=na+nb,result = J( nr+1, 1, 0 ),r="", carry,remainder},
    a = Reverse( a );
    For( ia = 1, ia <= na, ia++,
        amat[ia] = hextonum[Substr( a, ia, 1 )]
    );
    b = Reverse( b );
    For( ib = 1, ib <= nb, ib++,
        bmat[ib] = hextonum[Substr( b, ib, 1 )]
    );
    for(ia=1,ia<=na,ia+=1,
    	for(ib=1,ib<=nb,ib+=1, // this inner loop might speed up by replacing it with matrix math
            result[ia+ib-1] += amat[ia]*bmat[ib];
    	)
    );
    for(ir=1,ir<=nr,ir++, // do the carry. doing it last is fast, but places some sort of a limit on the total length
        remainder = mod(result[ir],16); 
        carry = floor(result[ir]/16); 
        result[ir] = remainder;
        result[ir+1] += carry;
    	r =  numtohex[result[ir]] || r; // build hex result
    );
    r // return value
);

// example hexmultiply
show( hextonumber(hexmultiply("AE2","3B75")));
show( hextonumber("AE2")*hextonumber("3B75"));

// test code...
ntests=0;
start = tickseconds();
For( num1 = 0, num1 < 2e9, num1 = Floor( num1 * 1.07 + 1 ),
    For( num2 = 0, num2 < 2e9, num2 = Floor( num2 * 1.05 + 1 ),
        If( num1 * num2 > 2 ^ 53,
            Continue(); // because double multiply gets wrong answer
        );
        ntests+=1;
        t1 = hexmultiply( Hex( num1, "integer" ), Hex( num2, "integer" ) );
        If( Hex To Number( Substr( t1, 2 ) ) != num1 * num2,
            Show( num1, num2, t1, Hex To Number( Substr( t1, 2 ) ), num1 * num2 )
        );
    )
);
stop=tickseconds();
show(ntests,stop-start,ntests/(stop-start));

// long test...no promises here...this is not a meaningful test...
show( hexmultiply("00123456789ABCDEF0","00200000000000000000000000000000000000"));

 

Craige

View solution in original post

2 REPLIES 2
Craige_Hales
Super User

Re: Limitation Of HexToNumber()

JMP uses 64 bit floating point to represent the JSL numbers; the number of fraction bits available is only 52 or 53. If you multiply two 32 bit integers, you might need 64 bits to represent the answer. If the answer is greater than 52 or 53 bits, you'll lose precision on the low order bits...maybe as many as 11 or 12 bits can be lost.

Here's some JSL that might help.

 

// crude unsigned hex multiply, poorly tested, no idea what negative numbers are
// do your own testing to make sure this meets your needs
// expects upper case hex ABCDEF, not abcdef
hextonum = [
"0"=>0,"1"=>1,"2"=>2,"3"=>3,
"4"=>4,"5"=>5,"6"=>6,"7"=>7,
"8"=>8,"9"=>9,"A"=>10,"B"=>11,
"C"=>12,"D"=>13,"E"=>14,"F"=>15	
];
numtohex = Associative Array( hextonum << getvalues, hextonum << getkeys );

hexmultiply = Function( {a, b}, // "123AE" and "EF012" for example
    {// locals
    ia,  na=length(a),amat = J( na, 1, 0 ), ib,nb=length(b), bmat = J( nb, 1, 0 ),ir,nr=na+nb,result = J( nr+1, 1, 0 ),r="", carry,remainder},
    a = Reverse( a );
    For( ia = 1, ia <= na, ia++,
        amat[ia] = hextonum[Substr( a, ia, 1 )]
    );
    b = Reverse( b );
    For( ib = 1, ib <= nb, ib++,
        bmat[ib] = hextonum[Substr( b, ib, 1 )]
    );
    for(ia=1,ia<=na,ia+=1,
    	for(ib=1,ib<=nb,ib+=1, // this inner loop might speed up by replacing it with matrix math
            result[ia+ib-1] += amat[ia]*bmat[ib];
    	)
    );
    for(ir=1,ir<=nr,ir++, // do the carry. doing it last is fast, but places some sort of a limit on the total length
        remainder = mod(result[ir],16); 
        carry = floor(result[ir]/16); 
        result[ir] = remainder;
        result[ir+1] += carry;
    	r =  numtohex[result[ir]] || r; // build hex result
    );
    r // return value
);

// example hexmultiply
show( hextonumber(hexmultiply("AE2","3B75")));
show( hextonumber("AE2")*hextonumber("3B75"));

// test code...
ntests=0;
start = tickseconds();
For( num1 = 0, num1 < 2e9, num1 = Floor( num1 * 1.07 + 1 ),
    For( num2 = 0, num2 < 2e9, num2 = Floor( num2 * 1.05 + 1 ),
        If( num1 * num2 > 2 ^ 53,
            Continue(); // because double multiply gets wrong answer
        );
        ntests+=1;
        t1 = hexmultiply( Hex( num1, "integer" ), Hex( num2, "integer" ) );
        If( Hex To Number( Substr( t1, 2 ) ) != num1 * num2,
            Show( num1, num2, t1, Hex To Number( Substr( t1, 2 ) ), num1 * num2 )
        );
    )
);
stop=tickseconds();
show(ntests,stop-start,ntests/(stop-start));

// long test...no promises here...this is not a meaningful test...
show( hexmultiply("00123456789ABCDEF0","00200000000000000000000000000000000000"));

 

Craige
Craige_Hales
Super User

Re: Limitation Of HexToNumber()

JMP 10 has some limitations; it looks like the strings of hex digits in JMP 10 need to be an even number of hex characters. The function above seems to work in JMP 10 (do your own testing!), but the test harness is using features from later releases.

Craige