cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Check out the JMP® Marketplace featured Capability Explorer add-in
Choose Language Hide Translation Bar
jbovingt
Level I

What are some reasons JMP fails to see an equivalence (==)? Is there an approximately equal function ~== to manage rounding errors?

I'm working on a script to check some parametric data generated by a test program. The table I have is for some laser spectra which have a single SpectralPeak and Left and/or Right Side Peaks. I would like to visualize the data in JMP, so I created this plot:

 

jbovingt_1-1724673785199.png

I'm able to highlight the Peak from the spectra with a column which uses a simple formula to populate the rows only at the SpectralPeakPower (RED).

If( :YValues == Col Max(:SpectralPeakPower, :LampWfr, :serial, :"LaserCurrent[mA]"n), :SpectralPeakPower)

 

However, when I use this same concept to calculate the power of the Left side peak, the equivalence statement fails, and therefore the left side peak power is not populated into the row and NOT highlighted in the plot.

 

jbovingt_2-1724674669515.png

 

In debugging this issue, there doesn't appear to be a logical reason that the == statement doesn't become 1...
(-46.81 == -46.81) = 1 right!?!?! I'm going a little crazy debugging this...

What are the other possible reasons that == isn't 1? 

I also have other instances in my script where there is a rounding error and I would like to use an ~== approximately equivalent function. Does such a function already exist?

 

I can share the JMP file privately to staff members for review.
Thanks,

 

 

 

 

 

1 REPLY 1
jthi
Super User

Re: What are some reasons JMP fails to see an equivalence (==)? Is there an approximately equal function ~== to manage rounding errors?

Could be due to Floating Point Math .

Show(0.1 + 0.2, 0.1 + 0.2 == 0.3);
//0.1 + 0.2 = 0.3;
//0.1 + 0.2 == 0.3 = 0;

You can use Round(), Floor() or Ceiling() to round your values.

-Jarmo