cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Try the Materials Informatics Toolkit, which is designed to easily handle SMILES data. This and other helpful add-ins are available in the JMP® Marketplace
Choose Language Hide Translation Bar
MWalther
Level II

Fast way to check for "wrong sensordata"

I have many different sensors (>>100) with many Data (continuous) each (>1mio) and update these each week.

Sometimes these sensors "glitch" and send wrong data (below/above a low/high threshold)

I want to calculate the number of "glitches" per week (KW=categorial) and need a fast way by JSL code.

Example: negative data for this sensor should be detected (low threshold 0)

 

MWalther_0-1618900705046.png

My Solutions are too slow:
1.) I set the threshold limits as "specification" in column properties and then perform process capability
"Observed outside" Spec per KW is the desired value, but it is quite slow

2.) Brute force: each row, each column, manual check, adding each cell (Excel style) takes forever.

 

Is there a more efficient way to do this?

 

5 REPLIES 5
txnelson
Super User

Re: Fast way to check for "wrong sensordata"

I suggest that you use the Range Check column property.  It is quite efficient.

Jim
MWalther
Level II

Re: Fast way to check for "wrong sensordata"

Thanks, sounds feasible.
So the way to go is use range check to set all data to missing, which do not comply, and the do a tabulate to "count" the missing values per calenderweek.

The only drawback ist, that all "wrong" data then are lost in the original data table.
I will need to use a copy, or is there a way to retrieve the (then) missing data in the original table?

Yours, Marten

txnelson
Super User

Re: Fast way to check for "wrong sensordata"

You will need to create a complete Subset
Tables=>Subset
Jim
P_Bartell
Level VIII

Re: Fast way to check for "wrong sensordata"

In addition to @txnelson 's suggestion, have you tried out the Explore Outliers utility as a means to finding the rows containing the suspicious observations? Here's a link to the utility in JMP documentation: Explore Outliers Utility 

ih
Super User (Alumni) ih
Super User (Alumni)

Re: Fast way to check for "wrong sensordata"

Yet another idea:  If you know the bad data is always a negative number, or below some specific value, you could make a new column which is 1 if the values is 'bad' and then use tabulate to show the number of bad values for each level of KW.