Outlier (👽) screening with Jmp Dashboards - how?

[#1]

I want to do some Outlier screening by listing the aliens in my class.

First attempt:  Local Data Filter
I was quite shocked with all the aliens in Big Class!
... but then I noticed that the data filter was just ignored (because on day 1 there were no aliens in the class - and with no matching rows, the Data Filter is just reset - instead of showing 0 matching rows !!!).

This seems to be related to Data filter: how to specify the limits :

If a value is just used in the data filter but not in the data table, it is removed from the data filer.
This behavior interferes with another behavior of Jmp: if the last filter value is deselected, all rows get selected (instead of none).

So, with the current behavior of Data Filters, this approach doesn't work

Next, I used a Where() clause to restrict the Tabulate Report.

Unfortunately, if one day there are no aliens in the class, the Where clause produces an error message - instead of restricting the data set to 0 rows:

This Where-clause for data table 'Big Class' resulted in no rows: :alien == 1

List of "aliens" for the 4 cases:

Please note:
For the case with Where(), there is another issue:
The row selection for the report is static.

After XX changed to non-alien - he is still listed in the report:

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
dt << New Column( "alien", nominal, set each value( 0 ) );


myDashboard = Function( {myTitle, withWhereClause},
	New Window( myTitle,
		<<Show Menu( 0 ),
		<<ShowToolbars( 0 ),
		Tab Box(
			Tab Page Box(
				"aliens", 
				Eval(
					Substitute(
							Expr(
								Tabulate(
									Show Control Panel( 0 ),
									Add Table( Row Table( Grouping Columns( :sex, :age, :name ) ) ),
									__selectAliens__
								)
							),
						Expr( __selectAliens__ ),
							If( withWhereClause,
								Expr( Where( :alien == 1 ) ),
								Expr(
									Local Data Filter( Close Outline( 1 ), Add Filter( columns( :alien ), Where( :alien == 1 ) ) )
								)
							)
					)
				)
			),
			Tab Page Box(
				"all Students",
				Tabulate( Show Control Panel( 0 ), Add Table( Row Table( Grouping Columns( :sex, :age, :name ) ) ) )
			)
		)	
	)
);

// 1) Data Filter --> so many aliens ?!?! no, Data filter was ignored (filter is OK, see below)
myDashboard( "1) Data Filter, no alien", 0 ); 

// 2) Where Cause  + no matching row --> error message instead of a report
myDashboard( "2) Where(), no alien", 1 ); 

dt << add rows( {name = "XX", alien = 1, sex = "M", age = 200} );

// 3) Data Filter + 1 alien
myDashboard( "3) Data filter, 1 alien", 0 ); // new test: --> filter OK --> works

myDashboard( "4) Where(), 1->0 aliens ...", 1 ); // Where Clause  + matching row --> dashboard is created, but ...


// XX gets human -> Dashboard  #4  stays the same :(
//dt[41, {"alien"}] = 0;
;

[#2]

@danschikore 
This video gives a hint what is going wrong:

The Dashboard starts with a wrong list in Tabulate - The Data Filter is restricted to "aliens", but in the list there are many entries with alien=missing.
To remove the wrong entries, the user has to change the selection in the "inner" data filter. (fist part of the video). (*)

On the other hand (second part), concerning the Data Filter in the Data filter Context Box - as soon as the user changes the selection there, the content of Tabulate is reset and the settings of the "inner" data filter are ignored

... till the user corrects it again via (*)

Support Ticket: TS-00046650

[#3]

Thanks @hogi - this definitely looks like a bug that we will fix for a future release.

[#4]

thanks

So, after all, I came back to @jthi 's first suggestion:

---

@jthi wrote:

• Add "blank" row with :alien = 1 (all other values missing except for the alien being 1)

Just a single blank line with alien=1, NO for loop, NO additional dummy column etc .

Then the dummy alien shows up by the additional Data filter entries sex = missing, name=missing, age=missing

• This makes it easy to remove the dummy alien from the reports by selecting an entry (!= missing) in the respective Data Filter.
• The benefit compared to the approach with the dummy column:
  In the case of  "no aliens today", the Outlier Screening will NOT be reset

Names Default to Here(1);
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
dt << New Column( "alien", Nominal );

dt << add rows( {:alien = 1} );

addAliens=0;

if(addAliens,
:alien[12]=1;
:alien[39]=1;
);

New Window( "Outlier Screening",
	Data Filter Context Box(
		H List Box(
			dt << Data Filter(
		Local,
				Add Filter( columns( :age, :sex ), Where( :age == 13 ) )

			), 

			dt << Tabulate(
				Show Control Panel( 0 ),
				Include missing for grouping columns( 1 ),
				Add Table(Row Table( Grouping Columns( :name, :sex ) ) ),
				Local Data Filter(
					Add Filter( columns( :alien ),  Where( :alien == 1 ) )
				)
			)
		)
	)
);

[#5]

Hi @danschikore ,

---

@danschikore wrote:

Thanks @hogi - this definitely looks like a bug that we will fix for a future release.

---

does this comment refer to the special case with the two data filters or to the general case:

A Data Filter forgets its settings, if the value is not in the current data table?

[#6]

Hi @hogi - I was referring to the first issue that you show in the video, that Tabulate was not showing the correct subset when the outer filter changes.  But I don't see your script for this provided here - did you send it through Tech Support?

-Dan

[#7]

The video was based on the code 2 posts before, with the same issue.
In Jmp18 it looks much better. Seems that this issue could be fixed .

On the other hand, there is still the issue with data filters which 'forget' their settings:

https://community.jmp.com/t5/Discussions/Can-animation-of-this-difficulty-be-achieved-with-JSL/m-p/7...

[#8]

With Jmp 17, Data Filter was introduced for Tables/Summary. 

Here the issue is much more severe: at the end, there is just the wrong result - no Data Filter on the left of a Dashboard showing the "obviously" wrong setting. 2 bugs with Tables/Summary