cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
%3CLINGO-SUB%20id%3D%22lingo-sub-225496%22%20slang%3D%22en-US%22%20mode%3D%22UPDATE%22%3EConsistent%20handling%20of%20excluded%20rows%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-225496%22%20slang%3D%22en-US%22%20mode%3D%22UPDATE%22%3E%3CP%3EHi%2C%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EI%20had%20a%20conversation%20with%20JMP%20support%20since%20I%20assumed%20a%20bug%20in%20JMP.%20However%2C%20I%20learned%20the%20following%3A%3C%2FP%3E%0A%3CP%3E-%20If%20one%20marks%20certain%20rows%20as%20%22excluded%22%20(no%20matter%20if%20chosen%20visible%20or%20nor)%20they%20will%20be%20removed%20from%20many%20computations%20(e.g.%20doing%20a%20histogram%20does%20not%20take%20excluded%20rows%20into%20account)%3C%2FP%3E%0A%3CP%3E-%20If%20one%20marks%20certain%20rows%20as%20%22excluded%22%20(no%20matter%20if%20chosen%20visible%20or%20nor)%20they%20will%20be%20%3CSTRONG%3Eincluded%20%3C%2FSTRONG%3Eas%20a%20validation%20set%20for%20platforms%20like%20boosted%20tree%20or%20bootstrap%20forest.%3C%2FP%3E%0A%3CP%3EApparently%20this%20is%20intended%20behavior.%20Personally%2C%20it%20confused%20me%20and%20fortunately%20I%20got%20in%20contact%20with%20support%20to%20learn%20about%20it%20before%20I%20published%20the%20results%2C%20since%20in%20my%20case%20the%20excluded%20rows%20are%20invalid%20data.%20I%20just%20keep%20those%20for%20tracking%20purposes.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CU%3ESo%2C%20finally%20my%20wish%3A%20could%20JMP%20be%20fully%20consistent%20in%20the%20use%20of%20excluded%20rows%3F%3C%2FU%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHere's%20a%20quote%20that%20JMP%20%3CFONT%20size%3D%223%22%3Esupport%3C%2FFONT%3E%20sent%20me%3A%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3E%22The%20Bootstrap%20Forest%20and%20several%20other%20platforms%20in%20JMP%20Pro%20have%20a%20feature%20that%20if%20some%20rows%20are%20excluded%2C%20and%20you%20do%20not%20otherwise%20specify%20a%20Validation%20set%2C%20those%20rows%20are%20used%20as%20the%20Validation%20set.%20%3C%2FSPAN%3E%3C%2FFONT%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3ETo%20avoid%20those%20rows%20from%20being%20included%20at%20all%2C%20you%20could%3A%3C%2FSPAN%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3E1)%20Subset%20the%20data%20table%20so%20it%20doesn't%20include%20those%20rows%2C%20then%20re-run%20the%20Bootstrap%20Forest%3C%2FSPAN%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3E2)%20Use%20a%20different%20Validation%20method%20(Holdback%20or%20Validation%20Column)%3C%2FSPAN%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3EIn%20general%2C%20it%20is%20often%20a%20good%20idea%20to%20devote%20some%20rows%20to%20a%20Validation%20set.%20This%20would%20give%20you%20the%20ability%20to%20use%20the%20Early%20Stopping%20option%2C%20which%20can%20help%20avoid%20overfitting.%22%3C%2FSPAN%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-225496%22%20slang%3D%22en-US%22%20mode%3D%22UPDATE%22%3E%3CLINGO-LABEL%3EPredictive%20Modeling%20and%20Machine%20Learning%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-245755%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3ERe%3A%20Consistent%20handling%20of%20excluded%20rows%20-%20Status%20changed%20to%3A%20Delivered%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-245755%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CP%3EStarting%20in%20JMP%2015%20excluded%20rows%20are%20not%20used%20for%20Validation%20unless%20you%20submit%20the%20JSL%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%3EUse%20Excluded%20Rows%20for%20Validation(1)%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Choose Language Hide Translation Bar
0 Kudos

Consistent handling of excluded rows

Hi,

 

I had a conversation with JMP support since I assumed a bug in JMP. However, I learned the following:

- If one marks certain rows as "excluded" (no matter if chosen visible or nor) they will be removed from many computations (e.g. doing a histogram does not take excluded rows into account)

- If one marks certain rows as "excluded" (no matter if chosen visible or nor) they will be included as a validation set for platforms like boosted tree or bootstrap forest.

Apparently this is intended behavior. Personally, it confused me and fortunately I got in contact with support to learn about it before I published the results, since in my case the excluded rows are invalid data. I just keep those for tracking purposes.

 

So, finally my wish: could JMP be fully consistent in the use of excluded rows?

 

Here's a quote that JMP support sent me:

"The Bootstrap Forest and several other platforms in JMP Pro have a feature that if some rows are excluded, and you do not otherwise specify a Validation set, those rows are used as the Validation set. To avoid those rows from being included at all, you could:

1) Subset the data table so it doesn't include those rows, then re-run the Bootstrap Forest

2) Use a different Validation method (Holdback or Validation Column)

In general, it is often a good idea to devote some rows to a Validation set. This would give you the ability to use the Early Stopping option, which can help avoid overfitting."

 

1 Comment
Jeff_Perkinson
Community Manager
Status changed to: Delivered

Starting in JMP 15 excluded rows are not used for Validation unless you submit the JSL:

 

Use Excluded Rows for Validation(1)