cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
%3CLINGO-SUB%20id%3D%22lingo-sub-225496%22%20slang%3D%22en-US%22%20mode%3D%22UPDATE%22%3EGestion%20coh%C3%A9rente%20des%20lignes%20exclues%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-225496%22%20slang%3D%22en-US%22%20mode%3D%22UPDATE%22%3E%3CP%3ESalut%2C%3C%2FP%3E%0A%3CP%3E%20%3C%2FP%3E%0A%3CP%3EJ'ai%20eu%20une%20conversation%20avec%20le%20support%20JMP%20car%20j'ai%20suppos%C3%A9%20un%20bug%20dans%20JMP.%20Cependant%2C%20j'ai%20appris%20ce%20qui%20suit%20%3A%3C%2FP%3E%0A%3CP%3E-%20Si%20l'on%20marque%20certaines%20lignes%20comme%20%22exclues%22%20(peu%20importe%20si%20elles%20sont%20choisies%20visibles%20ou%20ni)%20elles%20seront%20supprim%C3%A9es%20de%20nombreux%20calculs%20(par%20exemple%20faire%20un%20histogramme%20ne%20prend%20pas%20en%20compte%20les%20lignes%20exclues)%3C%2FP%3E%0A%3CP%3E-%20Si%20l'on%20marque%20certaines%20lignes%20comme%20%22exclues%22%20(peu%20importe%20si%20elles%20sont%20choisies%20visibles%20ou%20ni)%2C%20elles%20seront%20%3CSTRONG%3Eincluses%3C%2FSTRONG%3E%20comme%20ensemble%20de%20validation%20pour%20les%20plates-formes%20comme%20l'arbre%20boost%C3%A9%20ou%20la%20for%C3%AAt%20d'amor%C3%A7age.%3C%2FP%3E%0A%3CP%3EApparemment%2C%20c'est%20un%20comportement%20pr%C3%A9vu.%20Personnellement%2C%20cela%20m'a%20d%C3%A9rout%C3%A9%20et%20heureusement%20j'ai%20contact%C3%A9%20le%20support%20pour%20en%20savoir%20plus%20avant%20de%20publier%20les%20r%C3%A9sultats%2C%20car%20dans%20mon%20cas%20les%20lignes%20exclues%20sont%20des%20donn%C3%A9es%20invalides.%20Je%20les%20garde%20juste%20%C3%A0%20des%20fins%20de%20suivi.%3C%2FP%3E%0A%3CP%3E%20%3C%2FP%3E%0A%3CP%3E%3CU%3EAlors%2C%20enfin%20mon%20souhait%26nbsp%3B%3A%20JMP%20pourrait-il%20%C3%AAtre%20totalement%20coh%C3%A9rent%20dans%20l%E2%80%99utilisation%20des%20lignes%20exclues%26nbsp%3B%3F%3C%2FU%3E%3C%2FP%3E%0A%3CP%3E%20%3C%2FP%3E%0A%3CP%3EVoici%20un%20devis%20que%20%3CFONT%20size%3D%223%22%3Ele%20support%3C%2FFONT%3E%20JMP%20m'a%20envoy%C3%A9%26nbsp%3B%3A%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3E%22La%20for%C3%AAt%20Bootstrap%20et%20plusieurs%20autres%20plates-formes%20de%20JMP%20Pro%20ont%20une%20fonctionnalit%C3%A9%20selon%20laquelle%20si%20certaines%20lignes%20sont%20exclues%20et%20que%20vous%20ne%20sp%C3%A9cifiez%20pas%20d'ensemble%20de%20validation%2C%20ces%20lignes%20sont%20utilis%C3%A9es%20comme%20ensemble%20de%20validation.%3C%2FSPAN%3E%3C%2FFONT%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3EPour%20%C3%A9viter%20que%20ces%20lignes%20ne%20soient%20incluses%2C%20vous%20pouvez%26nbsp%3B%3A%3C%2FSPAN%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3E1)%20Sous-d%C3%A9finissez%20la%20table%20de%20donn%C3%A9es%20afin%20qu'elle%20n'inclue%20pas%20ces%20lignes%2C%20puis%20r%C3%A9ex%C3%A9cutez%20la%20for%C3%AAt%20Bootstrap.%3C%2FSPAN%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3E2)%20Utilisez%20une%20m%C3%A9thode%20de%20validation%20diff%C3%A9rente%20(colonne%20de%20retenue%20ou%20de%20validation)%3C%2FSPAN%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%3CFONT%20size%3D%222%22%3E%3CSPAN%20class%3D%22cs78ab32121%22%3EEn%20g%C3%A9n%C3%A9ral%2C%20c'est%20souvent%20une%20bonne%20id%C3%A9e%20de%20consacrer%20quelques%20lignes%20%C3%A0%20un%20ensemble%20de%20validation.%20Cela%20vous%20donnerait%20la%20possibilit%C3%A9%20d%E2%80%99utiliser%20l%E2%80%99option%20Early%20Stopping%2C%20ce%20qui%20peut%20aider%20%C3%A0%20%C3%A9viter%20le%20surajustement.%20%C2%BB%3C%2FSPAN%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%20class%3D%22cs95e872d0%22%3E%20%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-225496%22%20slang%3D%22en-US%22%20mode%3D%22UPDATE%22%3E%3CLINGO-LABEL%3EMod%C3%A9lisation%20pr%C3%A9dictive%20et%20apprentissage%20automatique%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-245755%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3EObjet%26nbsp%3B%3A%20Traitement%20coh%C3%A9rent%20des%20lignes%20exclues%20-%20Statut%20modifi%C3%A9%20en%26nbsp%3B%3A%20Livr%C3%A9%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-245755%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CP%3E%C3%80%20partir%20de%20JMP%2015%2C%20les%20lignes%20exclues%20ne%20sont%20pas%20utilis%C3%A9es%20pour%20la%20validation%2C%20sauf%20si%20vous%20soumettez%20le%20JSL%26nbsp%3B%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%3EUse%20Excluded%20Rows%20for%20Validation(1)%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Choose Language Hide Translation Bar
0 Kudos

Consistent handling of excluded rows

Hi,

 

I had a conversation with JMP support since I assumed a bug in JMP. However, I learned the following:

- If one marks certain rows as "excluded" (no matter if chosen visible or nor) they will be removed from many computations (e.g. doing a histogram does not take excluded rows into account)

- If one marks certain rows as "excluded" (no matter if chosen visible or nor) they will be included as a validation set for platforms like boosted tree or bootstrap forest.

Apparently this is intended behavior. Personally, it confused me and fortunately I got in contact with support to learn about it before I published the results, since in my case the excluded rows are invalid data. I just keep those for tracking purposes.

 

So, finally my wish: could JMP be fully consistent in the use of excluded rows?

 

Here's a quote that JMP support sent me:

"The Bootstrap Forest and several other platforms in JMP Pro have a feature that if some rows are excluded, and you do not otherwise specify a Validation set, those rows are used as the Validation set. To avoid those rows from being included at all, you could:

1) Subset the data table so it doesn't include those rows, then re-run the Bootstrap Forest

2) Use a different Validation method (Holdback or Validation Column)

In general, it is often a good idea to devote some rows to a Validation set. This would give you the ability to use the Early Stopping option, which can help avoid overfitting."

 

1 Comment
Jeff_Perkinson
Community Manager
Status changed to: Delivered

Starting in JMP 15 excluded rows are not used for Validation unless you submit the JSL:

 

Use Excluded Rows for Validation(1)