LDA cross-validation

[#1]

Hello,

I'm a new user of JMP (JMP 13.1.0). I have a question please: Is it possible to perform an linear discriminant analysis with cross-validation (leave-one-out method) in this version of the software?

Thank you,

Adias 

[#2]

Leave-one-out cross validation is not available in LDA, but it is in JMP Pro's GenReg. If your outcome is binary and you have JMP Pro, consider running a logistic regression from GenReg, which is considered as an alternative approach to LDA and is often preferred. http://www.jstor.org/stable/2286261?seq=1#page_scan_tab_contents

[#3]

Hello,

You will need JMP Pro 13 to run Discriminant Analysis and Leave-one-out cross-validation is an option there as well.  You would run this through the Fit Model platform.

HTH 

[#4]

Hello:  There are other great features in the discriminant platform like regularized discriminant analysis, canonical plots, interpretation of dimensions, etc.  If I want to stay in the discriminant platform and perform leave-one-out validation what are the choices?

[#5 · Accepted solution]

Within the Discriminant platform there is not a way to get leave-one-out validation. Within the Discriminant platform in JMP 14, there is an option "Cross Validate by Excluded Rows". In JMP Pro, you can create a Validation column with no more than 3 values to be used for Training, Validation, and Test sets.

[#6]

Dear all,

I'm trying to validate my LDA analysis, but I can't seem to find the "Cross Validate by Excluded Rows" option in my JMP15 (not PRO). Could you please help me to navigate and find that function?

I have already prepared a Validation Column and Excluded appropriate rows.

Thanks

[#7]

The Discriminant platform in JMP Pro provides the Validation role to which you can assign the validation data column that you created.

(Please do not start another discussion when you do not get an answer right away.)

[#8]

Dear Mark,

Thank you so much for responding so quickly and sorry for a naive question - but where do I find this 'Validation role' in the LDA platform? I'm quite new to JMP, and not everything is clear to me.

The only thing I found is the option on the main tab where I select variables for LDA. There I can tick the box 'Validate by excluded rows' which selects Train/Test data based on the validation column I've created. But this offers only a one-off split based on this column, not any iterative validation options (like leave-one-out or k-folds or Monte Carlo I'd like to run). But again, maybe I'm missing something very obvious here?

I didn't want to duplicate the posts, sorry about that! I wasn't sure if anyone will answer to my response to a very old post!

[#9]

Select Analyze > Multivariate Methods > Discriminant. You will see a button to assign the column to the Validate role.

[#10]

Thank you! I don't know how I missed that!

But, apart from being easier to use, doesn't it have the same effect as the function "Validate by excluding rows" mentioned earlier? Is that right?

Maybe I'm doing something wrong, but it looks like I can use only one validation column at the time and 'Validation role' doesn't give me any option to select the validation method or generate multiple validation columns without repetition in the test set (i.e., so that the variable selected as Test in the first column will not be selected anymore in the following validation columns). Is it possible to do that in JMP?

Once again, sorry for posting such naive questions, but I got totally lost in this!