cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
%3CLINGO-SUB%20id%3D%22lingo-sub-213304%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3Ecross%20validation%20confusion%20matrix%20should%20include%20all%20cross%20validated%20data%20not%20just%20last%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-213304%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CP%3EPredictive%20models%20with%20binary%20Y%20that%20allow%20cross%20validation%2C%20eg%20k-fold%20or%20leave%20one%20out%2C%20the%20confusion%20matrix%20reflect%20only%20the%20last%20iteration%20of%20the%20cross%20classification.%20For%20example%20in%205-fold%20cross%20validation%20the%20training%20set%20has%2080%25%20of%20the%20data%20and%20the%20Validation%20set%20shows%2020%25.%26nbsp%3B%20The%20validation%20set%20should%20combine%20all%205%20folds%20of%20the%20data%20to%20show%20100%25.%26nbsp%3B%20For%20leave%20one%20out%20it%20is%20even%20more%20useless%20as%20the%20validation%20set%20only%20has%20a%20single%20observation.%26nbsp%3B%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-213304%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CLINGO-LABEL%3EStatistics%20and%20Analytics%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Choose Language Hide Translation Bar

cross validation confusion matrix should include all cross validated data not just last

Predictive models with binary Y that allow cross validation, eg k-fold or leave one out, the confusion matrix reflect only the last iteration of the cross classification. For example in 5-fold cross validation the training set has 80% of the data and the Validation set shows 20%.  The validation set should combine all 5 folds of the data to show 100%.  For leave one out it is even more useless as the validation set only has a single observation.  

1 Comment
Ryan_Gilmore
Community Manager
Status changed to: Archived
We are archiving this request. If this is still important please comment with additional details and we will reopen. Thank you!