BookmarkSubscribeRSS Feed

cross validation confusion matrix should include all cross validated data not just last

Predictive models with binary Y that allow cross validation, eg k-fold or leave one out, the confusion matrix reflect only the last iteration of the cross classification. For example in 5-fold cross validation the training set has 80% of the data and the Validation set shows 20%.  The validation set should combine all 5 folds of the data to show 100%.  For leave one out it is even more useless as the validation set only has a single observation.  

Tracking Number:

Defect ID: