cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
JMP is taking Discovery online, April 16 and 18. Register today and join us for interactive sessions featuring popular presentation topics, networking, and discussions with the experts.
Choose Language Hide Translation Bar
FN
FN
Level VI

k-fold cross validation in time series with JMP / JMP PRO

I want to use Bootstrap or Boosted Trees but with a k-fold method as I use it in the partition trees or neural nets. However I only see the option of adding a fix validation column.
 
Since I am working with time series, random samples cannot be used well to fit my models. 
 
In addition to this, I am suspicious with the k-fold implemented in JMP as it always says random sample.
 
image.png
 
I am after a sequence like this one (at least):
 
image.png
And I think JMP does something like this (please confirm):
 
image.png
 
Here are all the options, the best for my case would be TimeSplit.
 
Any ideas or workarounds?
 
3 REPLIES 3
FN
FN
Level VI

Re: k-fold cross validation in time series with JMP / JMP PRO

Confirmed: In JMP 14, K-fold it is actually a random sampling done k times. 

 

k fold Neural Network JMP.png

FN
FN
Level VI

Re: k-fold cross validation in time series with JMP / JMP PRO

And if the cutpointvalidation method is used as a column to fit trees (recommended for time series in JMP):

 

cut point tree jmp 14.png

 

FN
FN
Level VI

Re: k-fold cross validation in time series with JMP / JMP PRO

When using Neural Networks and the validation column, the results are a simple extrapolation. Test rows are not used in this interface either.

 

Ideas on how to implement a proper k-fold for either trees or NN?JMP NN.png