cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Choose Language Hide Translation Bar
FN
FN
Level VI

k-fold cross validation in time series with JMP / JMP PRO

I want to use Bootstrap or Boosted Trees but with a k-fold method as I use it in the partition trees or neural nets. However I only see the option of adding a fix validation column.
 
Since I am working with time series, random samples cannot be used well to fit my models. 
 
In addition to this, I am suspicious with the k-fold implemented in JMP as it always says random sample.
 
image.png
 
I am after a sequence like this one (at least):
 
image.png
And I think JMP does something like this (please confirm):
 
image.png
 
Here are all the options, the best for my case would be TimeSplit.
 
Any ideas or workarounds?
 
3 REPLIES 3
FN
FN
Level VI

Re: k-fold cross validation in time series with JMP / JMP PRO

Confirmed: In JMP 14, K-fold it is actually a random sampling done k times. 

 

k fold Neural Network JMP.png

FN
FN
Level VI

Re: k-fold cross validation in time series with JMP / JMP PRO

And if the cutpointvalidation method is used as a column to fit trees (recommended for time series in JMP):

 

cut point tree jmp 14.png

 

FN
FN
Level VI

Re: k-fold cross validation in time series with JMP / JMP PRO

When using Neural Networks and the validation column, the results are a simple extrapolation. Test rows are not used in this interface either.

 

Ideas on how to implement a proper k-fold for either trees or NN?JMP NN.png