cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
  • Instantly extract effect sizes, F-ratios, and FDR-adjusted p-values from your models with the Calculate Effects Sizes extension, available now in the JMP Marketplace!
  • New to JMP? Join us Sept. 23-24 for the Early User Edition of Discovery Summit, tailor-made for new users. Register now for free!
  • Your voice matters! Tell us how you prefer to receive JMP updates, so we can tailor our communication to your needs. Take short survey.

Discussions

Solve problems, and share tips and tricks with other JMP users.
Choose Language Hide Translation Bar
FN
FN
Level VI

k-fold cross validation in time series with JMP / JMP PRO

I want to use Bootstrap or Boosted Trees but with a k-fold method as I use it in the partition trees or neural nets. However I only see the option of adding a fix validation column.
 
Since I am working with time series, random samples cannot be used well to fit my models. 
 
In addition to this, I am suspicious with the k-fold implemented in JMP as it always says random sample.
 
image.png
 
I am after a sequence like this one (at least):
 
image.png
And I think JMP does something like this (please confirm):
 
image.png
 
Here are all the options, the best for my case would be TimeSplit.
 
Any ideas or workarounds?
 
3 REPLIES 3
FN
FN
Level VI

Re: k-fold cross validation in time series with JMP / JMP PRO

Confirmed: In JMP 14, K-fold it is actually a random sampling done k times. 

 

k fold Neural Network JMP.png

FN
FN
Level VI

Re: k-fold cross validation in time series with JMP / JMP PRO

And if the cutpointvalidation method is used as a column to fit trees (recommended for time series in JMP):

 

cut point tree jmp 14.png

 

FN
FN
Level VI

Re: k-fold cross validation in time series with JMP / JMP PRO

When using Neural Networks and the validation column, the results are a simple extrapolation. Test rows are not used in this interface either.

 

Ideas on how to implement a proper k-fold for either trees or NN?JMP NN.png

 

 

 

Recommended Articles