topic Re: k-fold cross validation in time series with JMP / JMP PRO in Discussions

k-fold cross validation in time series with JMP / JMP PRO

FN — Mon, 25 Mar 2019 18:06:46 GMT

I want to use Bootstrap or Boosted Trees but with a k-fold method as I use it in the partition trees or neural nets. However I only see the option of adding a fix validation column.

Since I am working with time series, random samples cannot be used well to fit my models.

In addition to this, I am suspicious with the k-fold implemented in JMP as it always says random sample.

I am after a sequence like this one (at least):

And I think JMP does something like this (please confirm):

Here are all the options, the best for my case would be TimeSplit.

https://scikit-learn.org/stable/auto_examples/model_selection/plot_cv_indices.html#sphx-glr-auto-examples-model-selection-plot-cv-indices-py

Any ideas or workarounds?

Re: k-fold cross validation in time series with JMP / JMP PRO

FN — Thu, 28 Mar 2019 22:01:17 GMT

Confirmed: In JMP 14, K-fold it is actually a random sampling done k times.

Re: k-fold cross validation in time series with JMP / JMP PRO

FN — Thu, 28 Mar 2019 23:27:04 GMT

And if the cutpointvalidation method is used as a column to fit trees (recommended for time series in JMP):

Re: k-fold cross validation in time series with JMP / JMP PRO

FN — Thu, 28 Mar 2019 23:25:29 GMT

When using Neural Networks and the validation column, the results are a simple extrapolation. Test rows are not used in this interface either.

Ideas on how to implement a proper k-fold for either trees or NN?