turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- Discussions
- :
- How to generate a Learning Curve Plot for a machin...

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Aug 28, 2017 4:56 AM
(830 views)

Dear JMP users,

Is it possible to generate a plot of learning curve such as shown below? Both scripting answer or interactive answer would be greatly appreciated.

For example, for a machine learning method (e.g. linear regression), I would like to plot error rate in the Y-axis and training set size in the X-axis.

(figure source: http://blog.revolutionanalytics.com/2015/09/why-big-data-learning-curves.html)

Also, is it possible to plot such learning curve for multiple models (e.g., different regression models) in the same plot?

5 REPLIES

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Aug 28, 2017 6:18 AM
(819 views)

Yes, it is possible to generate this 'learning curve' with JMP. Scripting this simulation would probably be the best since the sample size is the simulation parameter. You might be able to use a formula with the simulation feature in JMP Pro.

Yes, you should be able to repeat this simulation with any modeling approach. You just capture a measure of the error for the training set and the validation set.

The blog post provides the R script, so you could use it, as is, if you have R installed on the same computer as JMP. A short JMP script is necessary to run R with this script.

Learn it once, use it forever!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Aug 28, 2017 7:27 AM
(809 views)

Thanks for your reply, Mark. I see running R script inside JMP would be an option.

I was wondering if you could point me to the general approach to or a brief outline of steps I need to take to use the similuation feature, which appears to be something new.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Aug 28, 2017 8:47 AM
(784 views)

I have some examples in the following blog entries, together with some sample scripts:

http://www.pega-analytics.co.uk/blog/logistic-regression-pt-2/

http://www.pega-analytics.co.uk/blog/stepping-carefully/

-Dave

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Aug 28, 2017 9:24 AM
(779 views)

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Aug 28, 2017 10:11 AM
(771 views)

I would only add to Dave's wonderful offer that the approach taken in the blog post that you cited could be used with JMP. The functions are different but there is nothing special or unique about the approach. I would start with Dave's links, though.

Learn it once, use it forever!