Subscribe Bookmark RSS Feed
julian

Staff

Joined:

Jun 25, 2014

Calculate Effect Sizes Add-in

This add-in ingests estimates from an ordinary least squares report (produced by Fit Model) and saves a table of model effect estimates, F-ratios, p-values, and three measures of effect size as formula columns (Eta Squared, Partial Eta Squared, Omega Squared). If pairwise analyses are present with 'Ordered Difference' reports available, a combined data table of estimates including pairwise effect sizes (Cohen's d) will be generated. If no pairwise analyses are present this add-in will issue a prompt to add these to the selected report.

Instructions (after installing add-in):

  1. Fit an ordinary least squares model using Analyze > Fit Model
  2. Go to Add-Ins > Calculate Effect Sizes > From Least Squares Report (Fit Model)
    • If more than one least squares report is open, a dialog will appear to select which analysis to use

     For models with a categorical factor:

  • If pairwise analyses (e.g. Student's t-test) and "Ordered Differences' reports are available, Cohen's d for available pairwise effects will be calculated and saved to a combined data table
  • If no pairwise analyses have been produced (or "Ordered Differences' reports have not been produced) a prompt will appear to add these analyses and calculate these effect sizes.

11664_Screen Shot 2016-05-23 at 2.34.56 PM.png

Output:

  • Table of model effects and effect sizes

11665_Screen Shot 2016-05-23 at 2.57.45 PM.png

  • If available, table of pairwise effect sizes

11662_Screen Shot 2016-05-23 at 2.01.12 PM.png

  • Overall model statistics are saved to tables as table variables.

Statistical Notes:

  • This add-in provides estimated effect sizes for ordinary least squares models (models with random effects are not supported)
  • Effect Sizes Produced:


Model Effect Sizes:


Eta Squared

11640_etasq.gif


Partial Eta Squared

11641_eta.gif


Omega Squared

11642_omega.png

    • Negative values of omega squared are possible and are automatically set to 0


Pairwise Effect Size:


Cohen's d

11649_cohens-d-formula.png

Update History

  • v0.03 - Added Cohen's d calculation for pairwise differences, and dialog to produce pairwise analyses if not present
  • v0.04 - Added code to suppress prompt to add pairwise comparisons for models without categorical factor
  • v0.05 - Fixed defect for running on Windows (Fit Least Squares reports were not detected). Added support dialog for selecting from available Least Squares reports:
Comments
cparlett0

11712_JMP eta.png

I keep getting this error message. What should be selected? The output of the Fit Least Squares output is my active window (On a PC).

julian

There appears to be some differences in window naming on the windows and mac -- working on a fix for this now.

julian

Done! Rather than using the active window the new version searches the open windows for least squares reports. If only one is available that report will be used, otherwise a dialog will appear to select which least squares report to use.

cparlett0

This is Great! It's easy enough to calculate these by hand, but its even easier to have it calculated automatically! I also love that it includes Eta^2, Partial Eta^2 and Omega^2.

For those who are not familiar with these can also see how they're calculated but right clicking the column and selecting "Formula"!

bernhard

Thanks for this add-in, this is extremely useful. And being able to see the actual formula used is indeed a great plus.

Do you have any plans for creating a effect size add-in that also works with repeated measures or mixed models? This would be quite valuable, especially when using JMP for teaching to non-statisticians - I just discovered your repeated measures add-in, and having a way to produce effect sized more easily for repeated measures and mixed designs would be a great plus.

julian

I'm so glad you are finding this add-in useful!  Calculating effect sizes for mixed-models is a bit more complicated, but it's definitely something I would like to incorporate. I'll post updates here as soon as that's possible.

bernhard

Thanks Julian, that would be really useful! I hope JMP will incorporate and work on some of these changes themselves - overall the usability and interactivity of the interfaces is quite good, but there are a few points where it falls short (e.g., calculating effect sizes, checking for assumptions and reporting on them, mixed models (although your add-in fixes most of that issue nicely and elegantly). I'd love to use JMP for all my analyses but find myself going back to SPSS or another software for a few things, hope I won't need to any more in the near future.