cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
  • Register to attend Discovery Summit 2025 Online: Early Users Edition, Sept. 24-25.
  • New JMP features coming to desktops everywhere this September. Sign up to learn more at jmp.com/launch.
Choose Language Hide Translation Bar
Zappy
Level III

Non-Parametric Variance Proportion

Dear JMP Community,

 

I have tried to read related post before on similar topics. It is not covering the questions I have.

Let me explain the background first.

 

Objective:

I am trying to find a statistical method to perform Variance Decomposition on a specific process.

We have huge number of data with lots of Factors (mixed of categorical and continuous data type) and Responses (continuous data type)..

 

Finding:

So far, the common method is to use ANOVA (eta square and omega square).

However, we need to consider the assumption violations for ANOVA. In case we do not meet the assumptions, especially those that are critical like variance homogeneity, we are worried about how reliable the eta square is.

 

Questions:

Is there any other recommended statistical method we could use like a "non-parametric" eta square, or anything that is equivalent to it.

We know the non-parametric "ANOVA" would be Kruskal Wallis, but I'm not sure if JMP can provide the "eta square" under Kruskal Wallis. At least I can't find it.

 

Another is Welch's ANOVA, however this method seems to work only on One-Way (one factor only), but my case have multiple factors.

 

Thanks in advance for you advice.

 

Note: I have a JMP Pro, in case there are available method under JMP Pro.

 

Thanks,

Chris

 

 

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Non-Parametric Variance Proportion

Hi Chris, 

I think the most direct answer to your question is that JMP doesn't incorporate this, except maybe in an Entropy R-squared context in Logistic Regression type models. All current methods, REML, EMS, Bayesian are inherently parametric (and use the normal distribution in some way). 

Nonparametric approaches for variance (or range) might be entropy-based, so you might research that (but it's out of scope in JMP's current implementation). You can also explore the MSA Design platform (Variance Component Estimator Expected Performance) to see if simulated components might help in the context of your specific problem. Also I recommend taking a look at Bootstrapping in JMP (https://www.jmp.com/support/help/en/18.1/jmp/bootstrapping.shtml) and Permutation testing (https://www.jmp.com/support/help/en/18.2/jmp/example-of-a-permutation-test.shtml). 

View solution in original post

6 REPLIES 6
Byron_JMP
Staff

Re: Non-Parametric Variance Proportion

I suppose you could use Fit model, then enter all your effects as random effects.  That would give you the variance attributed to each factor in the REML report

JMP Systems Engineer, Health and Life Sciences (Pharma)
Zappy
Level III

Re: Non-Parametric Variance Proportion

Hi Byron,

 

Thanks for the alternative method in case we are dealing with random effects, and most of the time we are.

 

As I understood, the REML are also subjected to similar assumption violation as ANOVA does.

In case there are assumption violations (equal variance, independence...), is the REML results for variance decomposition still reliable?

 

If not, do we have any other statistical method that does not require such assumption in the first place?

 

Thanks,

Chris

MRB3855
Super User

Re: Non-Parametric Variance Proportion

Hi @Zappy : With JMP Pro you have a lot of flexibility in modeling covariance/correlation structures, heteroskedasticity, etc.(via Mixed Model); However, even if the chosen mixed model properly accounts for heteroskedasticity and correlation (as you suggest may be needed in your case), simple eta squared type calculations (e.g., % of total variation for each factor) may be impossible since, for example, the total variance may be different for different groups. It seems to me then that those kind of simple eta squared calculations don't make much sense for those kinds of Mixed Models. 

MRB3855
Super User

Re: Non-Parametric Variance Proportion

Hi @Zappy : Another way to think of this perhaps:  If all you are interested in is eta squared (empirical measure % of total variability), but not modeling, parameter estimation, prediction, etc., then perhaps just apply the standard ANOVA model and calculate eta squared as you normally would...while recognizing that the goal is different than modeling etc. (via a Mixed Model).

 

Food for thought?

Zappy
Level III

Re: Non-Parametric Variance Proportion

Hi MRB3855,

 

Thanks for your comment and insight. Appreciate it.

 

I agree the eta square is not reliable given that we have to deal with huge amount of data, which have high probability for assumptions violation. So in case the eta square is not reliable under certain conditions, is there any other methods we could utilize apart from eta square?

 

From what I read so far, another alternative might be epsilon square which was derived from Kruskal Wallis.

Perhaps still not perfect for other reasons, at least the assumption requirement is removed.

 

I understood, JMP do not provide epsilon square at this moment.

 

Perhaps anyone know if there's an add-in for epsilon square. There's a nice add in back in 2016 (by Julian) but it only calculates eta, omega and partial eta.

https://community.jmp.com/t5/JMP-Add-Ins/Calculate-Effect-Sizes-Add-in/ta-p/22642

 

B.r,
Chris

 

 

 

Re: Non-Parametric Variance Proportion

Hi Chris, 

I think the most direct answer to your question is that JMP doesn't incorporate this, except maybe in an Entropy R-squared context in Logistic Regression type models. All current methods, REML, EMS, Bayesian are inherently parametric (and use the normal distribution in some way). 

Nonparametric approaches for variance (or range) might be entropy-based, so you might research that (but it's out of scope in JMP's current implementation). You can also explore the MSA Design platform (Variance Component Estimator Expected Performance) to see if simulated components might help in the context of your specific problem. Also I recommend taking a look at Bootstrapping in JMP (https://www.jmp.com/support/help/en/18.1/jmp/bootstrapping.shtml) and Permutation testing (https://www.jmp.com/support/help/en/18.2/jmp/example-of-a-permutation-test.shtml). 

Recommended Articles