topic Re: For RMSE, what value should be entered in ? in Discussions

For RMSE, what value should be entered in ?

MikeKim — Thu, 08 Jun 2023 21:15:11 GMT

https://www.jmp.com/support/help/en/16.2/#page/jmp/power-for-two-independent-sample-means.shtml#ww334584

I am referring to above link,

it says, input G1 RMSE to the Box.

But since Greek letter sigma is population SD, not RMSE.

Anyway, supposing the inputting RMSE is right procedure, How RMSE be calculated?

For case where data are {1,2,3,2,1,0},

MAD={0.5, 0.5, 1.5, 0.5, 0.5, 1.5}

MADsq={0.25, 0.25, 2.25, 0.25, 0.25, 2.25}

SS=5.5

SS/(n-1) <=sample Var

=1.1

SS/(n) <= population Var (Not proper this case)

=0.91

For this data, RMSE is sqroot of 1.1 ?

Just for information, sqroot of 0.91 is the best estimate of population SD ?

Thank you all.

Re: For RMSE, what value should be entered in ?

David_Burnham — Wed, 24 May 2023 07:43:17 GMT

Possibly I am being pedantic but reading the link, I didn't see it say "Input G1 RMSE". I read that G1 Std Dev "Specifies the assumed standard deviation for one of your groups". It then goes on to say that "the error standard deviation could be the root mean square error (RMSE) from a model fit".

So you could either

1. get the RMSE value from a model (e.g. Oneway)

2. Take the sample SD and apply a correction - something like a scale factor of (N/N-1)^2

3. calculate it yourself based on the formula for a population

4. If you have large sample sizes (>15?) just work with the sample SD, as the correction will be small

5. Just acknowledge that you are probably only doing an approximate calculation to get an order of magnitude estimate, and not worry about it

Re: For RMSE, what value should be entered in ?

MikeKim — Wed, 24 May 2023 08:27:42 GMT

I am sorry...

The link's original text, I made inference that I can input RMSE into that box.

I have insufficient back-knowledge to fully understand your points... sorry...

So,,, if I have G1 as {1,2,3,2,1,0},

What value can I enter as the 'Group 1 StdDev (sigma 1)' ?

as your recommendation 2,

(SQRT (1.1)) * ((6/5)^2) ?

=1.510284 ?

Thank you !

Re: For RMSE, what value should be entered in ?

MRB3855 — Wed, 24 May 2023 10:20:40 GMT

@MikeKim Since you want Sigma, you could calculate the upper 95% bound on Sigma and use that; for your example data {1,2,3,2,1,0}, the Sample SD is 1.049 and the 95% upper is 2.572. So you'd use Sigma = 2.572 (via the Distribution platform in JMP) in your power calculation; it may be conservative, but for most applications you'd rather overpower (n larger then necessary) than underpower.

Re: For RMSE, what value should be entered in ?

MikeKim — Wed, 24 May 2023 11:31:27 GMT

Best to understand!!!

Re: For RMSE, what value should be entered in ?

David_Burnham — Wed, 24 May 2023 12:18:47 GMT

nothing wrong with a conservative estimate - just not sure where you get the upper bound on the SD from - please don't confuse it with the "Upper 95% Mean" which is something completely different.

Re: For RMSE, what value should be entered in ?

MRB3855 — Wed, 24 May 2023 12:29:27 GMT

Good question @David_Burnham To get confidence intervals on m and s: In the Distribution platform, from the red triangle menu, choose the following:

You will then get the following intervals.