topic Re: DOE: how is Sum of Squares of each effects calculated? in Discussions

DOE: how is Sum of Squares of each effects calculated?

dadawasozo — Sat, 10 Jun 2023 23:20:14 GMT

Hi,

How is sum of squares (SS) of each effects calculated in DOE parameter estimate table?

any simple 2-3 factors DOE example to share?

Thanks

Re: DOE: how is Sum of Squares of each effects calculated?

Dan_Obermiller — Mon, 12 Oct 2020 13:25:30 GMT

The answer to your question can be found in any basic regression textbook. There are some fairly simple calculations that can be performed by hand if you happen to have only a single factor. Otherwise the calculations will involve matrix algebra.

You might want to consider enrolling in the free online statistics course titled "Statistical Thinking for Industrial Problem Solving". Based on the questions you have been asking, you might want to concentrate on the hypothesis testing, regression, and design of experiments portions of that course.

Re: DOE: how is Sum of Squares of each effects calculated?

MiBo — Mon, 31 Oct 2022 16:39:25 GMT

This post is a couple years old, but I too would like to know how the sum of squares is calculated. I refer specifically to the Effect Tests report.

I went to a basic regression textbook to learn. The book is called "Design and Analysis of Experiments" by Douglas C. Montgomery, 6th Edition. In that book there is an example calculation. Example 5-1 is a full factorial experiment with two factors and four replications. I have entered the data into Excel, Minitab and JMP and have compared the results to Montgomery. Minitab and Excel Analysis Toolpak match Montgomery, JMP does not.

The calculations from the text are attached as an image. So are images from Minitab and Excel. Also JMP, with and without the interaction term. The JMP project files is attached, with data table and Fit Least Squares report.

Across all the analyses, the sums of squares match except for JMP's Sum of Squares for Material Type in Effects Test. Therefore, its p-value is also different, making it appear as if the factor is insignificant. How is this particular SS calculated?

In the JMP output, Sum of Squares for Model in the Analysis of Variance matches the sum of SS from the model terms in Montgomery, Minitab, and Excel. It is not equal to the sum of SS for the model terms in JMP's Effect Test report. Why is this?

When the interaction is removed from the model, then the Sum of Squares for Material Type is the same as in Montgomery. Why is that? Why does the inclusion of the interaction term affect the sum of squares for that particular main effect? The sum of Sum of Squares in the Effect Test is equal to the Sum of Squares for the Model term in the ANOVA table. Why does it work if the interaction is not in the model?

According to Montgomery's analysis, all three terms are significant (two main effects and interactions). According to JMP, one main effect and the interaction are significant. I really need to understand why JMP gets a different result for the significance of Material Type.

Re: DOE: how is Sum of Squares of each effects calculated?

David_Burnham — Mon, 31 Oct 2022 17:31:24 GMT

I didn't see anything in the JMP project. Would you mind attaching the data table so we can take a closer look.

Re: DOE: how is Sum of Squares of each effects calculated?

MiBo — Mon, 31 Oct 2022 17:35:04 GMT

Here is the data table.

Re: DOE: how is Sum of Squares of each effects calculated?

David_Burnham — Mon, 31 Oct 2022 17:42:06 GMT

I've only taken a brief glance at this, but it would appear that to get the same results that you get from the other quotes sources you need to perform a Type I ANOVA. This is turned on from the red triangle:

Estimates> Sequential Tests

This I think, explains the difference in the outputs. It raises the question of which type of ANOVA is appropriate. Perhaps others can contribute to that discussion!

Re: DOE: how is Sum of Squares of each effects calculated?

Mark_Bailey — Mon, 31 Oct 2022 18:21:42 GMT

The problem is that you set the modeling type of Temperature to Ordinal. I would use Continuous. But Montgomery used Nominal for both Material and Temperature. When I change the modeling type to match, I get the same result:

I just verified that the data in your data table match the data presented in Doug's book.

I added the usual column properties for factors and responses, but it did not change the outcome. I thought that the significance might be affected if the factor levels were not coded first.

JMP makes sure that all the data types, modeling types, and column properties are properly set when you design the experiment with JMP. You have to manually add or adapt all the meta-data when you enter data by hand.

Re: DOE: how is Sum of Squares of each effects calculated?

MiBo — Mon, 31 Oct 2022 18:04:22 GMT

If I set the modeling type of Temperature to Continuous then I'm doing linear regression, with slopes and intercepts, and interactions between a linear effect and an attribute effect. There is then one degree of freedom for Temperature, plus loss of degrees of freedom for interactions between Temperature and Material Type. That is not what Montgomery did. Why do the calculations for JMP differ from Montgomery. Montgomery's demonstration is plain-vanilla ANOVA with two factors each at three (attribute) levels.

Re: DOE: how is Sum of Squares of each effects calculated?

David_Burnham — Mon, 31 Oct 2022 18:18:05 GMT

The book contains JMP output. Temperature has two degrees of freedom, so it's not being treated as continuous.

Re: DOE: how is Sum of Squares of each effects calculated?

MiBo — Mon, 31 Oct 2022 18:19:21 GMT

Thank you for that, using Sequential Tests did change the result and it makes it align with the other methods. Here's what Montgomery's supplemental material says about the topic:

"Type 1 sums of squares refer to a sequential or “effects-added-in- order” decomposition of the overall regression or model sum of squares. In sequencing the factors, interactions should be entered only after all of the corresponding main effects, and nested factors should be entered in the order of their nesting."

Also,

"For balanced experimental design data, Types 1, 2, 3, and 4 sums of squares are identical."

Now my questions are:

- What kind of Sums of Squares does JMP calculate by default (without my choosing Sequential Tests)?

- My data structure is balanced, why are the sums of squared different between the default and the Sequential Tests?

AHA: When I change Temperature to data type Nominal the result is the same as Montgomery! Something in JMP is using knowledge of the ordinality of Temperature and it changes the calculation for Sums of Square.

I sure do wish some information existed somewhere that explained how JMP calculated sums of squares. I'd now especially like to know how the model changes depending on Nominal versus Ordinal factors. Telling customers to refer to basic regression textbooks did not provide the answer.

Re: DOE: how is Sum of Squares of each effects calculated?

David_Burnham — Mon, 31 Oct 2022 18:35:40 GMT

Why do the calculations for JMP differ from Montgomery. Montgomery's demonstration is plain-vanilla ANOVA with two factors each at three (attribute) levels.

If you look at @Mark_Bailey 's latest example, the JMP output is the same. To have JMP give you plain-vanilla ANOVA based on two categorical factors, set both of the factors to nominal and the default ANOVA table will agree with Montgomery.

Re: DOE: how is Sum of Squares of each effects calculated?

Mark_Bailey — Mon, 31 Oct 2022 18:36:05 GMT

Doug was illustrating how experimenters and statisticians thought about the analysis 100 years ago when factorial designs were introduced. It is still valid, but it is not the only way to think about it. It is a balanced design and treats all factors as if they were nominal categories. Again, that is still valid but only one way. Why did they do it that way? It was the only way available to them at the time. So that they could perform the analysis by hand. A 'computer' in the 1920s was a job title - no kidding. Modeling the data this way allowed people to compute the simple sums that Doug shows by hand.

Today we use computers and software to perform regression analysis. We can realistically treat Temperature as a continuous effect. We can specify the linear model in very flexible terms and then compute an optimal design for estimating the parameters or the response. The design is fixed in this case but I can propose other modeling types and terms. Here is one possibility:

It might require additional runs to verify one of these models as accurate.

Re: DOE: how is Sum of Squares of each effects calculated?

Mark_Bailey — Mon, 31 Oct 2022 18:39:06 GMT

JMP uses Type III Sum of Squares by default.

Please see how the factor models determine the results.

Re: DOE: how is Sum of Squares of each effects calculated?

Dan_Obermiller — Mon, 31 Oct 2022 18:40:16 GMT

By default JMP uses the Type III Sums of Squares. This makes the results the same regardless of the order the terms appear in the model. Many people prefer Type III sums of squares for this reason. But Type I sums of squares are easier to calculate by hand, which is why many texts take that route.

As you noticed, Ordinal factors are certainly different than Nominal factors. For most ANOVA examples, you want a nominal modeling type (but this is problem specific). Most notably, how they are coded and the interpretations of the tests are different. For details, refer to the JMP help.

Ordinal factors: https://www.jmp.com/support/help/en/17.0/index.shtml#page/jmp/ordinal-factors.shtml#ww96069

Nominal factors: https://www.jmp.com/support/help/en/17.0/index.shtml#page/jmp/nominal-factors.shtml#ww65535

Or you can just search for "sums of squares for ordinal effects" and read the first paragraph to get an overview.

Re: DOE: how is Sum of Squares of each effects calculated?

jthi — Mon, 31 Oct 2022 18:41:15 GMT

Maybe these will give some information (JMP documentation):

Re: DOE: how is Sum of Squares of each effects calculated?

statman — Mon, 31 Oct 2022 19:52:23 GMT

Sure, I'll pile on...The default SS analysis for most, if not all, statistical software programs is Type III. This is because Type III's are robust to the order which you enter the terms in the model. When the computer calculates the Type III's, it first accounts for ALL of the other terms in the model and assesses how much does the term in question explain of the remaining variation. When using Type I, order matters. The calculations are performed in the order the model is written. Type I would be the appropriate analysis for nested or hierarchical components of variation.

Re: DOE: how is Sum of Squares of each effects calculated?

MiBo — Mon, 31 Oct 2022 21:41:54 GMT

It's not realy explained how the sums of squares are calculated for ordinal factors. I get it for nominal and continuous. A nominal model is one where subgroup means deviate from a grand mean, as in Montgomery's battery life example ANOVA. The continuous model has predictors and a grand mean and the sums of squares for the predictors are based on the difference between grand mean and predicted value at the limits of the model.

Ordinal models are some sort of build-up of effects starting from the lowest level and incrementally working up to the highest level. For those it's still not clear how Sum of Squares for factors are calculated, but it seems to have something to do with amount or variation of group means relative to the mean of a reference level.

Re: DOE: how is Sum of Squares of each effects calculated?

MiBo — Mon, 31 Oct 2022 22:05:24 GMT

For sure the setting of Nominal data type caused the difference to Montgomery. What I'd like to learn is how sums of squares are calculated for nominal factors.

Re: DOE: how is Sum of Squares of each effects calculated?

David_Burnham — Mon, 31 Oct 2022 22:54:01 GMT

on a separate note ... wouldn't it be nice if messages appeared in chronological order; would make it much easier to follow the discussion.

Re: DOE: how is Sum of Squares of each effects calculated?

MiBo — Mon, 31 Oct 2022 22:56:11 GMT

Or anyway more compact, like Reddit, to be able to see the indenting without so much which space. Kudo and Reply button could be on the left and much smaller.