Centering IVs in regression only in interaction

[#1]

I have read in several places that with regression, JMP will mean center IVs that are involved in interactions but it will NOT center the simple effect versions of those IVs? First, is this true? Second, if true, how does this not violate the linear independence requirement of regression?

For example,
Y = b0 + b1x1 + b2c2 + b3x1x2

If x1x2 is made into x1'x2' because JMP centers both only for the interaction, then b1 is no longer the estimate of the effect of x1 on Y when x2 = 0 because the interaction term is no longer 0. The same is true of b2.

Given these problems, I assume I am misunderstanding what JMP is really doing. Can anyone clarify?

[#2]

@MRB3855 ,

The three formulas for the three different model versions are shown in the three tables I provided. All three are different. I copied them again here to save time.

Version 1. With the JMP mean centering option turned off and raw IVs you get:

Term	Estimate	Std Error	t Ratio	Prob>|t|
Intercept	164.198	36.29445	4.52	0.0001*
X1	-50.64385	3.103592	-16.32	<.0001*
X2	-14.74154	8.796501	-1.68	0.1058
X1*X2	-6.260865	0.726168	-8.62	<.0001*

Version 2. With JMP mean centering option turned on and with raw simple effect terms and mean-centered interaction terms, we get

Term	Estimate	Std Error	t Ratio	Prob>|t|
Intercept	83.558404	35.53908	2.35	0.0266*
X1	-42.48702	2.977176	-14.27	<.0001*
X2	-76.63734	4.675518	-16.39	<.0001*
(X1-9.88614)*(X2+1.30283)	-6.260865	0.726168	-8.62	<.0001*

Version 3. With your manually mean-centered variables for all variables and with JMP mean centering turned off.

Term	Estimate	Std Error	t Ratio	Prob>|t|
Intercept	-236.6291	19.63316	-12.05	<.0001*
Centered X1	-42.48702	2.977176	-14.27	<.0001*
Centered X2	-76.63734	4.675518	-16.39	<.0001*
Centered X1*Centered X2	-6.260865	0.726168	-8.62	<.0001*

[#3]

@MRB3855 

Sorry, I realized you also asked me to work through the problems to simplify.  I did this in my earlier posts. To summarize, when X2=0 for Version 1 and 2, and X2'=0 for Version 3.

Version 1: Y = 164.198 - 50.644 * X1

Version 2: Y = 83.558 - 42.487 * X1 + 8.158 * X1'

Version 3: Y = -236.629 - 42.487 * X1'

[#4]

Hi @Jimvano7 . Your arithmetic is not complete and correct. You need to expand 8.158 * X1' in version 2

( where X1’ = X1-9.88614 )  and you will see the equation in version 2 matches the equation in version 1 exactly.

And X2’ = 0 in version 3 is not correct if you are assuming X2 = 0. When X2 = 0, X2’ = 0 + 1.30283 = 1.30283. And, as I’ve just described in the sentence above, you have to be careful with X1’ as well. 

Carefully multiply everything out and the equations in versions 2 and 3 match the equation in version 1 exactly…I promise (I just did it in excel).

[#5]

@MRB3855 

I appear to have lost my reply for some reason - Maximum flood limit reached???. Trying again...

Thank you for noticing that I missed the negative sign on the mean for X2!!! And, I said X2’ = 0 for Version 3, not X2=0. Version 3 is correct as it was. I am copying the formulas here and correcting the sign (8.158 becomes -8.158) on Version 2.

To summarize, when X2=0 for Version 1 and 2, and X2'=0 for Version 3, then we are left with:

Version 1: Y = 164.198 - 50.644 * X1

Version 2: Y = 83.558 - 42.487 * X1 - 8.158 * X1'

Version 3: Y = -236.629 - 42.487 * X1'

I totally agree that all three formulas produce the same value for Y when X1=0 and X2=0. 

But, this wasn't my question. 

My question is:

What is the meaning of b0, b1, and b2 in Version 2?

• If b1 (-42.487) in Version 2 is supposed to be the influence of X1 on Y when X2 = 0, then b1 should be -50.644 as in Version 1. The only way to get b1 in Version 2 to match b1 in Version 1 is to add b3X1’. So, b1 cannot be the influence of X1 on Y when X2 = 0.  Same holds for b2.
• If b1 (-42.487) in Version 2 is supposed to be the influence of X1' on Y when X2' = 0, then b0 (intercept) should be -236.629 as in Version 3. The only way to get b0 in Version 2 to match b0 in Version 3 is to add b3X1’. So, b0 cannot be the mean of Y when X1' = 0 and X2' = 0, or when X1 = 0 and X2 = 0.

Therefore, b0, b1, and b2 are not meaningful IMO.  What am I missing???

Thanks,

Jim

[#6]

Thanks @MRB3855 and @Victor_G for your comments and help! I learned some interesting things along the way. This is edited from the original post.

To summarize my original question:

What are the meanings of b0 (intercept), b1, and b2 in the JMP intermixed model created with "mean-centering polynomials" turned on where only the variables in the interaction are mean centered (Version 2)?

Version 2 is: Y = b0 + b1X1 + b2X2 +b3X1'X2', where X1' is a mean centered version of a continuous X1 and X2' is a mean centered version of a continuous X2. Further, I assume X1<>X1' and X2<>X2'. 

Here are my answers so far and I would love to hear from anyone if what I concluded is wrong.

b0: the JMP output labeled "intercept" is not actually the intercept, which is defined by numerous sources as the value of Y (not mean of Y!) when X1 and X2 are equal to 0 (e.g., Montgomery, Peck, & Vining, 2001). 

When X1=0 and X2=0, then the intercept coefficient in the output is not the intercept because one must sum b0 + b3(-mean of X1)(-mean of X2) to get the true intercept. Under both Versions 1 and 3, the intercept is the true intercept. I can find no situation where the “intercept” coefficient in the Version 2 JMP output is the actual intercept. Is there one?

b1: the coefficient labeled X1 in the JMP output is the measure of the influence of X1 on Y when X2=mean(X2), not when X2=0 as in Versions 1 and 3.  Jaccard & Turrisi (2003) defined it as when X2=0. The “intercept” is the sum b0 + b2X2.

b2: the coefficient labeled X2 in the JMP output is the measure of the influence of X2 on Y when X1 = mean(X1), not when X1 = 0 as in Versions 1 and 3.  The “intercept” is the sum b0 + b1X1.

[#7]

Hi @Jimvano7  . I can’t speak for JMP (to answer your fundamental question). But…you do have a choice to “mean center” (version 2) or not (version 1).

[#8]

Hi @Jimvano7 : In addition to @Victor_G  's very good points, this thread may prove helpful;

https://community.jmp.com/t5/Discussions/Intercept-of-a-parabola/m-p/805020

[#9 · Accepted solution]

Hi @Jimvano7, and everyone else,

I am wading into this answer a bit late so forgive me for not responding to all the pieces (or for missing some nuance), but I wanted to offer up an answer (and video) I gave on this same topic on the community about 10 years ago. I did a little demonstration in the video with the prediction profiler that I think helped make some of the estimates clear. 

https://community.jmp.com/t5/Discussions/estimates-in-multipule-regression/m-p/10965/highlight/true#...

And here is a direct link to that video:

https://www.youtube.com/watch?v=LLh1V9MtKvs

I hope this helps!

@julian 

[#10]

@julian 

Thanks Julian.  Very helpful.

What is the meaning of the intercept when it is never truly the intercept? In every case I looked at, some other value had to be added to the "intercept" coefficient to get the actual intercept. 

Thanks,

@Jimvano7 

[#11 · Accepted solution]

The intercept is always some variation of where the line (or plane of regression in models with more than a single term) is on the Y axis when the other coefficients contribute nothing (i.e. are set to 0). When we center just the interaction term, "nothing" of the main effects takes on a changed meaning. Not numerically, we're always still talking about 0 of each predictor, but what that zero is pointing at in the population is changing because mean-centering the interaction shifts the zero interaction effect to the average behavior in the population, not the literal origin point (0,0) of the predictors.

In short, in a model like this, the intercept is more like an estimate in an analysis of covariance, an adjusted estimate based on removing, statistically, the average effect of the interaction from the plane. I don't find that explanation particularly helpful conceptually, so if you'll allow it, I'm going going to talk it through with the example I used before. 

I always find it helpful to see these things visually. Here's that example I used before, and let's look at the regression planes (which will be the same) for the centered polynomial model (left) and the uncentered model (right). I've added in a response grid at 50 for both (which is the intercept of the centered model). I have also put blue dots to show where the intercepts of the models are

Starting on the right, the intercept has a very easy interpretation. It's the value of Y where the plane of the response crosses 0 for both X1 and X2. That is, when there is 0 of study hours and 0 of previous knowledge. Easy.  (Important for later: we aren't even thinking about the interaction term here because in this kind of model, when X1=0, and X2=0, we know that the interaction adds nothing because that b3 coefficient is being multiplied by zeros)

For the centered model on the left, the model intercept of 50 is well above the value when there is 0 of both Xs. But why the bump of roughly 20 exam points?

A score of 50 is where we have roughly 40 of Previous Knowledge and 0 Study Hours; or, where we have 0 Previous Knowledge and 4 Study Hours. Here I've toggled on the value grids so you can see them line up with the blue dots I put before:

So, what gives?! We know these are not the means of Previous Knowledge and Study Hours, so it's not as simple as holding one variable constant and the other at their mean. One thing might pop out to you here: these points are a symmetric distance up the plane of response from the "true" (X1=0, and X2=0) intercept. And the only term in our model that exerts symmetric influence (in a scaled sense) on Y across the factors of X1 and X2 is b3, the interaction term.

What we're not accounting for yet is setting the *interaction* term, B3, to 0. And that zero happens at a different place in a model like this than where X1 and X2 are 0 (because of that centering); it happens at the means of X1 and X2, so we're talking about *average* interaction. The intercept of 50 here reflects a kind of adjusted baseline: it's what we would get at (X1 = 0 or X2 = 0) if there were no interaction effect in the population. Conceptually, an estimate the intercept adjusted for the presence of the interaction.

To me, this term resists a conceptual interpretation quite a bit more than any typical intercept but here's how I would frame it in this case: With the negative coefficient for the interaction term, we know that these factors are interacting antagonistically (more of one decreases the strength of the relationship between the response Y, and the other factor). That is, the more people know ahead of time, the less they get value from studying on average. Or, the more people study, the less on average they get value from how much they knew. The intercept in this model is trying to tell us what exam scores would be like *if that were not the case.* If that interaction weren't the state of the world we measured, then people who studied 0 hours would have had more value from their previous knowledge, and so they would do better on the exam, a bump up from an intercept of 30 to 50. And if that interaction weren't the state of the world we measured, then people who had 0 previous knowledge would have had more value from their studying, hence that same bump up of the intercept from 30 to 50. Like an ANCOVA, this is a statistical "as if" thought experiment.

I hope this helps!

Jules