Main effects and Interaction effects

[#1]

I have 28 music clips. 14 of them are composed in major mode and rest 14 in minor mode. I am looking at emotional response for the music and preference for the music against the mode of the music and the tempo of the music. I also want to look at the interaction effect of tempo and mode. 

The way I do this is by using Analyse ->Fit Model. Then, emotional response for the music as y-variable. I then choose Tempo and Mode. Under Macros, I do full factorial this yields me both main effects of tempo and mode and interaction effect of the two quantities.

However, I am running into two problems:

1) When I put only one variable, lets say tempo and check it's main effect it is significant. Same goes for the mode. But when I use the Full factorial, the main effect of tempo is no longer significant. 

2) I also notice a change in the F-ratio values for the main effect, when I check for the main effect separately and when I use the Full factorial.

Which is the correct way to account for the main effect?

[#2 · Accepted solution]

You did not mention it, but I assume your interaction term of Tempo*Mode is sugnificant.  And in ANOVA, interactions rule.  Basically, what it means, is that you can not talk about either of your main effects independently of each other.  The interaction between the two is what is important.

[#3 · Accepted solution]

Yes, the interaction of tempo*mode is significant. So does it means that I take into account the main effect values for tempo and mode when I use Full factorial?

[#4 · Accepted solution]

Yes, I would

[#5 · Accepted solution]

Your description (Full Factorial) of your initial model seems to be a reasonable place to start. You pose all the possible effects. Now you decide which ones are supported by the data.

1. The statistics simultaneously test all the effects in the linear combination (model). When you include a single variable, the estimate is biased by any other important variables that were omitted. Together they can weaken or cancel the estimate or they can artificially inflate the estimate. Omitted variables lead to biased estimates of the variables in the model. Model selection precedes model interpretation.
2. The changes in the F ratio are related to the first point. You are asking different questions each time you estimate a different set of model parameters.

So a couple of general points to consider:

• Posing all possible effects (e.g., Macro for Full Factorial) is a good way to start if you have more observations (i.e., rows) than terms in the model.
• Eliminate higher order terms that you decide are not important to the model (e.g., eliminate X^2 before X).
• Maintain model hierarchy: if X is not significant but X^2 is significant or X*Z is significant, then keep X in the model.

Linear models such as these are powerful but you will benefit from some education about their nature, estimation, and interpretation. This Community is a good place to get help but we also offer many forms of training and education.