Why are my fit models showing different factor significance? In "model 1", Random Block is a Random Effect rather than a Fixed Effect, whereas "model 2" lists all the factors as Fixed Effects.

[#1]

When I clicked Fit Model, "model 1" popped up, including "Random Block & Random" as a factor, which means "Random Block" is a Random rather than Fixed Effect. However, When I run "model 2" with just "Random Block" as a factor, everything becomes less efficient. Which is more correct and why?

Also, I'm used to the Model 2 output, in which I can click on Scaled Estimates. And I was taught at a JMP training that those are half estimates, so I need to multiply those by 2 to get the real changes in the response. The Model 1 output, which I've never seen before, does not allow you to select Scaled Estimates, why is that?

Model 1 and Parameter Estimates:

Model 2 and Scaled Estimates:

[#2]

Fitting different models will yield different results. Treating the block as a fixed or a random effect is a big difference. First, it changes the parameters in the model. A fixed effect for 4 blocks requires 3 parameters. That difference will change the degrees of freedom for the sum of squares for the model and the error, which will affect significance metrics. A random effect for 4 blocks requires 1 parameter. A mixed effects models by default in JMP uses REML for estimation. REML adjusts the degrees of freedom so this method will also affect the significance metrics.

[#3]

If we use block as random effect to build model, what prediction method we should use for the future run? I saw one is normal prediction, the other is conditional prediction. The formula is if block A = y1, Block B = y2, otherwise is 0. Can you explain how this works? What’s equation to calculate confidence interval? I cannot see a formula in the software like normal model.

[#4]

Just to generalize Mark's points, statistical significance is a conditional statement. If the inference space or the terms in the model or estimates of the error change, so can statistical significance.