I don't have a lot of faith in the 'plot residuals by normal quantiles' for doing model diagnostics. I have typically saved the residuals from the model to a new column, then used the Distribution platform to create a normal quantile plot there.
The fit of the solid red line does not seem to match up well with where the residuals fall using Fit Least Squares. The slope of the solid red line is not a good fit. I looked at a few sample data sets and saw issues with the default model scripts in these data sets:
- 2x3x4 factorial
- Bounce data
- Wine data
Below is a plot of the 2x3x4 factorial Residual Normal Quantile Plot. I've been told that the slope of the solid red line is based on the RMSE of 2.02.
If I save the residuals to a new column and regenerate the Normal Quantile Plot from the Distribution plot, the line is based on the standard deviation of the Residual Y column of 1.03. It looks like a much more sensible fit from a model diagnostics perspective.
Is this something that can be updated in future versions of JMP? Note that when you use Fit Mixed, the results in the Residual Plots there match my desired result from manually saving the residuals to a new column.