Fit Model Custom Test Coefficients

[#1]

I am trying to jury-rig a post-hoc pairwise slope comparison with a custom test. I read through the documentation on custom tests but still don't understand how I am supposed to determine what coefficients to use to set up the tests. I have three treatments [PT CS OF] and want to do a pairwise test of the interaction between these treatments and a continuous variable [Duration]. If I understand the Custom test correctly each test will fill it's own column.

I want to test:            

      PT*Duration-CS*Duration=0;

      PT*Duration-OF*Duration=0;

      CS*Duration-OF*Duration=0;

Below I've attached a screen grab of the test coefficients and the results. I get a warning which I assume means I'm doing something wrong.

Thanks in advance for the help.

 

[#2]

If I am Bonferroni correcting the tests will that be enough to reduce false positives?

[#3]

I interpret what you ask to mean if you use the Bonferroni adjustment to your choice of alpha. Yes, that is one way to control the type I error rate with multiple comparisons.

[#4]

That's exactly what I'm planning to do. Thanks

[#5]

One follow-up question: 

I'm fitting the same basic model to a bunch of different factors. And I end up testing the same interactions in different circumstances.

Do I Bonferroni correct within the joint test within the model (so three levels), should Bonferroni correct all the tests within a single model, or should I pool all of these custom tests across all models and do one big Bonferroni correction?

Thanks

[#6]

I understand that the issue of multiple comparisons is within a study and a single data set. If I repeat a study, then I do not have to count the comparisons made in the first study in my adjustment. It sounds, though, as if you have multiple factors in the same study, so you would adjust based on the cumulative number of tests.

Is there a reason that you would not include all the factors at the same time in a single model instead of separate analyses for each factor?

[#7]

Just wanting to be clear, I am including all of the independent factors and significant interaction terms in the single model. I have run a MANOVA that collects all the dependent factors in the same model, and the whole model is significant. Beyond that, I was told to parse out the behavior of the dependent factors in the MANOVA by breaking it back down into individual standard least squares analyses. If you have a link to a video or slide deck explaining how to interpret an work with MANOVA's I would appreciate that.

[#8]

I think that you are on the right course!

Did you look at the Effect Tests? There is more than one way to analyze and model data, but I like to start with the Whole Model, proceed with the Effect Tests and other information to reduce the model, and then use Custom Tests to with interesting comparisons within a significant effect.

There is a lot of helpful material on the JMP site. I do not pretend to know about everything that is available. It is searchable, though. For example, I found this page with tutorials and videos for academic users (but anybody can use it).

[#9]

Okay, Custom test and MANOVA questions. 

When I would do the three column pairwise test for the interaction slope in the ANOVA it would give me all the test statistics and the warning about unable to make the contrast, but at least I would get all the estimates and test statistics. I tried the same thing in the MANOVA. I fit the model as I would with the factors and their interactions against all four response variables. (I removed some of the higher order non-significant interactions) I selected "test each column also" and "identity" as my response option. For a given column, say "total heading change" I tried a custom test the same way I would have in an ANOVA but the result was a single F-statistic and the same warning about being unable to test the contrast. When I do single pairwise tests I get results I would expect. Do I just do a billion pairwise tests in each of the Column tests I'm interested in and Bonferroni correct? 

 

 

[#10]

I suspect that it is an issue with degrees of freedom, but I am not familiar with the tests in the MANOVA procedure. Perhaps another member is able to advise you. I recommend that you consult the JMP or SAS software documentation for help. You can also contact JMP Technical Support (support@jmp.com) with your question.