Fit Model Custom Test Coefficients

[#1]

I am trying to jury-rig a post-hoc pairwise slope comparison with a custom test. I read through the documentation on custom tests but still don't understand how I am supposed to determine what coefficients to use to set up the tests. I have three treatments [PT CS OF] and want to do a pairwise test of the interaction between these treatments and a continuous variable [Duration]. If I understand the Custom test correctly each test will fill it's own column.

I want to test:            

      PT*Duration-CS*Duration=0;

      PT*Duration-OF*Duration=0;

      CS*Duration-OF*Duration=0;

Below I've attached a screen grab of the test coefficients and the results. I get a warning which I assume means I'm doing something wrong.

Thanks in advance for the help.

 

[#2]

These custom tests are also known as contrasts elsewhere.

The miss-specification is because you entered +1 for both levels in the first test. Make sure that the coefficients sum to zero. Then make sure that the value after "=" is the null hypothesis. Entering zero is usually appropriate if you are testing for a significant difference but might be otherwise in some situations.

[#3]

Thanks for the clarification. I've attached another window below. If I understand correctly I have set up the first contrast (CS*Duration-OF*Duration=0) correctly. I thought the negative coefficients were supposed to let me test against the third categorical effect (PT*duration), you can see that in the expanded estimates but not in the set up for the custom test. How do I test against PT*Duration? 

 

[#4]

I believe that you set the coefficient for the first estimate equal to 1 and leave the rest 0. You can check the result. The Value should be the difference between the estimates that you are comparing to null of zero difference.

[#5]

Still don't think I'm getting it right. 

First, I'm still getting a non-contrastable warning (see below).

Second, It appears that by making the second coefficient -1 for the CS*Duration -(-OF*Duration)=0 cross adds the value of the two which is not what I want. It ends up being -0.0447.

Third, if I am trying to compare PT*Duration to OF*Duration (OF*Duration-PT*Duration=0) the Value should be 0.00777 instead it is just the value of OF*Duration (0.0174926). So leaving the coefficient at 0 does not seem to work.

 

[#6 · Accepted solution]

I apologize for the confusion. I am not sure about all the confusion, so let's start at the beginning.

You started by saying that you want a post hoc pairwise slope comparison with a custom test. So the trend line for each group appears to be different and you want to test the difference. Let's try.

(NOTE: I received fantastic help from a colleague in technical support. My reply is based on his explanation.)

You can perform the custom tests in one step, but I am going to break it into two steps so you can see the development of the correct coefficients.

Step 1: Use custom tests of the slope of the line for each treatment. The first column in the custom test should be [ 0, 0, 0, 1, 1, 0, 0 ]. That is, the only parameters with a 1 are Pulse Duration and Treatment[PT]*Pulse Duration. The reason is because the slope of the line is the common slope, Pulse Duration, plus the differential slope for Treatment[PT]. Their sum is the slope you want to test against 0. The second column should be [ 0, 0, 0, 1, 0, 1, 0 ] because you are adding the Pulse Duration and the Treatment[CS]*Pulse Duration. Finally, add the differential slope for the third treatment but it is not explicitly reported. But we know that it is the negative of the sum of the other two parameters because of the way that the JMP parameterizes a categorical factor. So the third column should be [ 0, 0, 0, 1, -1, -1, 0 ].

Again, you do not need this step for the actual custom test that you want. It only helps my explanation of the actual tests. At this point we have determined the correct coefficients for the custom tests of the individual slopes.

Step 2: You want to compare the individual slopes, not to zero, but to each other, two at a time. We simply subtract the columns used in step 1 two at a time! To compare the slope of PT to CS, we subtract the column of coefficients in step one for CS from those for PT. So [ 0, 0, 0, 1, 1, 0, 0 ] - [ 0, 0, 0, 1, 0, 1, 0 ] = [ 0, 0, 0, 0, 1, -1, 0 ]. The column to test PT to OF is [ 0, 0, 0, 1, 1, 0, 0 ] - [ 0, 0, 0, 1, -1, -1, 0 ] = [ 0, 0, 0, 0, 2, 1, 0 ]. Finally, the column to test CS to OF is [ 0, 0, 0, 1, 0, 1, 0 ] - [ 0, 0, 0, 1, -1, -1, 0 ] = [ 0, 0, 0, 0, 1, 2, 0 ].

As before, you can always check the coefficients by comparing to the Value that JMP reports for the test with the hand calculations of the sums and differences of the parameter estimates.

[#7]

Thanks, this is the solution I'm looking for and the results makes sense. The only final hiccup is I am still getting the "Warning: Non-Testable Contrast". Is that saying the joint test is Non-Testable or a one of the contrasts is not testable? When I run the pairwise contrasts independently I don't get the same response.

[#8]

The warning is about the joint test. There are not enough degrees of freedom available for the number of joint tests.

[#9]

Thanks, I don't think I need the joint test. Just the independent contrasts.

[#10]

The joint test is used by some analysts to protect against a high false positive rate. You might perform a lot of tests. They are not adjusted individually for the number of tests.