topic Re: Cross Variable T-Tests in Discussions

Cross Variable T-Tests

NathanFisk — Fri, 09 Jun 2023 00:23:21 GMT

I have a dataset with a categorical variable (treatment).
The data columns have comparisons that take place at different times (1, 2, 4, and 6).
For each comparison variable there are columns for "Device applied" and "No Device".

So a person could be either an active or placebo (treatment) with either a device applied or not applied for each comparison variable.

For example ... "Comparision 01" could have four different combinations ...
Treatment - No device
Treatment - Device
Placebo - No device
Placebo - Device

Each of these would have its own mean. What I'm trying to figure out how to do in JMP is to do cross-comparison T-tests which would compare "Placebo - No device" against "Treatment - device". I can compare Active to active or Placebo to placebo, but I'm not seeing any way to compare across the groups.

Do I just take the results from "Distribution" with Treatment as a "By" splitter and then do a "Test mean" by plugging in the comparison I want into the Hypothesis test? That seems a bit clunky.

Re: Cross Variable T-Tests

Mark_Bailey — Wed, 21 Oct 2020 18:31:11 GMT

This case is a classic 'two-way ANOVA" scenario Set up your data table with Condition (Treatment or Placebo), Device (Yes or No) and the Y (response). Select Analyze > Fit Model. Select Condition and Device, click Macros, and select Full Factorial. Select Y and click A. Click Run.

Everything you need is there.

Re: Cross Variable T-Tests

NathanFisk — Wed, 21 Oct 2020 21:21:35 GMT

Thanks - I've done that report but there is no table I can see which shows the T-tests comparing the cross-groups. There is a Least Squares Means table for the Treatment*Device leverage plot, but there do not appear to be any t-tests of the means against each other. The red dropdown Student's T gives a generic letter report table but it isn't showing the actual t-test values.

Oh - and just to make sure I've got the data table set up properly ... the table has (essentially) two entries for each subject. For example - Subject "HMA" exists both as row #1 and row #61 (there are 60 cases). So there is a stacked effect where each subject has 2 entries to account for their Placebo/Active response and the Device/No Device response. So the 60 cases in the data table create 120 rows (2 for each respondent). That how the ANOVA needs to have its data structured?

Re: Cross Variable T-Tests

Mark_Bailey — Thu, 22 Oct 2020 12:17:32 GMT

First of all, I do not recommend using the Student t-test in this post hoc analysis. You will experience inflated type I error rates. Use Tukey's method instead. Also, are you asking for the t-ratios? Why is that necessary?

This sounds like two separate, parallel studies. You obtain a response for Treatment (Placebo, Active) and another response for Device (Yes, No). Is that interpretation correct? I assumed that each subject was observed once with a combination of Treatment and Device.

Re: Cross Variable T-Tests

NathanFisk — Thu, 22 Oct 2020 14:59:29 GMT

The trial was set up in this manner...

60 subjects were enrolled.
These 60 subjects were assigned to either Active or Placebo treatments (final 40 active, 20 placebo)
All 60 subjects in addition to the treatment used a device on a section of the body (right side) with no device on the left side.
Evaluations were performed for several characteristics both by the subject and an expert observer.

So all 60 subjects have a "device side" set of evaluations and a "no device" set of evaluations while 40 subjects are in the active arm and 20 in the placebo arm.

I'm able to get t-test mean comparisons for the different evaluations of the 1st visit to the last visit. I can also get the t-test results for comparing the placebo group means to the active means for each visit. I can also get the t-tests for comparing the evaluation means for the device side against the no-device side. What I'm trying to coax out are the t-tests that compare the means of the "Placebo - No Device" against the "Active - Device".

As an illustration, For "Evaluation01" I have the following means...
Placebo: No Device: Visit 01: 2.95
Placebo: Device Side: Visit 01: 2.95
Active: No Device: Visit 01: 2.65
Active: Device Side: Visit 01: 2.65

Doing a t-test comparing Placebo: No device versus Placebo: Device Side (2.95 to 2.95) yields a value of 1 (obviously) because the two are the same.
Doing a t-test comparing Placebo: No device versus Active: No device (2.95 to 2.65) yields a value of 0.19 for the first visit.

Doing a t-test comparing Placebo: No device: Visit 01 versus Placebo: No device: Final Visit (before/after test of 2.95 to 2.65) yields a value of 0.03 (a significant difference).

Comparing Placebo: No Device (2.95) directly to Active: Device Side (2.65) is where I'm having troubles getting that direct mean to mean t-test. Obviously the result will be similar to the Placebo: No device vs. Active: No device test (since the numbers are the same). However, not all the evaluations are the same and so I need to do the tests on all of these cross-comparisons to confirm the presence (or absence) of any significant differences.

Hope that helps explain the setup a bit better.

Re: Cross Variable T-Tests

Mark_Bailey — Thu, 22 Oct 2020 15:31:07 GMT

You used Subject for 'blocking,' which is smart. It will have a random effect, though. You should compute the Treatment difference for a 'matched pairs' analysis. Otherwise, the subject variation will obscure the Treatment effect.

Alternative, use Fit Least Squares with Treatment, Device Status, and Subject as the effecfs. You can include an interaction term. Then you can use the various tests in this platform.

Re: Cross Variable T-Tests

NathanFisk — Thu, 22 Oct 2020 16:20:15 GMT

I actually did a matched pair approach the first time around, but I could never get the result to produce a t-test on the comparison of interest. Below is an example of the output obtained when setting it up with the matched pair test in the Specialized Modeling menu. The comparison puts the Subject evaluation on the non-device side and matches it against the Subject evaluation on the device side as the Y variables for the paired response and then sets the Active/Placebo variable as the "X grouping".

The matched pair will generate a table that shows the "Across Groups" means properly. But is the "Test Across Groups" table's F-probability the T-test? I've never seen a T-test manifested in that manner.

Re: Cross Variable T-Tests

NathanFisk — Fri, 23 Oct 2020 14:45:18 GMT

Bumping to see if the matched pair "Test Across Groups" table's F-probability is the T-test I'm looking for that compares the cross groups.

Re: Cross Variable T-Tests

Mark_Bailey — Fri, 23 Oct 2020 18:41:12 GMT

Bump back, Nathan!

I meant 'matched pairs' in the general sense of blocking, not the specific sense of the Matched Pairs analysis. That analysis is not appropriate because it is a one-sample t-test of the mean. You have a two-way design with a random effect.

So I mocked up your data:

This set up allows me to specify this model for the analysis:

There are no fixed or random effects in the simulated response in this mock up. The results available are:

Now you can use one of several approaches to make the comparisons that you want.

Multiple Comparisons: open the Effect Details outline, click the red triangle next to the interaction term Treatment*Device, and select LSMeans Tukey HSD.

Contrasts: open the Effect Details outline, click the red triangle next to the interaction term Treatment*Device, and select LSMeans Contrast.

Use this interface to define one or more specific contrasts (tests) and the joint contrast.

Custom Tests: click the red triangle at the top of the platform, and select Estimates > Custom Tests.

Use this interface to construct one or more custom tests and the joint test.

Please see Effect Details, Contrasts, and Custom Test in the JMP Documentation.

Re: Cross Variable T-Tests

NathanFisk — Mon, 26 Oct 2020 18:55:23 GMT

Thank you for you assistance on this question ... it has been very helpful.

My understanding is that my primary interest is in the Tukey HSDs report where I believe the LSMeans Differences table has a red arrow that allows the "Ordered differences" report. In the Ordered Differences report it lists each comparison of the differences between the means along with the p-value of the test. This p-value is the summary statistic of the test comparing the two means. Sound right?

Re: Cross Variable T-Tests

Mark_Bailey — Tue, 27 Oct 2020 11:56:10 GMT

Sounds right.