topic Re: Choosing the right Comparison of means in Discussions

Choosing the right Comparison of means

bobmorrane — Mon, 15 May 2023 13:11:16 GMT

Hello,

I'm analyzing biological data where the output is a continuous variable. The tests consists of different treatments on plants which are then assessed for a level of disease (my output). If I plot a graph, all I get is a bunch of means with very large error bars that overlap each other. So, I'm trying to do the analysis with the little letters, to confidently be able to say which treatments are different from each other.

Now the issue I have, is that each repeat consists of two data points which are not independent from each other (two parts of the same plant). In this case I heard that usual comparisons like Tukey-Kramer don't apply.

So my question is, which comparison should I use, or which data transformation, to make sure I'm doing things right?

Re: Choosing the right Comparison of means

Byron_JMP — Mon, 15 May 2023 17:01:31 GMT

It might be easier to understand the problem with an example table.

It sounds like you have data like this

Three or more treatments.

Multiple plants per treatment

Multiple sample points (tissue types) per plant

Multiple samples of each tissue

In the end, you might want to see the effect of the treatment on both leaf and stem for example for each treatment.

If the multiple treatments are something like control and several dilutions of the goo, then that would change some things too.

Re: Choosing the right Comparison of means

P_Bartell — Mon, 15 May 2023 18:09:17 GMT

In addition to everything my former colleague (I'm retired) @Byron_JMP asked, is the arrangement/combination of treatments and plants some sort of designed experiment or is it just happenstance treatments applied willy nilly to plants as nature or someone decided to administer? This could have a large impact on how you should optimally analyze the responses...and not just multiple comparison tests.

Re: Choosing the right Comparison of means

Byron_JMP — Mon, 15 May 2023 18:39:16 GMT

@P_Bartell , I'm sure its a well-designed, well-controlled experiment. Plants are painfully slow, so planning is huge.

It's probably a multiple-variable problem, so a simple Turkey Kramer won't work. Just setting up the problem in Fit Model will help, and then we can look at the least squares means for the main treatment variable.

Re: Choosing the right Comparison of means

bobmorrane — Wed, 17 May 2023 16:47:30 GMT

Hello,

I like to believe it's well designed, cause I took part in the design process ;)

but jokes appart, it's plant especially grown for the test, with treatments applied to them in a standard, professional way.

So, i got various treatments applied to 2 types of plants (variations of the same crop), and the level of disease (my output) assessed at different times (1,2,3 days). So we need to at least group things by assessment time.

Here's the data in anonimysed way, with the scripts attached. I look at both Student and Tukey tests, since one is deemed too permissive and the other a bit too restrictive. I don't see much differenciation here.

By the way gents, it's quite a bit of a pain to generate these connecting letters report in the fit Y by X tool. because I have 6 groups, that's 6 red triangles to click for the compare means. It's needlessly tedious. Do you jnow of a method to make things quicker?

ANyone reading this : have a look at this wish from the wishlsit and give it a kudo ! like this we could get the connecting letters built-into the graph builder.

Display letters of significance in graph builder when using boxplot and in fit y... - JMP User Community

Re: Choosing the right Comparison of means

bobmorrane — Mon, 22 May 2023 15:05:53 GMT

@P_Bartell @Byron_JMP , data table attached

Re: Choosing the right Comparison of means

MRB3855 — Mon, 22 May 2023 16:12:51 GMT

Making sure I understand your data @bobmorrane : is each set of six rows one "subject" (what you are calling "Rep")? And then are the first three rows from each "subject" three repeated measures on one part of the plant, and the next three rows from each subject repeated measures on another part of the plant? And "Name" contains the treatments of interest (that you want to compare wrt their effects on those two parts of the plant, respectively). Do I have all of this correct?

Re: Choosing the right Comparison of means

statman — Mon, 22 May 2023 17:06:55 GMT

Bob, I am not an SME regarding your situation, but I have looked at your table and have the following thoughts. Some of my questions and comments are more Socratic rather than require an answer.

1. First and foremost, is the variation in your response of any practical value? There is no context provided regarding how much of a change in disease is of scientific interest? This is always more important than statistical significance.

2. If I understand your data, it looks like you have one factor (Name) and it is tested at 10 "levels". You have 2 Plant types (crossed with Name). You have 10 plants (Rep) for each level of Name and each plant is measured twice (is this in different locations on the plant or the exact same location on the plant?) and measures of Disease over 3 time periods for "within" plant (you might consider the plants nested within Type and Name). We could debate whether the systematic sampling of time periods is crossed with Name. I have some questions regarding this structure. The numbering scheme for Rep looks questionable. Can the same plant be given a different level of Name? If not, there should be no repeated Rep numbers.How many plants are actually in the study? It also looks like you have 2 measures of disease for each plant? I have attached another version of the data table. It is imperative the data table match how the data was actually collected or any analysis will be suspect.

What is the intent of the study? (e.g., Are you trying to pick the "winner" (e.g., best Name to reduce max disease?) or trying to understand what contributes to disease (growth))? Why 10 plants? Why 3 time periods, each a day apart? Are you interested in the rate of change of disease over time or just maximum disease or what? I guess the question is what do you hypothesize the Name will do to disease? Why?

3. Have you assessed the measurement system? Did you measure the disease multiple times on the same location on same plant at the same time period? As it appears now, the measurement errors are likely confounded with within plant?

4. A graphical look at the data without any summarization shows the time periods (DAI) are the largest source of variation in the study. This may be expected and may be of no interest (hence why you may want to look at the data by DAI), but also notice the variation increases with DAI. There may also be some "outliers" in your data.

Colored by DAI

Colored by Name for DAI=1

Re: Choosing the right Comparison of means

Byron_JMP — Mon, 22 May 2023 18:15:39 GMT

Regarding: "By the way gents, it's quite a bit of a pain to generate these connecting letters report in the fit Y by X tool. because I have 6 groups, that's 6 red triangles to click for the compare means. It's needlessly tedious. Do you jnow of a method to make things quicker?"

Yes, Much quicker. Hold down your control key (command on Mac) when you do compare means for the first by group. The Control key broadcasts your key stroke to all like objects in the window. It works for resizing graphs too.

(still looking at the bigger question)

Re: Choosing the right Comparison of means

Byron_JMP — Mon, 22 May 2023 19:26:38 GMT

Dude!!, this is such a great data set, wow.

I changed a couple column's modeling types and added a couple of new scripts to the table.

I tried modeling the data with a Response Surface Method (RSM). Rep(order?) seems to have an effect, so in model 2 I made that a Random Effect.

I'm not sure what you were hoping to find, but this is very interpretable data, and the study design is just fantastic too.


Fit Model(
	Y( :Disease ),
	Effects(
		:Name, :"Assessment time (DAI)"n & RS, :Plant type,
		:Name * :"Assessment time (DAI)"n,
		:"Assessment time (DAI)"n * :"Assessment time (DAI)"n, :Name * :Plant type,
		:"Assessment time (DAI)"n * :Plant type
	),
	Random Effects( :"Rep (Order)"n ),
	Personality( "Standard Least Squares" ),
	Emphasis( "Effect Leverage" ),
	Method( "REML" )
);

I changed the DAI column to numeric continuous, and the Rep Column to Numeric Continuous.

Re: Choosing the right Comparison of means

bobmorrane — Tue, 23 May 2023 09:08:59 GMT

Hi @MRB3855 ,

reps, or "repeats", are the same measurements done on different plants. So, plant 1, plant 2, plant 3. Each plant is grown separately in its own pot. for each rep, there are two measurements, which are done on two leaves of the same plant.

Then there's the assessment time. So, one measruement of the output value (disease) done on day one, then another the next day, and another one on day 3. Datapoints generated on the same day can be compared, but not over several days.

Name contains the treatment of interest yes.

"Plant type" are two slightly different types of plants. The goal is to see whether one type is more resistant to the disease than the other. So, interesting to compare the disease value over both types of plant overall, or both types on a single treatment.

Re: Choosing the right Comparison of means

bobmorrane — Tue, 23 May 2023 10:39:23 GMT

Hi @statman ,

1) is the variation in your response of any practical value?

Good question. Do it is a level of disease that can be measured relatively easily to the 0.1 unit. That being said, in a real life scenario, the end-user would not see a difference that small. I'd say the evarage values between two treatments (what you call levels) would need to have at least 4-5 units difference to really be of use.

2. If I understand your data, it looks like you have one factor (Name) and it is tested at 10 "levels". You have 2 Plant types (crossed with Name). You have 10 plants (Rep) for each level of Name and each plant is measured twice (is this in different locations on the plant or the exact same location on the plant? Two different parts of the plant ) and measures of Disease over 3 time periods for "within" plant (you might consider the plants nested within Type and Name). We could debate whether the systematic sampling of time periods is crossed with Name. I have some questions regarding this structure. The numbering scheme for Rep looks questionable. Can the same plant be given a different level of Name? If not, there should be no repeated Rep numbers.How many plants are actually in the study? 10 plants per "Name". They are grown independently. Then when treatment is applied, two leaves of each plants are isolated and the disease is applied. Because two leaves come from the same plant, my colleagues made the argument that the two leaves are NOT independent statistical units, hence why they are regrouped under the same "rep". It's like if we assessed 10 patients on their right hand and left hand.

It also looks like you have 2 measures of disease for each plant? Yes, two measures on two different leaves. I have attached another version of the data table. It is imperative the data table match how the data was actually collected or any analysis will be suspect.

What is the intent of the study? (e.g., Are you trying to pick the "winner" (e.g., best Name to reduce max disease?) or trying to understand what contributes to disease (growth))? We're trying to see if we can find additives (what I called adjuvant in my table) that significantly improve disease reduction from the active. So, what we want to see :

The active alone must be better than the untreated
Hopefully, we can find some treatments of active + adjuvant that are significantly better than the active alone

Why 10 plants? Cause we thought it would be a good number. Given the relatively high variability of the output and the relatively low efficacy of the treatments, it's kind of a minimum. Also, since they are grown plants, it's difficult to go higher than that, cause there's already 200 plants in this trial. That takes a lot of space and handywork. We could have more plants per treatment, to maybe get better stats, but then we'd loose on the number of treatments, which is not desirable. So 10 is a good compromise.

Why 3 time periods, each a day apart? Because it's easy to do. Once you measure the disease at one particular time, given all the effort you've put in to get there, it doesn't cost much more to wait another day and make another observation. It's quite usual with this kind of study. It depends on the nature of the disease of course, but this one progresses fast enough that you can see some growth within a one day difference.

Are you interested in the rate of change of disease over time or just maximum disease or what? I guess the question is what do you hypothesize the Name will do to disease? Why? Not rate of change, just absolute value. Basically, you got a good plant you want to protect against an bad disease. The idea is to minimize the amount of disease observed on treated plants. In an ideal world, you want 100% efficacy (aka zero disease). Or at least a significant reduction on the level of disease so that bad disease doesn't rob the yield from the good plants. But there is only so much the active can do on its own. That's where the additives (or adjuvants) come in. Ideally, we want to find additives that will have a synergystic effect with the active and in the best best case, the additive doesn't have much of an effect on its own. But here, there's clearly nothing exciting, which is why I'm trying to dig deeper to see if we can draw any conclusions. Usually, I'd be happy just looking at a Student's and a Dunett's test on the means. (I know student is not so suitable with more than 2 levels, but then Dunnett's can be a bit too restrictive, so a comparison of the two can be interesting). Here Dunnett's says there are no differences between the different levels, and student says maybe additive E is bringing something.

3. Have you assessed the measurement system? Not sure what that means. The values are a physical measurement that is easily done, so I wonuldnet worry about it too much.

Did you measure the disease multiple times on the same location on same plant at the same time period? yes, it's measured at three different time periods on the same leaves (day1, 2 and 3).

As it appears now, the measurement errors are likely confounded with within plant? no idea

Re: Choosing the right Comparison of means

bobmorrane — Tue, 23 May 2023 11:55:19 GMT

@Byron_JMP ,

thanks for the tip with CTRL, it does save a lot of time. Though the bigger issue remains, it's still highly impractical to Add the letters on a graph.

Re: Choosing the right Comparison of means

bobmorrane — Tue, 23 May 2023 11:56:04 GMT

Sorry, I meant Student and Tukey, not Student and Dunnett's.

Re: Choosing the right Comparison of means

Byron_JMP — Tue, 23 May 2023 12:00:56 GMT

I agree, its very difficult to get the letters on the graph.

Please go to the Wishlist tab at the top of this web page and add a request.

In the mean time, here is a procedure and script to get the figure I think you are looking for.

https://community.jmp.com/t5/Byron-Wingerd-s-Blog/One-Way-ANOVA-Figure-for-Scientists/ba-p/265156

Re: Choosing the right Comparison of means

statman — Tue, 23 May 2023 17:52:06 GMT

Bob, nice of you to take the time to answer the questions. I don't think this forum is an efficient way to discuss/argue the issues and appropriate analysis so I'll bow out.