topic Re: Probit Analysis & Fit Testing in Discussions

Probit Analysis & Fit Testing

sassymenace — Tue, 20 Jun 2017 20:17:19 GMT

I am using the Simple Probit Analysis script add-in to determine LD50, LD90, and LD95.

Is this script capable of correcting with Abbott's, or how do you set this up? Currently, I corrected the data myself by applying the Abbott's correction to the raw data to adjust for mortalities.

After running the Probit Add-In, how do I determine goodness of fit? There are no Chi Squared results. Is there another way I can run this analysis to get my LD values, as well as getting the Chi Square?

Re: Probit Analysis & Fit Testing

Mark_Bailey — Tue, 20 Jun 2017 22:29:04 GMT

Did you contact the author of the add-in or read the description and instructions first?

JMP has several built-in platforms that can estimate such quantities and provide goodness of fit statistics. These leads should get you started.

Fit Curve can fit common non-linear functions.
Nonlinear is more general and it can fit custom models.
Generalized Linear Models can perform a probit analysis.

Did you search probit yet in Help or Help > Books > Fitting Linear Models?

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 14:20:35 GMT

I have not.

I have been attempting to run the model in the Generalized Linear with Probit, however, my LD values do not match up with what the Probit Add-In is giving me, so I'm not sure what is going on. I've read through the Fitting Linear models, but what I am trying to run analysis on is nonlinear, which I honestly don't have any experience with in JMP.

Re: Probit Analysis & Fit Testing

Mark_Bailey — Wed, 21 Jun 2017 14:36:13 GMT

This example is from our training course, JMP Software: Analyzing Discrete Responses.

We are testing seals for failures at increasing pressure levels. Here is the data table with the preferred layout:

Note that we do not enter the failure rate (Failed / Tested) for the response.

Select Analyze > Fit Model. Select Pressure and click Add. Select Failed and click Y. Select Total and click Y. (Note that (1) you must enter a column with the count of targets and another column with the count of opportunities and (2) the must be entered exactly in that order.) Click Standard Least Squares and select Generalized Linear Model. Select Binomial for the Distribution model. Select Probit for the Link function. Your launch dialog should look like:

Click Run.

You can see the non-linear response and the regression diagnostics below:

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 14:38:30 GMT

Okay,

It seems if I include my 0s (Control for Mortality) into the model, both the general linear probit model and the probit add-in match up the LD recommendations. If I use Abbott corrected data, then they have discrepancy between the two.

Should I run the model with the 0s and the raw data (no Abbott's adjustment), or is it more correct to adjust the raw data with Abbott's prior to inputing into the probit and disclude the 0s?

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 14:54:07 GMT

Abbott's Gen. Lin.pg 2Abbott's Gen. Lin. pg 1Abbott's ProbitRaw Data with 0s Gen. Lin. 1Raw Data with 0s Gen. Lin.2Probit Raw Data with 0s 1Probit Raw Data with 0s 2

Here are the two different results I'm getting with the different data sets. If I use the 0s and my raw data, both the probit model and the general linear probit model seem to match up. However, if I correct my data with Abbott's correction and leave out the 0s, then the two models don't match up.

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 15:09:34 GMT

I attached the wrong Linear models with raw data. Here are the correct: Gen. Lin. Raw Data 1Gen. Lin. Raw Data 2

Re: Probit Analysis & Fit Testing

Mark_Bailey — Wed, 21 Jun 2017 15:32:35 GMT

Abbott's correction is applied to mortality rates (i.e., proportions) to account for the natural mortality in the absence of the agent be tested. GLM is modeling a binomial distribution with a linear predictor. The GLM model has an intercept for that purpose (baseline mortality). So it is not necessary to apply this correction and doing so will, in fact, change the output. So don't do it if you are using GLM for your analysis.

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 15:38:55 GMT

So if I use the GLM analysis, do I include my 0s/Control Data and my initial raw data? And in doing so, is it correct to adjust a "0" to "0.005" for the log transformation? I am testing for mortality rates in the presence of an agent at various time points.

Re: Probit Analysis & Fit Testing

Mark_Bailey — Wed, 21 Jun 2017 16:04:15 GMT

I am confused as much as you are.

You should have two counts for your response for each level of the predictor: one count is the number that died and the other count is the number that might have died. You should have one predictor. It is usually concentration or log concentration. Is that true? Or is it time? Are you trying to estimate the time for LD50 at a fixed concentration?

Please use the data format exhibited in the example that I shared and not rates or proportions. Counts are necessary for the inference. A rate of 0.1 might result from a case of 1/10 or a case of 100/1000. The rate is the same in both cases but the sample size is very different.

Once again, if you enter the data as I showed and set up the GLM as I showed, you can forget about Abbott's correction for rates. You should enter both counts for time=0.

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 17:22:04 GMT

Here is an example of the data:

Dose (% Conc.)	Start (#live)	Mortality @ 15 min. (#dead)
100.00	13	13
100.00	10	10
100.00	11	11
100.00	13	13
25.00	10	9
25.00	23	17
25.00	15	9
25.00	9	6
10.00	14	3
10.00	10	1
10.00	10	4
10.00	16	7
1.64	11	1
1.64	12	3
1.64	13	4
1.64	24	4

I am trying to get LD50/LD90/LD95 results for dosage recommendations. I have this information for 5, 10, and 15 minute applications.

This is the data for my control:

Dose (% Conc.)	Start (#live)	Mortality (#dead)
0	10	1
0	11	6
0	10	3
0	15	4

Re: Probit Analysis & Fit Testing

Mark_Bailey — Wed, 21 Jun 2017 17:41:06 GMT

Your last comment about testing mortality rates over time changes everything, I think. My example is not relevant. My example is a case where a number of samples or subjects (20) is assigned to each factor level (5). Each level is an independent sample (5x20). The total number of samples or subjects is the sum of the samples at each level (100). I believe that you start with a fixed number of subjects (e.g. 60 total), the agent level is introduced at a fixed level, and you monitor the same group (subjects) over time. Yes?

Assuming I am correct, then you want to use repeated measures in survival analysis.

The methods and platforms in JMP were developed for reliability analysis, but it is the same thing. Just be warned that the words and labels are for reliability engineers and not biologists, if you know what I mean.

Set up your data to look like this example:

In this example, I have a total of 60 subjects that I monitor from the start of a 6 month program. I need two columns to represent the response (life) because I am not monitoring the subjects continuously but at monthly intervals. This process leads to interval censoring. I don't observe the exact life but instead I observe that it occurs during an interval.

The first row represents the start of the test. In this simulation, 2 subjects died at the start.
The second row represents the number of subjects who died between the start and the end of the first month (3).
The third row represents the number of subjects who died in the second month (3), and so on.
The last row represents the number of subjects who did not die by the end of the study (20).

I added another column, Status, to indicate the status of the observations but it is not necessary for the analysis. Perform the analyis by following these steps:

Select Analyze > Reliability and Survival > Life Distribution.
Select Beginning and Ending and click Y, Time to Event.
Select Count and click Freq.

The launch dialog should look like this:

(Note that the Censor analysis role is for right censored data. It is not needed when two response columns are used for interval censoring.)

Click OK.

The initial analysis is based on the non-parametric estimates of the failure probability from the Kaplan-Meier product limit. (You can click the red triangle at the top to change it from a Failure plot to a Survival plot.) You use the check boxes and radio buttons on the left side to scale the plot and fit a distribution model, respectively, to the data. Let's fit a normal distribution. Check the Normal check box and click the Normal radio button:

After selecting a model, use the Statistics outline to obtain the desired results. For example, your LD50 is available here as the quantile corresponding to a failure probability = 0.5, so estimate LD50 with the Quantile Profiler:

The LD50 in this example is 3.688803 months (3.22981 months to 4.1478 months with 95% confidence).

Re: Probit Analysis & Fit Testing

Mark_Bailey — Wed, 21 Jun 2017 17:57:24 GMT

OK, I think I understand now. I was correct to begin with and it is not a survival analysis (life time). We are back to my GLM example but now I will use your data. Here is your data in a JMP data table:

Set up the Fit Model launch dialog as before:

Click Run and then you get these results:

Click the red triangle next to Generalized Linear Model Fit and select Inverse Prediction. Type 0.5 (and any others that you want) for Probability (Died) and click OK.

So now we agree, no?

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 18:04:08 GMT

So I used my raw, unadjusted data without my controls and came up with this.

Do I interpret this as: LD90 for 100% Dose is 4.33 minutes? How do I determine the fit of this since there is no Chi Square? (I'm being asked to show a goodness of fit for the model I use.)

Do I count mortality as a "failure"?

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 18:11:01 GMT

Here's how I set it up and my data:

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 19:08:25 GMT

I'm not sure that is going to work. I'm getting resistence from others around me on running the analysis that way. They want the probit.

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 19:26:56 GMT

Here is the comparison of the same data in the probit add in vs the gen. linear probit. If I revert predicted log dose back, it does not match up with the same results in the general linear model.

Re: Probit Analysis & Fit Testing

Mark_Bailey — Wed, 21 Jun 2017 20:25:13 GMT

We already determined that you do not have life time or time to event data. You have a case of dose-response data. You collected observations at 15 minutes, so time is fixed. You counted the number of deaths and the total number subjects at each dose. This case is handled by GLM as I showed you. Please disregard the example based on life time that used Life Distribution. Survival analysis methods do not apply here. You should use a Probit analysis as you first started.

Re: Probit Analysis & Fit Testing

sassymenace — Wed, 21 Jun 2017 20:26:43 GMT

I figured it out. Probit Add-In needs live data and it calculates the dead counts itself. Generalize Linear takes the dead counts.

So if I run the GLM, do I include my controls/0 doses?

Re: Probit Analysis & Fit Testing

Mark_Bailey — Wed, 21 Jun 2017 20:33:59 GMT

I apologize (though it is not my fault) for the replies getting out of order. It does not help the confusion that already exists.

I have one question before I think we can conclude this discussion: why do you have four observations (rows) for each dose? What does each row represent?

Again, forget Life Distribution. Your colleagues are correct, you need a probit analysis. I am working now to make sure that your data table is set up correctly and the analysis is properly defined in Fit Model for a probit analysis. I want to use your previous example of data after you explain why there are four rows for every dose: