topic Re: method comparison for censored data in Discussions

method comparison for censored data

david_arteta — Mon, 19 Jun 2017 08:19:04 GMT

Hi group,

I would like to compare thow analytical methods that are interval-censored (limit of quantitation - upper limit of quantitation). So I have observations with values of <3 (LOQ) as well as >120 (Upper LOQ). Usually Bland-Altman would do but in this case I am not sure.

Some people suggest for the lowest to substituting BQL observations with LOQ/2. But how about the upper LOQ?

There was discussion in the JMP Discovery Summit 2014 about censored data (https://community.jmp.com/kvoqx44227/attachments/kvoqx44227/discovery-2014-content/53/1/Discovery%20Summit%202014%20(Your%20Data%20Is%20Censored).pdf) but little is explained in this document.

Any suggestions as to how to deal with these data?

Thanks beforehand for any input

Dave

Re: method comparison for censored data

Mark_Bailey — Mon, 19 Jun 2017 11:23:10 GMT

Your data are not interval censored. Your observations that are < LOQ are left censored and your observations that are > LOQ are right censored. Your observations in between these limits are exact.

Do not use LOQ/2 for the left censored observations as this ad hoc substitution will bias your answers.

How do you "compare two analytical methods?"

Re: method comparison for censored data

david_arteta — Mon, 19 Jun 2017 11:58:27 GMT

Dear Mark,

thanks for your answer and for clarifying the data nature, I will keep that in mind.

Regarding your question, I am not sure I get it right. I have two quantitative methods to detect proteins and I want to compare them. If I am not wrong the most appropriate methods (or at least some that I have found on the literature) for comparing such measurements are Bland-Altman plots and Passing-Bablok regression. Is this correct? Any other method you can suggest?

I understand that substituting data introduces bias, but then how can I analyze data with "less than" or "greater than" results? I cannot just ignore it as it provides valuable information (mostly absence of protein but also large amounts of protein on the other hand)

Best,

David

Re: method comparison for censored data

Mark_Bailey — Mon, 19 Jun 2017 12:58:58 GMT

My question was simply to learn which methods you intended to use.

Unfortunately, most statistical methods do not recognize censoring. Most comparisons that I know (e.g., CLSI), therefore, omit data that is beyond the quantitative range of the assay. You might compare the detection capability and linearity of the two assays separately.

Re: method comparison for censored data

david_arteta — Tue, 20 Jun 2017 11:53:36 GMT

Dear Mark, thanks for your very useful input. I wonder if you had time to go through the document from the JMP Discovery Summit I refer to in mi original post. On page 13 and 14, José G Ramirez seems to be discussing on just a similar problem I am facing. And he performs what I think it is a Life Distribution analysis.

David

Re: method comparison for censored data

Mark_Bailey — Tue, 20 Jun 2017 12:57:56 GMT

Yes, I read Jose's paper before responding to your question. The only similarity between his data analysis and yours is that you both have censored data. The Life Distribution platform has the same purpose as that of the Distribution platform, but it is extended to provide different distribution models and results that are commonly expected from the analysis of time to event data. (Think survival or reliability analysis.) Would you use either Distribution or Life Distribution to model the distribution of the assays? Do you want to compare the distribution of values from the two assays?

Re: method comparison for censored data

david_arteta — Tue, 27 Jun 2017 08:37:33 GMT

Hi again Mark,

thanks for your useful comments. I understand the use of life distribution for survival analysis and my data are indeed different to those cases. It is just a matter of finding out whether there is a method that can take advantage of those values below and above LOQs rather than just discarding them. Survival analysis accomodates censored data and I wanted to find out whether there is something similar in method comparison analysis. To my understanding, Jose's problem on page 13 states that the question of interest is whether both analyzers perform the same, when there are several observations at the limit of detection of 0.05. This is not survival data, is it?.

From literature people use several different approaches to handle them. Some directly discard these values. Others transform them (as commented previously) to have something as an approximate value, which as you perfectly metioned may introduce some bias. But there is a trade-off there between introducing some bias on the edges rather than throwing away observations. You may have the situation of a clinical study where you have 20 patients and 6 or 7 of them have a measured concentration below LOQ for one of the analyzers but have 100% measurements within the range on the other. It is difficult to just throw them away.

David

Re: method comparison for censored data

Phil_Kay — Tue, 27 Jun 2017 08:47:30 GMT

Hi David,

It might be useful to know that the Generalized Regression platform in JMP Pro can handle censored data. You could use this to try comparing the means of the two protein measurement techniques, including the <LoQ and >LoQ data.

Regards,
Phil

Re: method comparison for censored data

david_arteta — Tue, 27 Jun 2017 09:52:46 GMT

Thanks Phil!

I have JMP 13, not JMP Pro, can this be achieved using Fit Model and selecting Generalized Linear Model? how would you enter these values in the database and define the column info?

Re: method comparison for censored data

Phil_Kay — Tue, 27 Jun 2017 10:00:14 GMT

Hi David,
No, this is not possible in JMP. In fact, I am not sure that it is possible to analysis data that is both left-censored and right-censored in Gen Reg in JMP Pro. You can definitely do one or the other. But not both. You can do interval censored in Gen Reg. But, as Mark says, that is not what you have.
I think you may need to think about a different solution.
Does it make sense to be using samples that are outside of LoQ for your method comparison? How does that help you to answer the question of "are my methods equivalent?"

Re: method comparison for censored data

david_arteta — Tue, 27 Jun 2017 10:32:52 GMT

Well, to me this is a new field of research and I am learning. But the first thing that came to my mind was about these special observations. Please have a look at the following table, representing concentration measurements in the same units from two different methods. they have different LOQs:

Method A	Method B
6.8	9.8
<0.1	<0.035
1.5	<0.035
5.4	7.2
9.9	9.1
5	6
2	2.9
6.9	13.9
<0.1	<0.035
<0.1	<0.035
<0.1	0.34
<0.1	<0.035
<0.1	<0.035
<0.1	0.49
<0.1	<0.035
<0.1	<0.035
4	1.9
6.7	3
6.8	5.2
0.3	<0.035
6.3	3.5
3.2	3
>10	>15
6.7	1.2
7.8	1.5
6.5	4.1
4	2.5
5.1	1.9
3.6	3.9
1.2	0.8

If I have to discard these observations, that will be ~30% of my dataset because that will have to be pairwise for a comparative analysis. But looking closely one can see that in most of the <LOQ observations, both methods seem to agree that there is a very low concentration. And the only observation that is above the upper LOQ, it is so for both methods. So actually this is very valuable information, qualitatively.

An initial thought can be, ok, so the observation is above 10 and 15 units respectively, so this observation has at least 10 units for one method and 15 units for the other. This is useful information, isn't it?. On the other hand, <0.035 cannot or should not be replaced by a 0.035, can it? As said before, some authors use LOQ/2 for these values, others, use 0, others discard the information.

Best,

David

Re: method comparison for censored data

Mark_Bailey — Tue, 27 Jun 2017 12:17:13 GMT

Unfortunately, Generalized Regression cannot deal with errors in both X and Y. Right now, only Bivariate can handle such a case using the Fit Orthogonal command. Method comparison involves a regression analysis of a new assay against a standard assay. Both assays have errors.