cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Browse apps to extend the software in the new JMP Marketplace
Choose Language Hide Translation Bar
Gain accuracy and precision for model estimates from limits of detection (LOD) control

Researchers commonly use complex laboratory methods to evaluate critical quality attributes (CQA) of drugs in development and throughout the product lifecycle. Instruments have validated ranges for obtaining data, and researchers often get results that are beyond the limits of detection (LOD). If you're a technical person running predictive models that include responses with limits on the range of values that can be obtained, this post is for you. JMP Pro now includes an easily deployed technique to appropriately deal with limits of detection within statistical models and reduce the risk for error.

What is the practical value of utilizing the new LOD controls in JMP Pro?

Let's consider an example: We are planning a set of experiments to determine the maximum amount of a solid impurity that can be extracted and measured with a laboratory method utilizing three inputs. The measurement method includes a lower detection limit of 1 unit. We've tried different techniques for dealing with LOD in the past, including using missing values, creating a value that is very small but not zero to allow for a logarithm to be used, imposing the limit for the observations affected by the LOD, and using the LOD control included in JMP Pro 16.

statpharmer_14-1623784535010.png

We use a generalized linear model with a log normal distribution utilizing forward selection to model the amount of impurity that can be extracted and measured. Running the model without LOD control results in predictions that are highly suspect to include significant error.

The maximized model using missing values provides an estimate of 137.8, which is hugely beyond the highest impurity amount of 34.7 realized in the experimentation. Note the extremely wide spread of the 95% confidence interval limits.

statpharmer_1-1623784443792.png

The maximized model using values that are close to zero provides the unrealistic estimate of 472.4.

statpharmer_2-1623784443796.png

The maximized model using the 1-unit low limit seems to underestimate the optimum amount of impurity at 16.6, which is roughly half of the maximum amount realized during the experimentation.

statpharmer_3-1623784443798.png

We use LOD controls in the final model. The optimum amount of impurity that can be measured is estimated to be 36.7. Note that the realistic estimate comes from a model that includes all three method inputs. There are local maxima for each of the three method inputs, which makes sense to subject matter experts.

statpharmer_4-1623784443801.png

The comparison clearly illustrates how LOD controls mitigate error in prediction estimates for models. It is always a best practice to create confirmation runs in the space surrounding the amounts for the inputs identified as optimum.

What are limits of detection?

Instruments typically produce values from a matrix without an analyte, which are known as blank readings. A group of blank readings form a distribution of “noise” that is inherent with the measurement system. This “noise” is the random variability that you can expect from taking multiple readings of a homogenous sample tested by the instrument.

statpharmer_5-1623784443802.png

The limit of detection is the smallest concentration/quantity that is statistically different than a blank reading[1].

Smallest quantity = mean of blank measures + (confidence level x standard deviation of blank measures)

Using a 99% level of confidence, the limit of detection is the mean of the smallest signal that is roughly 3 standard deviations (blank) above the mean of the blanks.

statpharmer_6-1623784443805.png

A limit of quantification is often utilized to mitigate a false negative result (that is, we assume the measure is outside of the limit when an actual measure is available).

statpharmer_7-1623784443809.png

Some measurements have bi-lateral limits. Load cells that measure force typically have a zone of accuracy within bi-lateral limits.

statpharmer_8-1623784443814.png

Scientists and engineers deal with many different types of limits of detection including: Instrument detection limit, method detection limit, limit of quantification, and practical quantification limit. Regardless of the type of limit used, it is up to the scientist or engineer to identify the cut-off values for censoring with LOD to meet the goals of experimentation.

Why do limits of detection affect model estimates?

Parameter estimates are typically made with the cumulative distribution function (CDF). With limits present, the result is a shifted CDF and altered parameters. The plot below illustrates the effect of an LOD for a symmetric distribution.

statpharmer_9-1623784443815.png

The shift of the CDF due to limits within measurements is especially profound for skewed distributions, which is seen below as a significant amount of potential data that is not included in the CDF that creates a big shift in parameters.

statpharmer_10-1623784443817.png

When we use limits of detection appropriately as censor values, estimates are made from the probability density function. The shape of the PDF mitigates the shift of the parameters since the missing portion can be   Parameters are more accurate yielding realistic model estimates.

statpharmer_11-1623784443819.png

How can you utilize LOD controls in JMP Pro?

Let's consider a similar scenario: We are planning a set experiments to find out the maximum amount of a solid impurity that we can extract and measure using a laboratory method. We use the DOE> Custom Design menu option to set up the structure of the experiments. JMP 16 includes the ability to add detection limits as part of the design process as shown below.  

statpharmer_12-1623784443821.png

Another way to set up LOD controls is to add them to the column properties.

statpharmer_13-1623784443825.png

JMP Pro will automatically control for LOD without you needing to create additional columns required to censor appropriately for the limits within models.

Realize practical value by utilizing LOD controls

Use of this LOD controls in JMP Pro 16 yields significant improvements in the precision and accuracy of model estimates to enhance quality by design (QbD) efforts. Technical people now have a great new tool to deal with measurement methods that are known to have limits.

Reference

[1] IUPAC. Compendium of Chemical Terminology, 2nd ed. (the "Gold Book"). Compiled by A. D. McNaught and A. Wilkinson. Blackwell Scientific Publications, Oxford (1997). Online version (2019-) created by S. J. Chalk. ISBN 0-9678550-9-8. https://doi.org/10.1351/goldbook.

 

Last Modified: Dec 19, 2023 4:00 PM
Comments
statpharmer
Staff

A JMP Public post with the data and analyses shown in the blog can be accessed at the link shown.https://public.jmp.com/packages/bJ451snNNdFwWC11kr8fx 

gwenhallberg
Level III

Nice article, Rob!  This will certainly be a useful feature for us where low-level impurities are concerned.  I have a coworker asking how you made the graphic to demonstrate bi-lateral limits.  I assume JSL was involved.  Any chance you can share that script?  Thanks!

gail_massari
Community Manager

Hi Gwen @gwenhallberg , to make sure Rob gets this (and to make sure anyone you want to see any comment/question you add sees it fast), please use the @mention tag.  In this case, Rob @statpharmer , Gwen wrote:

 

Nice article, Rob!  This will certainly be a useful feature for us where low-level impurities are concerned.  I have a coworker asking how you made the graphic to demonstrate bi-lateral limits.  I assume JSL was involved.  Any chance you can share that script?  Thanks!

navy
Level I

@statpharmer as microbiologist, counts less than LOD is very common. If LOD=10, I used to adopt 9 to replace <10 and it can be transformed with log calculation. So may I ask you to try your data with 0.9 to replace 1 and check what could be the final difference in prediction profile from other scenario you anlayzed, i.e. miss, extreamly low to 0, 1, LOD? I am only a user of JMP v15 and could not try. thank you. 

navy
Level I

@statpharmer to be honest, I love this function but just feel sad that it's part of JMP Pro instead of JMP. 

statpharmer
Staff

When I plug in 0.9 instead of 1, with all else the same, the profiler continues to underestimate the optimum result.

statpharmer_0-1626715998432.png

navy
Level I

 @statpharmer thank you for sharing the result. Just wonder if you could share the raw data for me to play with this function in my colleague's JMP Pro and may justify my application for upgrading. I am also interested in the lognormal distribution. How JMP is processing those zero indicating LOD? thank you. 

statpharmer
Staff

I posted the impurity dataset to the article to allow for people to play with the concepts noted.