BookmarkSubscribeRSS Feed
Choose Language Hide Translation Bar
JD23

Not applicable

Joined:

Apr 3, 2019

How to script calibration curve

Hi, my level of scripting is really basic and I hope you can help me.

How do I write a script to achieve the following?

I have a table with the known concentrations of the calibrators. In each assay (one table per assay) I have the signal of several unknowns as well as the signal for the calibrators, it is basically an internal calibration curve for each assay.

Is it possible to create a script that takes the values from the table of the known concentrations, plots them against their signals for that specific assay (assay table) and then uses the linear regression to calculate the concentration of the unknowns?

Thanks in advance for the help!

2 ACCEPTED SOLUTIONS

Accepted Solutions
markbailey

Staff

Joined:

Jun 23, 2011

Solution

Re: How to script calibration curve

OK, thanks for clarifying.

 

Names Default to Here( 1 );

// check for sufficient tables
n tables = N Table();

If( n tables < 2,
	Throw( "Two data tables are required for this analysis" )
);

tables = List();
For( t = 1, t <= n tables, t++,
	Insert Into( tables, Eval( Data Table( t ) << Get Name ) );
);

// identify the calibration and unknown data tables
dlg = New Window( "Assay Unknown Samples", << Modal,
	H List Box(
		lb = List Box( tables ),
		Line Up Box( N Col( 2 ),
			Button Box( "Calibrator Table",
				sel = lb << Get Selected;
				cal lb << Append( sel );
			),
			cal lb = List Box( {},
				N Lines( 1 )
			),
			Button Box( "Unknown Table",
				sel = lb << Get Selected;
				unk lb << Append( sel );
			),
			unk lb = List Box( {},
				N Lines( 1 )
			)
		)
	),
	H List Box(
		Button Box( "OK",
			cal table = cal lb << Get Items;
			Show( cal table );
			unk table = unk lb << Get Items;
			Show( unk table );
		),
		Button Box( "Cancel" )
	)
);

If( dlg["Button"] == -1,
	Throw( "User cancelled" )
);

// unload the dialog
cal table = Data Table( cal table[1] );
unk table = Data Table( unk table[1] );

// include AU 2 values for calibrators from calibration table
combined = cal table << Join(
	With( unk table ),
	Copy formula( 0 ),
	Select With( :Sample, :Peps, :AU 1 ),
	Select( :AU 2 ),
	By Matching Columns( :Peps = :Peps ),
	Drop Multiples( 0, 0 ),
	Include Nonmatches( 0, 1 ),
	Preserve Main Table Order( 0 )
);
Close( cal table, No Save );
Close( unk table, No Save );

// distinquish peps
Current Data Table( combined );
combined << New Column( "Status", "Character", "Nominal" );
For Each Row(
	:Status = If( 1 <= Num( Regex( :Peps, "\w{3}(\d+)", "\1" ) ) <= 29, "Calibrator", "Unknown" );
);


// fit the calibration curve
dlg = combined << Fit Model(
	Y( :AU 1 ),
	Effects( :AU 2 ),
	Personality( "Standard Least Squares" ),
	Emphasis( "Minimal Report" ),
	Where( :Status == "Calibrator" )
);
fit = dlg << Run;

// determine the assay for the unknown samples from the calibration curve
For Each Row(
	fit << Inverse Prediction( Response( :AU 1[] ) );
);

// extract the assays into a new data table
assays = Report( fit )["Inverse Prediction"][TableBox(1)] << Make Combined Data Table;
fit << Close Window;
assays << Delete Columns( Column( assays, 1 ) );

// add the assays to the unknown table
final = combined << Join(
	With( assays ),
	Copy formula( 0 ),
	By Matching Columns( :AU 1 = :Specified AU 1 ),
	Drop Multiples( 0, 0 ),
	Include Nonmatches( 1, 0 ),
	Preserve Main Table Order( 1 )
);
Close( combined, No Save );
Close( assays, No Save );

final << Sort(
	By( :Peps, :Sample ),
	Order( Ascending, Ascending ),
	Replace Table
);

 

Some of the predicted AU 2 values are missing due to the random measurements of AU 1.

Learn it once, use it forever!
markbailey

Staff

Joined:

Jun 23, 2011

Solution

Re: How to script calibration curve

Names Default to Here( 1 );

// check for sufficient tables
n tables = N Table();

If( n tables < 2,
	Throw( "Two data tables are required for this analysis" )
);

tables = List();
For( t = 1, t <= n tables, t++,
	Insert Into( tables, Eval( Data Table( t ) << Get Name ) );
);

// identify the calibration and unknown data tables
dlg = New Window( "Assay Unknown Samples", << Modal,
	H List Box(
		lb = List Box( tables ),
		Line Up Box( N Col( 2 ),
			Button Box( "Calibrator Table",
				sel = lb << Get Selected;
				cal lb << Append( sel );
			),
			cal lb = List Box( {},
				N Lines( 1 )
			),
			Button Box( "Unknown Table",
				sel = lb << Get Selected;
				unk lb << Append( sel );
			),
			unk lb = List Box( {},
				N Lines( 1 )
			)
		)
	),
	H List Box(
		Button Box( "OK",
			cal table = cal lb << Get Items;
			Show( cal table );
			unk table = unk lb << Get Items;
			Show( unk table );
		),
		Button Box( "Cancel" )
	)
);

If( dlg["Button"] == -1,
	Throw( "User cancelled" )
);

// unload the dialog
cal table = Data Table( cal table[1] );
unk table = Data Table( unk table[1] );

// include AU 2 values for calibrators from calibration table
combined = cal table << Join( With( unk table ),
	Copy formula( 0 ),
	Select With( :Sample, :Peps, :AU 1 ),
	Select( :AU 2 ),
	By Matching Columns( :Peps = :Peps ),
	Drop Multiples( 0, 0 ),
	Include Nonmatches( 0, 1 ),
	Preserve Main Table Order( 0 )
);
Close( cal table, No Save );
Close( unk table, No Save );

// distinquish peps
Current Data Table( combined );
combined << New Column( "Status", "Character", "Nominal" );
For Each Row(
	:Status = If( 1 <= Num( Regex( :Peps, "\w{3}(\d+)", "\1" ) ) <= 29, "Calibrator", "Unknown" );
);

// placeholders for matching information
combined
	<< New Column( "Predicted AU 2", "Numeric", "Continuous" )
	<< New Column( "Lower 95%", "Numeric", "Continuous" )
	<< New Column( "Upper 95%", "Numeric", "Continuous" );

// handle each sample set separately
For( s = 1, s <= 3, s++,
	// isolate sample
	ss = Char( s );
	combined
		<< Select Where( :Sample == ss )
		<< Invert Row Selection
		<< Exclude;

	// fit the calibration curve for this sample
	dlg = combined << Fit Model(
		Y( :AU 1 ),
		Effects( :AU 2 ),
		Personality( "Standard Least Squares" ),
		Emphasis( "Minimal Report" ),
		Where( :Status == "Calibrator" )
	);
	fit = dlg << Run;

	// determine the assay for the unknown samples from the calibration curve
	For Each Row(
		If( Not( Excluded( Row State( Row() ) ) ),
			fit << Inverse Prediction( Response( :AU 1[] ) );
		);
	);

	// access report layer
	fit rep = fit << Report;

	// remove extraneous string column box that interferes with making combined data table
	(fit rep << XPath( "//StringColBox[StringColBoxHeader=\!"Type of CI\!"]")) << Delete;

	// extract the assays into a new data table
	assays = fit rep["Inverse Prediction"][TableBox(1)] << Make Combined Data Table;
	fit << Close Window;

	// prepare predicted values for join
	assays
		<< Delete Columns( Column( assays, 1 ) )
		<< New Column( "Sample", "Character", "NominaL" )
		<< New Column( "Peps", "Character", "NominaL" );
	Current Data Table( combined );
	r = 1;
	For Each Row(
		If( Not( Excluded( Row State( Row() ) ) ),
			assays:Sample[r] = combined:Sample;
			assays:Peps[r] = combined:Peps;
			r++;
		);
	);
	combined << Clear Row States;
	combined << Update(
		With( assays ),
		Match Columns(
			:Sample = :Sample,
			:Peps = :Peps
		)
	);
	Close( assays, No Save );
);
Learn it once, use it forever!
24 REPLIES 24
markbailey

Staff

Joined:

Jun 23, 2011

Re: How to script calibration curve

I suggest that you first learn how to perform this analysis interactively using JMP. See Help > Books > Predictive and Specialized Modeling > chapter about Fit Curve. It will explain how to use this platform for your purpose. It includes examples to guide you and help you practice before you attempt to analyze your own data.

 

What model do you currently use for your calibration curve? You mention linear regression. If this is the case, then click the red triangle next to Fit Curve and select Polynomials > Fit Linear. Now click the red triangle next to Linear and select Custom Inverse Prediction. A dialog appears in which you can enter the unknown responses.

 

Use the Fit Curve platform interactively first to be sure that this analysis is appropriate and yields the results you seek. Then we can help with a script.

Learn it once, use it forever!
JD23

Not applicable

Joined:

Apr 3, 2019

Re: How to script calibration curve

Thanks Mark.

I have followed the steps you mention to get the custom inverse prediction. Yes, it is a linear regression.

My problem with that is the limited number of unknows I can enter. The scripting question was asked having in mind that I have thousands of unknowns and I assumed that scripting the procedure would allow me to values for all those unknowns. 

markbailey

Staff

Joined:

Jun 23, 2011

Re: How to script calibration curve

Thanks for verifying the analysis method and process. Onward to the script!

 

Can you provide a sample of data? You can use generic names (anonymous data). That way I will understand what the script should expect from the data table.

Learn it once, use it forever!
JD23

Not applicable

Joined:

Apr 3, 2019

Re: How to script calibration curve

Here are two example tables. One is the calibrator table. The other table is the one that has the measured values (AU 1) and needs calibration to obtain the unknown values (AU 2). To complicate things a bit, this table has the measurements for 3 different samples and each of these 3 samples also contained the components that were used to make the calibrator. So in each sample we have an internal calibrator, sort of speak. 

It would be great to know how to script a calibration that completely omits the fact that we have an internal calibrator (plain simple linear regression of what is on the calibrator table and use the expression to obtain the unknowns from samples table). And it would be even better to know how to script a calibration that uses the measured values (AU 1) of the calibrator components in each sample (samples table), plots it against the known values (AU 2) in the calibrator table and calculates the unknowns for that specific sample, and does the same for each sample present in the samples table. 

English is not my native language so please let me know if you need me to be a bit clearer. 

I hope you can help me.

Cheers.

markbailey

Staff

Joined:

Jun 23, 2011

Re: How to script calibration curve

I think I understand the basic problem but I might have missed a detail. Try this script with both data tables open (calibration table and unknow table):

 

Names Default to Here( 1 );

// find all open data tables
tables = Get Data Table List();
If( N Items( tables ) < 2,
	Throw( "Two data tables are required for this analysis" )
);

dlg = New Window( "Assay Unknown Samples", << Modal,
	H List Box(
		lb = List Box( tables ),
		Line Up Box( N Col( 2 ),
			Button Box( "Calibrator Table",
				sel = lb << Get Selected;
				cal lb << Append( sel );
			),
			cal lb = List Box( {},
				N Lines( 1 )
			),
			Button Box( "Unknown Table",
				sel = lb << Get Selected;
				unk lb << Append( sel );
			),
			unk lb = List Box( {},
				N Lines( 1 )
			)
		)
	),
	H List Box(
		Button Box( "OK",
			cal table = cal lb << Get Items;
			Show( cal table );
			unk table = unk lb << Get Items;
			Show( unk table );
		),
		Button Box( "Cancel" )
	)
);

If( dlg["Button"] == -1,
	Throw( "User cancelled" )
);

// unload dialog
cal table = Data Table( cal table[1] );
unk table = Data Table( unk table[1] );

// perform the linear calibration
dlg = cal table << Fit Model(
	Y( :AU 1 ),
	Effects( :AU 2 ),
	Personality( "Standard Least Squares" ),
	Emphasis( "Minimal Report" )
);
fit = dlg << Run;

// perform inverse prediction on unknown
Current Data Table( unk table );
For Each Row(
	fit << Inverse Prediction( Response( :AU 1[] ) );
);

// unload unknown assays
assays = Report( fit )["Inverse Prediction"][TableBox(1)] << Make Combined Data Table;
fit << Close Window;
assays << Delete Columns( Column( assays, 1 ) );

// join with unknown table
unk table << Join(
	With( assays ),
	By Row Number
);
Close( assays, No Save );

// plot predictions with 95% confidence intervals versus actual
unk table << Graph Builder(
	Size( 534, 454 ),
	Show Control Panel( 0 ),
	Variables(
		X( :AU 1 ),
		Y( :Lower 95% ),
		Y( :Upper 95%, Position( 1 ) ),
		Y( :Predicted AU 2, Position( 1 ) )
	),
	Elements(
		Bar( X, Y( 1 ), Y( 2 ), Y( 3 ), Legend( 8 ), Bar Style( "Interval" ) )
	),
	SendToReport(
		Dispatch( {}, "AU 1", ScaleBox, {Format( "Best", 12 )} ),
		Dispatch( {}, "Lower 95%", ScaleBox, {Format( "Fixed Dec", 12, 0 )} )
	)
);

 

 

The script is not flexible but it is a start.

Learn it once, use it forever!
JD23

Not applicable

Joined:

Apr 3, 2019

Re: How to script calibration curve

Maybe it's a JMP version issue (I have JMP 13.1) but the script does not work. I used the JSL Debugger and I get an error right at the start: Name Unresolved: Get Data Table List in access or evaluation of 'Get Data Table List' , Get Data Table List() /*###*/

markbailey

Staff

Joined:

Jun 23, 2011

Re: How to script calibration curve

It is a version issue. I opened JMP 13 and (1) the function was no longer highlighted as such and (2) it is not found in the Scripting Index.

 

I hate when that happens!

 

Try this work-around:

 

Names Default to Here( 1 );

// find all open data tables

/* for JMP 14 and later:
tables = Get Data Table List();
*/

n tables = N Table();

If( n tables < 2,
	Throw( "Two data tables are required for this analysis" )
);

tables = List();
For( t = 1, t <= n tables, t++,
	Insert Into( tables, Eval( Data Table( t ) << Get Name ) );
);

// present open tables for assignment as calibration or unknown.

dlg = New Window( "Assay Unknown Samples", << Modal,
	H List Box(
		lb = List Box( tables ),
		Line Up Box( N Col( 2 ),
			Button Box( "Calibrator Table",
				sel = lb << Get Selected;
				cal lb << Append( sel );
			),
			cal lb = List Box( {},
				N Lines( 1 )
			),
			Button Box( "Unknown Table",
				sel = lb << Get Selected;
				unk lb << Append( sel );
			),
			unk lb = List Box( {},
				N Lines( 1 )
			)
		)
	),
	H List Box(
		Button Box( "OK",
			cal table = cal lb << Get Items;
			Show( cal table );
			unk table = unk lb << Get Items;
			Show( unk table );
		),
		Button Box( "Cancel" )
	)
);

If( dlg["Button"] == -1,
	Throw( "User cancelled" )
);

// unload dialog
cal table = Data Table( cal table[1] );
unk table = Data Table( unk table[1] );

// perform the linear calibration
dlg = cal table << Fit Model(
	Y( :AU 1 ),
	Effects( :AU 2 ),
	Personality( "Standard Least Squares" ),
	Emphasis( "Minimal Report" )
);
fit = dlg << Run;

// perform inverse prediction on unknown
Current Data Table( unk table );
For Each Row(
	fit << Inverse Prediction( Response( :AU 1[] ) );
);

// unload unknown assays
assays = Report( fit )["Inverse Prediction"][TableBox(1)] << Make Combined Data Table;
fit << Close Window;
assays << Delete Columns( Column( assays, 1 ) );

// join with unknown table
unk table << Join(
	With( assays ),
	By Row Number
);
Close( assays, No Save );

// plot predictions with 95% confidence intervals versus actual
unk table << Graph Builder(
	Size( 534, 454 ),
	Show Control Panel( 0 ),
	Variables(
		X( :AU 1 ),
		Y( :Lower 95% ),
		Y( :Upper 95%, Position( 1 ) ),
		Y( :Predicted AU 2, Position( 1 ) )
	),
	Elements(
		Bar( X, Y( 1 ), Y( 2 ), Y( 3 ), Legend( 8 ), Bar Style( "Interval" ) )
	),
	SendToReport(
		Dispatch( {}, "AU 1", ScaleBox, {Format( "Best", 12 )} ),
		Dispatch( {}, "Lower 95%", ScaleBox, {Format( "Fixed Dec", 12, 0 )} )
	)
);
Learn it once, use it forever!
JD23

Not applicable

Joined:

Apr 3, 2019

Re: How to script calibration curve

Thanks Mark, it worked now. 

From what I understood, looking at the script, this is for the straightforward calibration using AU 1 and AU 2 values from the calibrator table, yes?

How can we rearrange it so that the calibrator is the AU 1 values specified in the calibrator table vs the AU 2 from the unknown table (making it a different calibrator for each sample) and then predict the AU 2 values for the rest of the components in the sample?

Highlighted
markbailey

Staff

Joined:

Jun 23, 2011

Re: How to script calibration curve

I am confused.

 

First of all, did it work or not? Did it work up to a point?

 

Second, yes, the calibration is performed entirely with the AU 1 and AU 2 values in the calibration table. Is that not correct?

 

Third, you don't have AU 2 values in the unknown table. Do you mean to use the predicted unknown AU 2 values?

 

Fourth, is this a change to the calibration definition or is it a new requirement (follow on step) after predicting the AU 2 in the unknown table?

 

Fifth, what is Sample? How does it relate to Peps?

 

Sixth, shouldn't the AU 1 in the calibrator table for Pep1 to Pep29 be correlated? They are not:

 

Screen Shot 2019-04-13 at 4.05.27 PM.png

 

I am probably missing something very important and very basic.

Learn it once, use it forever!