Hi JMP friends,
(On regular JMP 18.0.1, example data attached)
I am analyzing some data wherein I need to find the time (x point) at which my data has an "elbow" or inflection. I have an initial mini-peak in my data and have been doing this work manually. However, this is slow and I would like to automate it, as I will be doing this for hundreds of tests that return graphs similar to the below, with varying peak times/amounts and varying 'flat' times. The 'flat' times are generally not very flat, but rather very gradually decreasing. You can see in the attached image of the graph (Example/dummy data) what spot/area I would like to find.
Any help on what I could do would be really helpful. I think setting up a flag column would be a good way to find this, but am not sure of the JSL or formula I can use to have the flag work.
Any help on figuring this out would be great, thank you!
I approached finding the elbow as the data point, where the next 10 data points are all greater than the previous data point. This is absolutely a made up algorithm. But it looks like it might be a possible way to identify the elbow.
Here is the formula I created for the column I named Elbow
As Constant(
flag = 0;
lookAhead = 10;
);
val = .;
If( flag == 0 & Row() < N Rows( Current Data Table() ) - lookAhead,
If(
Min(
:Counts[Index( Row() + 1, Row() + lookAhead + 1 )] - :Counts[
Index( Row(), Row() + lookAhead )]
) > 0,
val = (flag = 1)
)
);
val;
Ok, so this solution is way-WAY more complicated.
Imagine if you fitted a function to the peak curve, and then calculated the second derivative?
It's easier to see graphically. In the figure, the first trace is the original data, a model for a fit, the first and second derivatives of the fit. It might not be easy to see, but the maximum of the second derivative (Blue line) is the middle of the elbow on the first line (triangle marker).
Here's how I got there with JSL. (Note: I changed the name of the first column)
In this script I let JMP sort out all the calculus along with the curve fit.
Names Default To Here( 1 );
dt=Open( "$Examples/Example for Elbow.jmp" );
obj=dt <<Fit Curve( Y( :Counts ), X( :time), Fit Skew Normal Peak );
obj << (Fit["Skew Normal Peak"] << Save Prediction Formula);
obj << Close Window;
formula=column(3)<<get formula;
derivative = Derivative(name expr(formula), Time);
second = Derivative(name expr(derivative), Time);
dt<<new column("First Derivative", formula (name expr(derivative)));
dt<<new column("Second Derivative", formula (name expr(second)));
dt<<new column("Peak Start", formula( Col Maximum( :Second Derivative ) == :Second Derivative, 1 ) );
dt<<Select where(:Peak Start == 1)<<Colors( "Red" )<<Markers( "Filled Up Triangle" );
The last step turns the "elbow" row into a red triangle marker. But you could just as easily subset the selected row into a table and add it to a table with all your other fits. I didn't try this with hundreds of tables but I think it would execute fairly quickly if you put it in a loop to iterate through a lot of data tables.
