Integrate(), Interpolate(), and SplineEval()

[#1]

Combine Integrate with Interpolate or SplineEval to get the area under the curve.

@sseligman  wrote Using JMP to Find the Area Under a Curve ; here's a little more information on some builtin functions.

The Interpolate function uses a sorted list of X coordinates and a corresponding list of Y coordinates and answers the question "if X was 42, what would Y be?" using linear interpolation between the nearest two X coordinates in the sorted list. Linear interpolation produces straight line segments with data points on the corners of the curve.

The SplineEval function does the same thing (with help from the Spline Coef function) using a spline interpolation. A spline is a flexible, springy metal wire that passes close to the data points, depending on how stiff it is. Usually data points do not fall on the spline curve.

Both of those functions produce all the data points in a curve from the first X to the last X.

The Integrate function answers a different question: "how much area is under this curve in this range?" Interpolate has four arguments; the first two describe a f(x) function that generates a curve and the last two describe a range of x values. The result of Integrate( f(x), x, low, high ) is the area under the curve as x goes from low to high. The first two variables seem to repeat x because f(x) might be an expression like a+2*b and the second variable tells if a or b is the variable between low and high.

The JSL for this post is about using Interpolate or SplineEval as Interpolate's f(x) value.  It builds an interactive graph to help understand how finding the area under a curve defined by some noisy data points might work with linear or spline interpolation. It also shows the exact answer (red) before the noise (black data points) was added to the data. The blue curve is the spline fit to the black data points. The bar graph in the middle shows the three area computations side-by-side; only the middle blue bar moves when the spline tension changes. With the right random data, any bar can be taller or shorter than the others.

JSL makes a graph like this; the lambda slider changes the blue spline's tension.

JMP 16 required...for each...

New Window( "Integrate Area Under Curve", 
	// make a function, in the window: namespace, that returns a sine wave
	window:f = Function( {t},
		50 + Sin( t / 10 ) * 40
	);
	
	// create the x dimension of the graph
	window:xmin = 20; // integrate over this range
	window:xmax = 80;
	window:xcoarse = window:xmin :: window:xmax :: 4; // the coarse data is used to make random data from sin function
	window:xfine = window:xmin :: window:xmax :: 0.125; // the fine data is used for outlining smooth polygons
	
	// create the y dimension of the graph from x and function f(x) and some randomness
	window:random = 10; // random variation, slider below
	// this function called here, and from slider below
	window:generatedata = Function( {}, // regenerate the points with new random error
		window:ycoarse = window:f( window:xcoarse ) + J( 1, N Col( window:xcoarse ), Random Normal( 0, window:random ) );
		// ycoarse is the y data for the points; limit it to stay on the y axis...
		window:ycoarse[Loc( window:ycoarse > 99 )] = 99;
		window:ycoarse[Loc( window:ycoarse < 1 )] = 1;
	);
	window:generatedata(); // prepare the initial data in the graph
	
	// lambda is the tension on the spline interpolation, slider below
	window:loglambda = 2;// spline tension, a nice middle value
	
	// random patterns for filled areas. the exponents are artistically adjusted for blending the colors
	// with gray drawn first, and dark, and yellow drawn last, medium, and blue between, but light.
	// there are 7 overlaps with different shades.
	window:patternsize = 299; // big square patch for a pattern
	window:pat01 = J( window:patternsize, window:patternsize, (Random Integer( 0, 255 ) / 255) ^ 0.5 );//gray
	window:pat02 = J( window:patternsize, window:patternsize, (Random Integer( 0, 255 ) / 255) ^ 3 );//blue
	window:pat03 = J( window:patternsize, window:patternsize, (Random Integer( 0, 255 ) / 255) ^ 2 );//yellow
	window:bar01 = 38;
	window:bar02 = 44;// bar x-locations, in graph coordinates; they look good here, for this sine wave
	window:bar03 = 50;
	window:show = [1 1 1];
	window:barwide = 5;
	window:barbottom = 50;// bottom of chopped off bar
	window:bartop = window:barbottom + 5;

	// map values of the AUC onto the graph
	window:aucMin = 1500;
	window:aucMax = 4000;
	window:aucToGraph = Function( {y}, // convert auc to graph Y to make bar heights; 0 is for a label
		Interpolate( y, 0, window:barbottom - window:bartop, window:aucMin, 7, window:aucMax, 97 - window:bartop )
	);
	
	// draw a broken bar; x in graph units, y in auc units
	window:drawAucBar = Function( {x, y},
		y = window:aucToGraph( y );
		Polygon(// ragged edge bottom poly below the break
			x || x || x + .1 * window:barwide || x + .3 * window:barwide || x + .5 * window:barwide || x + .8 * window:barwide || x + window:barwide
			 || x + window:barwide,
			window:barbottom || window:barbottom + 5 || window:barbottom + 6 || window:barbottom + 4 || window:barbottom + 7 || window:barbottom + 3
			 || window:barbottom + 5 || window:barbottom
		);
		If( y >= 0, // the bar is below the break, don't draw
			Polygon(
				x || x || x + .1 * window:barwide || x + .3 * window:barwide || x + .5 * window:barwide || x + .8 * window:barwide || x + window
				:barwide || x + window:barwide,
				window:bartop + y || window:bartop + 5 || window:bartop + 6 || window:bartop + 4 || window:bartop + 7 || window:bartop + 3 || window
				:bartop + 5 || window:bartop + y
			)
		);
	);
	
	// make the GUI
	window:g = Graph Box(
		// this script runs when the graph needs to reshow, either the first time, or because a slider script uses <<reshow
		framesize( 1200, 600 ), // size in pixels
		X Scale( window:xmin - 5, window:xmax + 5 ), // a little bigger than the data
		
		// generate the spline parameters from the data and lambda. spline data is made from these parms several times.
		window:splineparms = Spline Coef( window:xcoarse, window:ycoarse, Power( 10, window:loglambda ) );
		
		// capture the AUC. interpolate is a simple linear interpolation and works well with integrate
		window:auc01 = Integrate( Interpolate( t, window:xcoarse, window:ycoarse ), t, window:xmin, window:xmax );
		// capture the AUC. SplineEval uses Spline Coef (window.m) to produce a smooth interpolation
		window:auc02 = Integrate( Spline Eval( t, window:splineparms, 0 ), t, window:xmin, window:xmax );
		// integrate uses f(x)
		window:auc03 = Integrate( window:f( t ), t, window:xmin, window:xmax );

		// use the three auc values to adjust grid for the bar chart to include a 500 label on the small and large end
		window:aucMin = 500 * Floor( (Min( window:auc02, window:auc03, window:auc01 )) / 500 );
		window:aucMax = 500 * Ceiling( (Max( window:auc02, window:auc03, window:auc01 )) / 500 );
		
		// draw the filled areas under the curves and the bars
		
		// the gray polygon is the area of the polygon formed from the random points, connected with straight lines
		Fill Color( "gray" );
		Pen Color( "black" );
		Pen Size( 1 );
		window:poly = 0 || window:ycoarse || 0;// prepend and append extra points for 
		window:polx = window:xmin || window:xcoarse || window:xmax;// drawing a polygon
		Fill Pattern( window:pat01 );// use different random patterns for the three overlays
		If( window:show[1],
			Polygon( window:polx, window:poly );
			Line( window:polx, window:poly );// this one needs an outline
		);
		window:drawAucBar( window:bar01, window:auc01 );// add bar to bar graph
		
		// the blue poly just like above, but for spline
		Fill Color( "blue" );
		window:poly = 0 || Spline Eval( window:xfine, window:splineparms, 0 ) || 0;
		window:polx = window:xmin || window:xfine || window:xmax;
		Fill Pattern( window:pat02 );
		If( window:show[2],
			Polygon( window:polx, window:poly )
		);
		window:drawAucBar( window:bar02, window:auc02 );
		
		// the yellow poly with the red outline is the sin function, f(), without random noise
		Fill Color( "yellow" );// would be red, but yellow and blue mix better
		window:poly = 0 || window:f( window:xfine ) || 0;
		window:polx = window:xmin || window:xfine || window:xmax;
		Fill Pattern( window:pat03 );
		If( window:show[3],
			Polygon( window:polx, window:poly )
		);
		window:drawAucBar( window:bar03, window:auc03 );
		
		// draw grid on top of bars
		For Each( {y}, 0 || (window:aucMin :: window:aucMax :: 100), // grid lines.
			yy = window:bartop + window:aucToGraph( y );
			H Line( 37, 56, yy );
			If( 0 == Mod( y, 500 ),
				Text( {56.3, yy - 1}, Char( y ) )// label grid lines
			);
		);

		Pen Size( 3 );// outlines red and blue are thick
		
		Pen Color( "black" );
		Text Color( "black" );
		Text( {window:bar01 + .5, 44}, "AUC linear\!n" || Char( window:auc01, 10, 2 ) );
		
		Pen Color( "blue" );
		Text Color( "blue" );
		Y Function( Spline Eval( t, window:splineparms, 0 ), t );// thick outline
		Text( {window:bar02 + .5, 44}, "AUC spline\!n" || Char( window:auc02, 10, 2 ) );

		Pen Color( "red" );
		Text Color( "red" );
		Y Function( window:f( a ), a );// thick outline
		Text( {window:bar03 + .5, 44}, "AUC sine\!n" || Char( window:auc03, 10, 2 ) );
		
		Fill Pattern();// turn OFF before drawing points!
		Marker( Combine States( Color State( "black" ), Marker State( 12 ) ), window:xcoarse, window:ycoarse );
	);//
,
	H List Box(// add controls below graph
		Text Box( "Lambda: " ), // lambda controls the tension of the spline
		Slider Box( -2, 6, window:loglambda, window:g << reshow ),
		Spacer Box( size( 20, 1 ) ),
		Button Box( "Change Data Points",
			window:generatedata();
			window:g << reshow;
		),
		Spacer Box( size( 20, 1 ) ),
		Text Box( "random: " ),
		Slider Box( // random controls the size of the random Y added to the sine curve
			0,
			20,
			window:random,
			window:generatedata();
			window:g << reshow;
		),
		Spacer Box( size( 20, 1 ) ),
		window:cb = Check Box( // show[] controld which AUC polygons to draw
			{"linear", "spline", "sine"},
			Function( {v, i},
				window:show[i] = v;
				window:g << reshow;
			),
			<<set( 1, window:show[1] ),
			<<set( 2, window:show[2] ),
			<<set( 3, window:show[3] )
		)
	)
);

integrateSpline.jsl