SavePredicteds is slow and wrong

mann — Tue, 08 Aug 2023 13:31:10 GMT

		// section 1
		bivar = dt << Bivariate( Y( :PolarRadius ), X( :PolarThetaRound ),
			Fit Spline( 0.008, {Line Color( {212, 73, 88} )} ),
			Local Data Filter( Add Filter( columns( :PolarRadius, :i ),
					Where( :PolarRadius >= 22.5 & :PolarRadius <= 26.5 ),
					Where( :i >= 0.4 & :i <= 0.41 )
				)
			)
		);
		
		//	section 2
		bivar << (curve[1] << savePredicteds);

I am using JMP 14.3.0 on Windows 10. I am working in a table with 5M rows - a photograph of a disc, I am trying to find the edge. When I run the above code as one block, it takes over two minutes and the new column ("Spline Predictor for PolarRadius") is full of incorrect values; the formula for the column includes an array [9980x5]. If I run section 1 alone, it takes a 5 seconds. And then if I run section 2 alone, it takes 2 seconds, and the new column is correct and includes an array 10x smaller: [971x5]. I have tried inserting a few lines between section 1 and 2, but nothing changes. How do I get that "SavePredicteds" function to do the right thing without babysitting it?

Re: SavePredicteds is slow and wrong

danschikore — Tue, 08 Aug 2023 13:49:46 GMT

Try adding a Wait(0) to force separation between the two sections - I believe that would be sufficient. The issue is that versions of JMP prior to JMP 17 run the initial analysis on the full data prior to applying the filter. Some platforms will still do that, but starting in JMP 17 Bivariate and others that are designed to work with empty tables will apply the filter on the first pass.

topic Re: SavePredicteds is slow and wrong in Discussions

SavePredicteds is slow and wrong

Re: SavePredicteds is slow and wrong