topic Re: Nonlinear Fit - How to check for success? in Discussions

Nonlinear Fit - How to check for success?

AApperson — Thu, 08 Jun 2023 16:36:20 GMT

Overview:

How can I check to see a Nonlinear fit succeeded?

(Real-data doesn't always converge and may need manual review for automation/test/product issues.)

Detail:

I would like to script a response after a nonlinear fit fails to converge. If the fit fails, I need to process another way, and/or eventually label for manual review. (I'm sampling a population, then for each part, doing a non-linear fit. Generally if I fail convergence, there was a test issue somewhere; I don't actually want to loosen the convergence criteria.)

('Failed' message: "Failed: Cannot Decrease Objective Function")

Modified script from Numeric Derivatives Only (Help > Scripting Index > "<<Nonlinear")

(Currently running JMP 15.2.1)

Names Default To Here( 1 );
dt = Open(
	"$SAMPLE_DATA/Nonlinear Examples/US Population.jmp"
);

// Random, intentionally divergent, formula on this dataset
dt << New Column( "Random", Numeric, Formula( Random LogLogistic(5) ) );

dt << New Column( "FancyPredictionFormula",
	Numeric,
	Continuous,
	Formula(
		Parameter(
			{Rth0 = 735, n = 0.5},
			(:year / :pop) * ((1 - ((n - 1) * :pop * Rth0) / (:year
			 * (300 / :year) ^ n)) ^ (-1 / (n - 1)) - 1)
		)
	)
);

nonlinear = dt << Nonlinear(
	Y(  :Name( "Random" ) ),
	X( :Name("FancyPredictionFormula") ),
	Expand Intermediate Formulas( 1 ),
	Numeric Derivatives Only( 1 ),
	QuasiNewton SR1,
	Interation Limit(50),
	Finish,
	Plot( 0 ),
	Confidence Limits,
	Save Estimates
);

// How to check if nonlinear fit succeeded?

Ideal.

If there is a bool status variable returned by the Nonlinear fit platform somewhere indicating success or failure, that would be the ideal method to verify success, I think.

Alternative 1.

I can also tabulate or take a summary of the data to see if all the :Name("FancyPredictionFormula") values are zero. (Values seem to be 0 if the fit fails).

Alternative 2.

There's probably a way to check the XML and read the 'failed' message.

Thank you for your time and consideration.

-Aubrey

Re: Nonlinear Fit - How to check for success?

Jed_Campbell — Mon, 13 Feb 2023 21:28:25 GMT

Here's an approach using Alternative 2a (scraping the words from the report). The first 2 lines do the actual work, and the third line is just a simple example of what to do with the information; you likely have something else in mind for this part.

// How to check if nonlinear fit succeeded?
r = nonlinear << report;
alltext = r[Outline Box("Control Panel")] << get text;
if (contains(alltext, "Cannot Decrease"), caption ("Failed: Cannot Decrease ..."));

Re: Nonlinear Fit - How to check for success?

AApperson — Tue, 14 Feb 2023 12:25:10 GMT

That'll do it. Thanks Jed!

Re: Nonlinear Fit - How to check for success?

Mark_Bailey — Tue, 14 Feb 2023 19:27:20 GMT

I wonder if a better check is if the text is anything other than success:

If( Not( Contains( alltext, "Converged" ) ),
	// trap failure
	,
	// proceed after success
);

Re: Nonlinear Fit - How to check for success?

AApperson — Tue, 14 Feb 2023 21:53:21 GMT

You are closer to where I am heading with it.

I've been working with:

if (contains(alltext, "Failed"), caption ("Failed: Do additional action now ..."));

As a catch all, but I'm considering adding more of a case statement (or I guess multiple 'ifs'), to specifically handle the fail reason.

e.g. I can run again with a higher iteration count if I catch this fail-reason:

contains(alltext, "Maximum Iteration Exceeded");

The real motivation is, 'most' parts will converge with a numerical approach, but some seem to require analytical. I'm not sure what's happening to cause this. Convergence criteria? But the 'fix' is to run again without numerical derivatives, something like:

nonlinear = dt << Nonlinear(
	Y(  :Name( "Random" ) ),
	X( :Name("FancyPredictionFormula") ),
	Expand Intermediate Formulas( 1 ),
	Newton,
	Finish,
	Plot( 0 ),
	Confidence Limits,
	Save Estimates
);

After catching with:

contains(alltext, "Cannot Decrease Objective Function");

Re: Nonlinear Fit - How to check for success?

Mark_Bailey — Wed, 15 Feb 2023 16:58:52 GMT

The usual culprit behind failing to converge is the starting parameter values. It is often sufficient to use domain-specific heuristics to initialize them when available.