cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
%3CLINGO-SUB%20id%3D%22lingo-sub-600498%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3ENichtlineare%20Anpassung%20%E2%80%93%20Wie%20%C3%BCberpr%C3%BCfe%20ich%20den%20Erfolg%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-600498%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CP%3E%3CSTRONG%3E%C3%9Cberblick%3A%3C%2FSTRONG%3E%3C%2FP%3E%3CP%3E%3CSTRONG%3EWie%20kann%20ich%20%C3%BCberpr%C3%BCfen%2C%20ob%20eine%20nichtlineare%20Anpassung%20erfolgreich%20war%3F%3C%2FSTRONG%3E%3C%2FP%3E%3CP%3E(Echte%20Daten%20stimmen%20nicht%20immer%20%C3%BCberein%20und%20erfordern%20m%C3%B6glicherweise%20eine%20manuelle%20%C3%9Cberpr%C3%BCfung%20auf%20Automatisierungs-%2FTest-%2FProduktprobleme.)%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSTRONG%3EDetail%3A%3C%2FSTRONG%3E%3C%2FP%3E%3CP%3EIch%20m%C3%B6chte%20ein%20Skript%20f%C3%BCr%20eine%20Antwort%20erstellen%2C%20nachdem%20eine%20nichtlineare%20Anpassung%20nicht%20konvergiert.Wenn%20die%20Anpassung%20fehlschl%C3%A4gt%2C%20muss%20ich%20die%20Verarbeitung%20auf%20andere%20Weise%20durchf%C3%BChren%20und%2Foder%20sie%20eventuell%20zur%20manuellen%20%C3%9Cberpr%C3%BCfung%20kennzeichnen.%20(Ich%20bemustere%20eine%20Grundgesamtheit%20und%20f%C3%BChre%20dann%20f%C3%BCr%20jeden%20Teil%20eine%20nichtlineare%20Anpassung%20durch.Wenn%20die%20Konvergenz%20fehlschl%C3%A4gt%2C%20liegt%20im%20Allgemeinen%20irgendwo%20ein%20Testproblem%20vor.%20Eigentlich%20m%C3%B6chte%20ich%20die%20Konvergenzkriterien%20nicht%20lockern.)%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E(%E2%80%9EFehlgeschlagen%E2%80%9C-Meldung%3A%20%E2%80%9EFehlgeschlagen%3A%20Zielfunktion%20kann%20nicht%20verringert%20werden%E2%80%9C)%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Nonlinear%20Fail%20Screenshot.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%22%20image-alt%3D%22Nonlinear%20Fail%20Screenshot.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3Cspan%20class%3D%22lia-inline-image-display-wrapper%22%20image-alt%3D%22Nonlinear%20Fail%20Screenshot.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3Cimg%20src%3D%22https%3A%2F%2Fcommunity.jmp.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F50091i16B5465CA55A7137%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Nonlinear%20Fail%20Screenshot.png%22%20alt%3D%22Nonlinear%20Fail%20Screenshot.png%22%20%2F%3E%3C%2Fspan%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EGe%C3%A4ndertes%20Skript%20von%20Nur%20numerische%20Ableitungen%20(Hilfe%20%26gt%3B%20Skriptindex%20%26gt%3B%20%E2%80%9E%26lt%3B%3CNICHTLINEAR%3E%3C%2FNICHTLINEAR%3E%3C%2FP%3E%3CP%3E(Derzeit%20l%C3%A4uft%20JMP%2015.2.1)%3C%2FP%3E%3CPRE%3E%3CCODE%20class%3D%22%20language-jsl%22%3ENames%20Default%20To%20Here(%201%20)%3B%0Adt%20%3D%20Open(%0A%20%22%24SAMPLE_DATA%2FNonlinear%20Examples%2FUS%20Population.jmp%22%0A)%3B%0A%0A%2F%2F%20Random%2C%20intentionally%20divergent%2C%20formula%20on%20this%20dataset%0Adt%20%26lt%3B%26lt%3B%20New%20Column(%20%22Random%22%2C%20Numeric%2C%20Formula(%20Random%20LogLogistic(5)%20)%20)%3B%0A%0Adt%20%26lt%3B%26lt%3B%20New%20Column(%20%22FancyPredictionFormula%22%2C%0A%20Numeric%2C%0A%20Continuous%2C%0A%20Formula(%0A%20%20Parameter(%0A%20%20%20%7BRth0%20%3D%20735%2C%20n%20%3D%200.5%7D%2C%0A%20%20%20(%3Ayear%20%2F%20%3Apop)%20*%20((1%20-%20((n%20-%201)%20*%20%3Apop%20*%20Rth0)%20%2F%20(%3Ayear%0A%20%20%20%20*%20(300%20%2F%20%3Ayear)%20%5E%20n))%20%5E%20(-1%20%2F%20(n%20-%201))%20-%201)%0A%20%20)%0A%20)%0A)%3B%0A%0Anonlinear%20%3D%20dt%20%26lt%3B%26lt%3B%20Nonlinear(%0A%20Y(%20%20%3AName(%20%22Random%22%20)%20)%2C%0A%20X(%20%3AName(%22FancyPredictionFormula%22)%20)%2C%0A%20Expand%20Intermediate%20Formulas(%201%20)%2C%0A%20Numeric%20Derivatives%20Only(%201%20)%2C%0A%20QuasiNewton%20SR1%2C%0A%20Interation%20Limit(50)%2C%0A%20Finish%2C%0A%20Plot(%200%20)%2C%0A%20Confidence%20Limits%2C%0A%20Save%20Estimates%0A)%3B%0A%0A%2F%2F%20How%20to%20check%20if%20nonlinear%20fit%20succeeded%3F%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CU%3EIdeal.%3C%2FU%3E%3C%2FP%3E%3CP%3EWenn%20irgendwo%20eine%20Bool-Statusvariable%20von%20der%20Nonlinear-Fit-Plattform%20zur%C3%BCckgegeben%20wird%2C%20die%20Erfolg%20oder%20Misserfolg%20anzeigt%2C%20w%C3%A4re%20das%20meiner%20Meinung%20nach%20die%20ideale%20Methode%2C%20um%20den%20Erfolg%20zu%20%C3%BCberpr%C3%BCfen.%3C%2FP%3E%3CP%3E%3CU%3EAlternative%201.%3C%2FU%3E%3C%2FP%3E%3CP%3EIch%20kann%20die%20Daten%20auch%20tabellarisch%20darstellen%20oder%20eine%20Zusammenfassung%20erstellen%2C%20um%20zu%20sehen%2C%20ob%20alle%3CFONT%20face%3D%22verdana%2Cgeneva%22%3E%20%3A%3CFONT%20color%3D%22%230000FF%22%3E%20Name%3C%2FFONT%3E%20(%3CFONT%20color%3D%22%23993366%22%3E%20%E2%80%9EFancyPredictionFormula%E2%80%9C%3C%2FFONT%3E%20)%3C%2FFONT%3E%20Werte%20sind%20Null.(Werte%20scheinen%200%20zu%20sein%2C%20wenn%20die%20Anpassung%20fehlschl%C3%A4gt).%3C%2FP%3E%3CP%3E%3CU%3EAlternative%202.%20%3C%2FU%3E%3C%2FP%3E%3CP%3EEs%20gibt%20wahrscheinlich%20eine%20M%C3%B6glichkeit%2C%20das%20XML%20zu%20%C3%BCberpr%C3%BCfen%20und%20die%20Meldung%20%E2%80%9Efehlgeschlagen%E2%80%9C%20zu%20lesen.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EVielen%20Dank%20f%C3%BCr%20Ihre%20Zeit%20und%20R%C3%BCcksicht.%3C%2FP%3E%3CP%3E-Aubrey%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-600498%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CLINGO-LABEL%3EAutomatisierung%20und%20Skripterstellung%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-601522%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3EBetreff%3A%20Nichtlineare%20Anpassung%20%E2%80%93%20Wie%20%C3%BCberpr%C3%BCfe%20ich%20den%20Erfolg%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-601522%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CP%3EDer%20%C3%BCbliche%20Grund%20f%C3%BCr%20das%20Scheitern%20der%20Konvergenz%20sind%20die%20Startparameterwerte.%20Oft%20reicht%20es%20aus%2C%20dom%C3%A4nenspezifische%20Heuristiken%20zu%20verwenden%2C%20um%20sie%20zu%20initialisieren%2C%20sofern%20verf%C3%BCgbar.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-601121%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3EBetreff%3A%20Nichtlineare%20Anpassung%20%E2%80%93%20Wie%20%C3%BCberpr%C3%BCfe%20ich%20den%20Erfolg%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-601121%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CP%3EDu%20bist%20n%C3%A4her%20dran%2C%20wohin%20ich%20damit%20gehe.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EIch%20habe%20gearbeitet%20mit%3A%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%3E%3CCODE%20class%3D%22%20language-jsl%22%3Eif%20(contains(alltext%2C%20%22Failed%22)%2C%20caption%20(%22Failed%3A%20Do%20additional%20action%20now%20...%22))%3B%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3EAls%20Haken%20f%C3%BCr%20alles%2C%20aber%20ich%20denke%20dar%C3%BCber%20nach%2C%20eher%20eine%20Case-Anweisung%20(oder%20ich%20vermute%20mehrere%20%E2%80%9EWenns%E2%80%9C)%20hinzuzuf%C3%BCgen%2C%20um%20speziell%20den%20Fehlergrund%20zu%20behandeln.%3CCODE%20class%3D%22%20language-jsl%22%3E%3C%2FCODE%3E%3C%2FP%3E%3CP%3E%3CCODE%20class%3D%22%20language-jsl%22%3E%3C%2FCODE%3E%3C%2FP%3E%3CP%3EBeispielsweise%20kann%20ich%20mit%20einer%20h%C3%B6heren%20Iterationszahl%20erneut%20ausf%C3%BChren%2C%20wenn%20ich%20diesen%20Fehlergrund%20erkenne%3A%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%3E%3CCODE%20class%3D%22%20language-jsl%22%3Econtains(alltext%2C%20%22Maximum%20Iteration%20Exceeded%22)%3B%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EDie%20eigentliche%20Motivation%20besteht%20darin%2C%20dass%20%E2%80%9Edie%20meisten%E2%80%9C%20Teile%20mit%20einem%20numerischen%20Ansatz%20konvergieren%2C%20einige%20jedoch%20offenbar%20eine%20Analyse%20erfordern.Ich%20bin%20mir%20nicht%20sicher%2C%20was%20die%20Ursache%20daf%C3%BCr%20ist.Konvergenzkriterien%3FDie%20%E2%80%9EL%C3%B6sung%E2%80%9C%20besteht%20jedoch%20darin%2C%20erneut%20ohne%20numerische%20Ableitungen%20zu%20arbeiten%2C%20etwa%20so%3A%3C%2FP%3E%3CPRE%3E%3CCODE%20class%3D%22%20language-jsl%22%3Enonlinear%20%3D%20dt%20%26lt%3B%26lt%3B%20Nonlinear(%0A%20Y(%20%20%3AName(%20%22Random%22%20)%20)%2C%0A%20X(%20%3AName(%22FancyPredictionFormula%22)%20)%2C%0A%20Expand%20Intermediate%20Formulas(%201%20)%2C%0A%20Newton%2C%0A%20Finish%2C%0A%20Plot(%200%20)%2C%0A%20Confidence%20Limits%2C%0A%20Save%20Estimates%0A)%3B%3C%2FCODE%3E%3C%2FPRE%3E%3CP%3ENach%20dem%20Fang%20mit%3A%3C%2FP%3E%3CPRE%3E%3CCODE%20class%3D%22%20language-jsl%22%3Econtains(alltext%2C%20%22Cannot%20Decrease%20Objective%20Function%22)%3B%3C%2FCODE%3E%3C%2FPRE%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-601016%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3EBetreff%3A%20Nichtlineare%20Anpassung%20%E2%80%93%20Wie%20%C3%BCberpr%C3%BCfe%20ich%20den%20Erfolg%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-601016%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CP%3EIch%20frage%20mich%2C%20ob%20eine%20bessere%20%C3%9Cberpr%C3%BCfung%20darin%20besteht%2C%20ob%20der%20Text%20alles%20andere%20als%20erfolgreich%20ist%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%3E%3CCODE%20class%3D%22%20language-jsl%22%3EIf(%20Not(%20Contains(%20alltext%2C%20%22Converged%22%20)%20)%2C%0A%20%2F%2F%20trap%20failure%0A%20%2C%0A%20%2F%2F%20proceed%20after%20success%0A)%3B%3C%2FCODE%3E%3C%2FPRE%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-600846%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3EBetreff%3A%20Nichtlineare%20Anpassung%20%E2%80%93%20Wie%20%C3%BCberpr%C3%BCfe%20ich%20den%20Erfolg%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-600846%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CP%3EDas%20reicht.Danke%20Jed!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-600622%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3EBetreff%3A%20Nichtlineare%20Anpassung%20%E2%80%93%20Wie%20%C3%BCberpr%C3%BCfe%20ich%20den%20Erfolg%3F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-600622%22%20slang%3D%22en-US%22%20mode%3D%22NONE%22%3E%3CP%3EHier%20ist%20ein%20Ansatz%20unter%20Verwendung%20von%20Alternative%202a%20(Ausschneiden%20der%20W%C3%B6rter%20aus%20dem%20Bericht).%20Die%20ersten%20beiden%20Zeilen%20erledigen%20die%20eigentliche%20Arbeit%20und%20die%20dritte%20Zeile%20ist%20nur%20ein%20einfaches%20Beispiel%20daf%C3%BCr%2C%20was%20mit%20den%20Informationen%20geschehen%20soll%3B%20Wahrscheinlich%20haben%20Sie%20f%C3%BCr%20diesen%20Teil%20etwas%20anderes%20im%20Sinn.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%3E%3CCODE%20class%3D%22%20language-jsl%22%3E%2F%2F%20How%20to%20check%20if%20nonlinear%20fit%20succeeded%3F%0Ar%20%3D%20nonlinear%20%26lt%3B%26lt%3B%20report%3B%0Aalltext%20%3D%20r%5BOutline%20Box(%22Control%20Panel%22)%5D%20%26lt%3B%26lt%3B%20get%20text%3B%0Aif%20(contains(alltext%2C%20%22Cannot%20Decrease%22)%2C%20caption%20(%22Failed%3A%20Cannot%20Decrease%20...%22))%3B%3C%2FCODE%3E%3C%2FPRE%3E%3C%2FLINGO-BODY%3E
Choose Language Hide Translation Bar
AApperson
Level II

Nonlinear Fit - How to check for success?

Overview:

How can I check to see a Nonlinear fit succeeded?

(Real-data doesn't always converge and may need manual review for automation/test/product issues.)

 

Detail:

I would like to script a response after a nonlinear fit fails to converge.  If the fit fails, I need to process another way, and/or eventually label for manual review. (I'm sampling a population, then for each part, doing a non-linear fit.  Generally if I fail convergence, there was a test issue somewhere; I don't actually want to loosen the convergence criteria.)

 

('Failed' message: "Failed: Cannot Decrease Objective Function")

Nonlinear Fail Screenshot.png

 

Modified script from Numeric Derivatives Only (Help > Scripting Index > "<<Nonlinear")

(Currently running JMP 15.2.1)

Names Default To Here( 1 );
dt = Open(
	"$SAMPLE_DATA/Nonlinear Examples/US Population.jmp"
);

// Random, intentionally divergent, formula on this dataset
dt << New Column( "Random", Numeric, Formula( Random LogLogistic(5) ) );

dt << New Column( "FancyPredictionFormula",
	Numeric,
	Continuous,
	Formula(
		Parameter(
			{Rth0 = 735, n = 0.5},
			(:year / :pop) * ((1 - ((n - 1) * :pop * Rth0) / (:year
			 * (300 / :year) ^ n)) ^ (-1 / (n - 1)) - 1)
		)
	)
);

nonlinear = dt << Nonlinear(
	Y(  :Name( "Random" ) ),
	X( :Name("FancyPredictionFormula") ),
	Expand Intermediate Formulas( 1 ),
	Numeric Derivatives Only( 1 ),
	QuasiNewton SR1,
	Interation Limit(50),
	Finish,
	Plot( 0 ),
	Confidence Limits,
	Save Estimates
);

// How to check if nonlinear fit succeeded?

 

 

 

Ideal.

If there is a bool status variable returned by the Nonlinear fit platform somewhere indicating success or failure, that would be the ideal method to verify success, I think.

Alternative 1.

I can also tabulate or take a summary of the data to see if all the :Name("FancyPredictionFormula") values are zero.  (Values seem to be 0 if the fit fails). 

Alternative 2.

There's probably a way to check the XML and read the 'failed' message. 

 

Thank you for your time and consideration.

-Aubrey

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Nonlinear Fit - How to check for success?

Here's an approach using Alternative 2a (scraping the words from the report). The first 2 lines do the actual work, and the third line is just a simple example of what to do with the information; you likely have something else in mind for this part.

 

// How to check if nonlinear fit succeeded?
r = nonlinear << report;
alltext = r[Outline Box("Control Panel")] << get text;
if (contains(alltext, "Cannot Decrease"), caption ("Failed: Cannot Decrease ..."));

View solution in original post

5 REPLIES 5

Re: Nonlinear Fit - How to check for success?

Here's an approach using Alternative 2a (scraping the words from the report). The first 2 lines do the actual work, and the third line is just a simple example of what to do with the information; you likely have something else in mind for this part.

 

// How to check if nonlinear fit succeeded?
r = nonlinear << report;
alltext = r[Outline Box("Control Panel")] << get text;
if (contains(alltext, "Cannot Decrease"), caption ("Failed: Cannot Decrease ..."));
AApperson
Level II

Re: Nonlinear Fit - How to check for success?

That'll do it.  Thanks Jed!

Re: Nonlinear Fit - How to check for success?

I wonder if a better check is if the text is anything other than success:

 

If( Not( Contains( alltext, "Converged" ) ),
	// trap failure
	,
	// proceed after success
);
AApperson
Level II

Re: Nonlinear Fit - How to check for success?

You are closer to where I am heading with it.

 

I've been working with:

 

if (contains(alltext, "Failed"), caption ("Failed: Do additional action now ..."));

As a catch all, but I'm considering adding more of a case statement (or I guess multiple 'ifs'), to specifically handle the fail reason.

e.g. I can run again with a higher iteration count if I catch this fail-reason:

 

contains(alltext, "Maximum Iteration Exceeded");

 

The real motivation is, 'most' parts will converge with a numerical approach, but some seem to require analytical.  I'm not sure what's happening to cause this.  Convergence criteria?  But the 'fix' is to run again without numerical derivatives, something like:

nonlinear = dt << Nonlinear(
	Y(  :Name( "Random" ) ),
	X( :Name("FancyPredictionFormula") ),
	Expand Intermediate Formulas( 1 ),
	Newton,
	Finish,
	Plot( 0 ),
	Confidence Limits,
	Save Estimates
);

After catching with:

contains(alltext, "Cannot Decrease Objective Function");

Re: Nonlinear Fit - How to check for success?

The usual culprit behind failing to converge is the starting parameter values. It is often sufficient to use domain-specific heuristics to initialize them when available.