Robust Optimization - maximize desirability based on feedback from simulation?

[#1]

Consider a system where an optimal point is at the edge of an acceptable operating envelope.  If attempting to run there, small changes in controlled variables can lead to poor results in some fraction of the total runs.  It is possible to use the optimization built into the profiler to find optimal conditions, and to use simulation to find more robust target operating conditions, but can they be used together?

I see several papers describing the use of simulation alone to find robust optimal points, including simulation for points in a grid and picking the best.  Instead, can I specify a simulation to do at each point and some passing criteria, and have the optimizer use that to find optimal solutions without having to calculate a grid across the entire potential operating space?  Is there a better method?

Today I typically use the optimizer first and then do simulations around those points to find more robust operating ranges. This is a pretty manual process and can be time intensive for complex desirability functions in many dimensions.

Example

Here the optimal conditions for a system were identified, but simulation at those points shows a high number of low desirability results.  Here the 0.05 standard deviation is meant to represent how accurately X1 and X2 can be controlled.

By manually moving X2 away from the 'cliff', the majority of results are good, even though the median desirability is lower:

Visualized in a contour plot, here are operating ranges around the optimal solution and the robust solution:

Script to recreate example data and profilers:

View more...

Names default to here(1);

dt = New Table( "Robust Simulation Example",
	Add Rows( 1000 ),
	New Column( "x1",
		Numeric,
		"Continuous",
		Format( "Best", 12 ),
		Formula( Random Uniform( -1, 1 ) )
	),
	New Column( "x2",
		Numeric,
		"Continuous",
		Format( "Best", 12 ),
		Formula( Random Uniform( -1, 1 ) )
	),
	New Column( "y",
		Numeric,
		"Continuous",
		Format( "Best", 12 ),
		Formula( -(:x1 ^ 2 + :x2 ^ 2) )
	),
	New Column( "Feasible Solution",
		Numeric,
		"Continuous",
		Format( "Best", 12 ),
		Formula( If( :x2 > 0.1, 1, 0 ) ),
		Set Selected
	),
	New Column( "Desirability",
		Numeric,
		"Continuous",
		Format( "Best", 12 ),
		Formula( If( If( :x2 > 0.1, 1, 0 ) == 1, -(:x1 ^ 2 + :x2 ^ 2), -2 ) )
	)

);


New Window("Finding robust solution",
	(
		H List Box(
			dt << Profiler(
				Y( :y, :Feasible Solution ),
				Profiler(
					1,
					Desirability Functions( 1 ),
					Term Value(
						x1( -0.0000000003269293581, Lock( 0 ), Show( 1 ) ),
						x2( 0.100000001061837, Lock( 0 ), Show( 1 ) )
					)
				),
				SendToReport(
					Dispatch( {}, "Profiler", OutlineBox, {Set Title( "Optimized Solution" )} )
				)
			),
			dt << Profiler(
				Y( :y, :Feasible Solution ),
				Profiler(
					1,
					Desirability Functions( 1 ),
					Term Value(
						x1( -0.0000000003269293581, Lock( 0 ), Show( 1 ) ),
						x2( 0.100000001061837, Lock( 0 ), Show( 1 ) )
					),
					Simulator(
						1,
						Factors(
							x1 << Random( Normal( -0.0000000003269293581, 0.05 ) ),
							x2 << Random( Normal( 0.100000001061837, 0.05 ) )
						),
						Responses( y << No Noise, Feasible Solution << No Noise ),
						Resimulate
					)
				),
				SendToReport(
					Dispatch(
						{},
						"Profiler",
						OutlineBox,
						{Set Title(
							"Simulation of optimized solution if only able to control each x to have standard deviation of 0.05, note the large number of infeasible solution"
						)}
					),
					Dispatch(
						{"Prediction Profiler"},
						"Profile Simulator Histogram",
						FrameBox,
						{DispatchSeg(
							Hist Seg( 1 ),
							{Line Color( {65, 104, 68} ), Fill Color( "Green" ),
							Bin Span( 2, 0 )}
						)}
					),
					Dispatch(
						{"Prediction Profiler"},
						"Profile Simulator Histogram",
						FrameBox( 2 ),
						{DispatchSeg(
							Hist Seg( 1 ),
							{Line Color( {65, 104, 68} ), Fill Color( "Green" ),
							Bin Span( 2, 0 )}
						)}
					),
					Dispatch(
						{"Prediction Profiler"},
						"Profile Simulator Histogram",
						FrameBox( 3 ),
						{DispatchSeg(
							Hist Seg( 1 ),
							{Line Color( {65, 104, 68} ), Fill Color( "Green" ),
							Bin Span( 2, 0 )}
						)}
					),
					Dispatch(
						{"Prediction Profiler"},
						"",
						Border Box( 21 ),
						Add Text Annotation(
							Text( "Lots of Low
	Desirability
	Results" ),
							Fixed Size( 0 ),
							Text Box( {-104, 38, -15, 133} ),
							Filled( 0 ),
							Text Color( "Medium Dark Magenta" ),
							"Font"("Segoe UI", 10, "Bold")
						)
					),
					Dispatch(
						{"Prediction Profiler"},
						"Profile Simulator Histogram",
						FrameBox( 3 ),
						{DispatchSeg(
							Hist Seg( 1 ),
							{Line Color( {65, 104, 68} ), Fill Color( "Green" ),
							Bin Span( 2, 0 )}
						), Add Line Annotation(
							Line( {-8, 68}, {-22, 65} ),
							Color( "Medium Dark Magenta" ),
							Thick( 1 )
						)}
					)
				)
			),
			dt << Profiler(
				Y( :y, :Feasible Solution ),
				Profiler(
					1,
					Desirability Functions( 1 ),
					Term Value(
						x1( -0.0000000003269293581, Lock( 0 ), Show( 1 ) ),
						x2( 0.203, Lock( 0 ), Show( 1 ) )
					),
					Simulator(
						1,
						Factors(
							x1 << Random( Normal( -0.0000000003269293581, 0.05 ) ),
							x2 << Random( Normal( 0.203, 0.05 ) )
						),
						Responses( y << No Noise, Feasible Solution << No Noise ),
						Resimulate
					)
				),
				SendToReport(
					Dispatch(
						{},
						"Profiler",
						OutlineBox,
						{Set Title(
							"Solution moved away from the 'edge' so an acceptable fraction of simulated solutions are infeasible"
						)}
					),
					Dispatch(
						{"Prediction Profiler"},
						"Profile Simulator Histogram",
						FrameBox,
						{DispatchSeg(
							Hist Seg( 1 ),
							{Line Color( {65, 104, 68} ), Fill Color( "Green" ),
							Bin Span( 2, 0 )}
						)}
					),
					Dispatch(
						{"Prediction Profiler"},
						"Profile Simulator Histogram",
						FrameBox( 2 ),
						{DispatchSeg(
							Hist Seg( 1 ),
							{Line Color( {65, 104, 68} ), Fill Color( "Green" ),
							Bin Span( 2, 0 )}
						)}
					),
					Dispatch(
						{"Prediction Profiler"},
						"Profile Simulator Histogram",
						FrameBox( 3 ),
						{DispatchSeg(
							Hist Seg( 1 ),
							{Line Color( {65, 104, 68} ), Fill Color( "Green" ),
							Bin Span( 2, 0 )}
						)}
					),
					Dispatch(
						{"Prediction Profiler"},
						"",
						Border Box( 21 ),
						Add Text Annotation(
							Text( "Acceptable
	fraction of low
	Desirability
	Results" ),
							Fixed Size( 0 ),
							Text Box( {-111, 25, 8, 108} ),
							Filled( 0 ),
							Text Color( "Medium Dark Magenta" ),
							"Font"("Segoe UI", 10, "Bold")
						)
					),
					Dispatch(
						{"Prediction Profiler"},
						"Profile Simulator Histogram",
						FrameBox( 3 ),
						{DispatchSeg(
							Hist Seg( 1 ),
							{Line Color( {65, 104, 68} ), Fill Color( "Green" ),
							Bin Span( 2, 0 )}
						), Add Line Annotation(
							Line( {-8, 68}, {-22, 65} ),
							Color( "Medium Dark Magenta" ),
							Thick( 1 )
						)}
					),
					Dispatch(
						{"Prediction Profiler"},
						"Profiler",
						FrameBox( 6 ),
						Add Text Annotation(
							Text( "Moved from 0.1 to 0.2" ),
							Fixed Size( 1 ),
							Text Box( {7, 20, 72, 89} ),
							Filled( 0 ),
							Text Color( "Medium Dark Magenta" ),
							"Font"("Segoe UI", 10, "Bold")
						)
					)
				)
			)
		)
	)
);

[#2 · Accepted solution]

JMP Pro has some good tools to help you out here.

Specifically, you can build your model in any of the modeling platforms. Then, in the Profiler you can (as you described) use a Simulation Experiment to run multiple Monte Carlo simulations across the factor space. In the resulting table you'll have a defect (out of spec) rate column for each simulation. There's a script attached to that table that will use the Gaussian Process platform to model that defect rate. You can use that model to optimize and get the minimal expected defect rate.

Check out these resources to learn more and see it in action:

Robust Optimisation of Processes and Products by Using Monte Carlo Simulation Experiments 

Example of Stochastic Optimization

Does that point you in the right direction or is there a detail that I missed in your original question?

[#3]

The detail I was looking for was to simulate each point requested by the optimizer instead of having to simulate the entire model space and then represent the simulation results using another model.  For small problems this does not matter, but as more dimensions are added, the simulation experiment starts to act like brute force method.  I came to the conclusion that this is not possible though.