How can I update a shift detected in a new tool to effect my predictive Model?

[#1]

I have done a DOE and developed a LMS model and NN model for prediction purposes. On a new tool we ran some tests and noticed a slight shift, now I would like to build on previous interactions we already discovered but add the appropriate shifts to improve the predictions. Any suggestions?

[#2]

I see three potential ways to handle this issue. First off, I'm going to assume when you say 'shift', you mean a change in the grand mean of the response space,,,but all other aspects of the fitted surface are considered consistent. If the 'shift' is anything else (variance, slope (linear, curvilinear, or any other shape), asymptote (if one is evident) and on and on) then my first suggestion is mute. Probably the simplest thing you can do is save the prediction formula to a column and then just add a constant to the formula that is the magnitude of the 'shift'. The next simplest thing you could do is just build a new model with 'tool' as a column and then within the column levels for the 'old' tool(s) and adding the 'new tool' as a term and build a new model. A third option is to just maintain two models...one for the original analysis, and a second for the 'new tool'. If it were me...I'd try all three and see which pathway helps you solve the practical problem at hand.

But now I have some additional questions.

1. What is an  'LMS model'?

2. I'm curious as to why you gravitated to a Neural Network model? Usually in DOE based modeling work the design is explicitly constructed to estimate a linear model with a specific structure. Especially when using optimal DOE methods. And even if using classic DOE methods, there's an implicit model behind the scenes. So why a Neural Network? I'm guessing the linear model came up way short? And you've got some really nonlinear type surface you are trying to estimate? But again I'm guessing...so what's behind the thought process?

[#3]

Hi, 

Firstly thanks alot for your response.

So part of the issue is that I have a large database of runs I am not going to redo from scratch for each tool.  And each tool has a shift but is not consistent across the range (we saw this in the rerun data). Considering these options I guess the mid one of added a tool column would make sense and not leave out entirely previous effects. 

1. LMS - least means squares -- the basic linear modeling option. 

2.  So this was interesting because we wanted to use existing databases that were not properly designed for DOE so trying to run an LMS DOE analysis turns up a lot of confounding factors and hence singularity points which can be 'turned-off' by controlling the interpolation but it does not correctly describe the space.  I found the NN is more robust in this sense, which is not ideal because it's a probabilistic model but it does fill in these blanks. 

[#4]

Just a point of language clarification...The method is usually called Least Squares, Ordinary Least Squares or Standard Least Squares (hence the confusion using LMS).  This is a regression procedure which estimates the best fit line through the data by minimizing (least) the squared deviation (sum of squares) of residuals from the line.

https://www.jmp.com/support/help/en/18.0/?os=mac&source=application#page/jmp/example-of-simple-linea...

[#5]

Pete is way more generous with his advice given the information you provided.  Unfortunately, for me, you haven't provided near enough context.    I have some thoughts, but they are just guesses.

Where are you in the knowledge continuum? You ran 1 DOE and think you have arrived at a useful prediction formula? (unlikely, typically model development requires iteration).  You only have 1 Y (response variable, unlikely since we live in a multivariate world)?  My guess is you mean Least Squares model in addition to neural network? These are two completely different strategies for model building.  One based on knowledge and the other based on probability (patterns in  the data).  How similar are they?   How useful are they?  How well do they actually predict?   What do the residuals look like? 

Since you introduced a new tool and saw an effect, what is your hypothesis? Realize, you may not be able to just "add" for the shift as the new tool may interact with other factors in the model.  Also, how sure are you it was the new tool explaining the shift?  Did anything else change?

I am suspicious you haven't identified all the possible factors that might affect the response(s).  Perhaps you should go back to screening and increase your inference space (more factors across changing noise) and continue sequential iterations to build your model.

[#6]

Hi, 

Thanks a lot for your input.

I totally agree with your reasoning and running a proper screening design across all parameters makes a lot of sense.  I'll try this approach but sometimes we can't control all the testing that's done.  

As I mentioned in the previous comment why NN provided some better 'space-filling' on interactions I am not comfortable with this assumption of knowledge about the process.

Regardless, I believe my question still stands with its own merit about how to constantly update a NN model for prediction purposes from updated results. I was thinking of something with a weighted functionality for current process results similar to EWMA charts in SPC.