neural network and control chart

[#1]

hello
Is a neural network affected by outliers?
I want to design a Phase I control chart based on a neural network, but after each removal of outliers and re-modeling of the neural network, the accuracy of the model decreases.
What solution do you suggest?

[#2]

Hi, @maryam_nourmand !

NN's are dependent more on the size of the training data set - particularly when it's small.  They also generally have a random number generator built into them as part of the training process.  I'd suggest three things:  

First - set the random seed (that will make your NN reproducible on the same data set). The seed can be any number you choose - just put something in that field in the dialog.  

Second - make sure you're setting up your validation strategy correctly for your data set. There are lots of discussions around how to do this on the community. 

Third - ensure that you're using a large enough data set.  I've seen a lot of suggestions around this, but one I like is 10-20 measurements for every factor the NN is considering in both the training and validation sets.   So, if you have 10 factors - that would be between 200 and 400 measurements.  

Hope that helps!

Best,

M

[#3]

To add on to @MikeD_Anderson's excellent advice, a neural network can certainly be affected by an outlier. It may not be too severe because you should have a larger sample size, as Mike stated. However, because neural networks are extremely flexible models, they do not do very well with very noisy data. They will tend to "overreact" to the extra noise. And that is why they can be affected by an outlier. They will try to explain the outliers, which can cause model bias. This points to the validation strategy that Mike mentioned.

[#4]

Hello,

Thank you for your response.

My data is in the healthcare field, and due to the real nature and the difficulty of sampling, I could only collect 94 samples and have 7 predictors. In previous research, logistic regression was used for classification. In my research, I wanted to use a neural network. However, for designing a Phase I control chart, due to setting alpha and identifying outliers and removing data from the dataset and retraining the neural network, I encountered a problem. Do you have any solutions for designing a Phase I control chart based on the neural network model (preferably without removing data)?

[#5]

In your situation you're really going to be hurting for data with a formal NN.  You might have a look at SVEM's (self-validating ensemble models).  They're designed to cross-validate on DOE's (very small data sets) and sometimes can help out in situations like this.  They also behave somewhat like NN (which are actually just ensemble models with a fancy name ). 

You can learn more about them here: https://community.jmp.com/t5/Elevated-Thinking/SVEM-in-JMP-17/ba-p/538553 

or here: https://community.jmp.com/t5/Discovery-Summit-Europe-2021/Re-Thinking-the-Design-and-Analysis-of-Exp...

[#6]

I'm curious as to why you are choosing to use a neural network modeling approach to the initiation of a Phase 1 control chart? How does a NN model help you establish control limits...especially for a response that, from what you've shared so far, is a nominal response typically used for classification purposes? A control chart has an explicit production order inherent to the process under study. Maybe you can share more information about the process under study and the practical objective of establishing a Phase 1 control chart...which almost by definition says you are interested in studying/evaluating the process in some future time space for stability in a Phase II mode.