topic Re: some experiments failed, what next can be done in Discussions

some experiments failed, what next can be done

VarunK — Fri, 18 Aug 2023 20:07:04 GMT

Hello:

I was working on a university project and performed a 3 factor (A, B, C) DOE, where 2 factors (A & C) had 3 levels and 1 factor (B) had 2 levels. All the factors were continuous. In total 18 experiments were done. Two experiments failed and I came to know this only after the tests were done (the outcome is not as expected, it seems that the combination of lower level of A and lower level of C is not feasible).

What are the next steps that can be taken.

Any help is highly appreciated.

Best Regards,

Varun

Re: some experiments failed, what next can be done

statman — Fri, 18 Aug 2023 21:56:32 GMT

Welcome to the community. What do you mean by failed? You are trying to exaggerate variation in the response. It seems you learned something. That is the purpose of experimentation.

What are the response variables? Should you create a new response which capture the phenomena better? Have you assessed the measurement system.

Losing 2/18 is tough. You have the following options:

Use the grand average of the existing data.
Use your predicted value (or modified prediction). This assumes you did predictions before the experiment.
Use regression to approximate the lost treatment by leaving the highest order (or least likely to be active) term out of the model. Model the remaining data and use the value from the prediction equation (saved prediction formula).
A combination of the above.

If the above results disagree, then:

Rerun the missing treatment combination (and perhaps some others?). Be aware of potential blocking effects and changes in Unit Structure.
Rerun the entire DOE (in the same space?).
If you had anticipated (predicted) this and run repeats you could use the other data points for that treatment.
Create Y’s that reflect the lost/special data point phenomena.

Can you graph the data to get clues on direction for the factors?

Re: some experiments failed, what next can be done

VarunK — Mon, 21 Aug 2023 12:39:57 GMT

Thank you statman:

I am a novice in DOE and so please excuse me if I make a foolish statement. Please correct me and help me learn as it would be very helpful to me.

These results also show that how much important is the domain knowledge.

I was planning on doing the point-3 that you mentioned by removing the two data points and only analyzing the remaining 16 data points, but want to ask if I should take some care, look out for something or make some settings before analyzing the remaining data and formulizing my regression on 16 data points.

I was doing some research and read about custom DOE designs where I can apply some constraints to control the design space. I would like to ask, having gained some knowledge on not-feasible combination, if I create a custom design with constraints now (eliminating those two settings) and just plugin the existing response which I got from my previous DOE,

Will it work and what care should I take before setting up the DOE?

Will it be different than what I have already done?

Is it worth doing?

Your help is highly appreciated.

Best Regards,

Varun Katiyar

Re: some experiments failed, what next can be done

statman — Mon, 21 Aug 2023 14:23:41 GMT

"I was planning on doing the point-3 that you mentioned by removing the two data points and only analyzing the remaining 16 data points, but want to ask if I should take some care, look out for something or make some settings before analyzing the remaining data and formulizing my regression on 16 data points." The point I make is you should try multiple substitution methods (not 1). Compare the results from each analysis. If they are in general agreement, perhaps you can get by with those results. If not you will have to perform additional experiments.

Also, you have done no replication. How many times did the results from A low and C low create the result "failed"? I would replicate this a few times before conclusion.

If you are going to design a new experiment, you should iterate. That is, run the new experiment (not use existing data). This is called Scientific Method. Hypotheses>Data>Hypotheses>Data...

I would take all of the clues from the first experiment (factor level setting, direction, etc.) and move the design space towards optimum response. Plan a new experiment and continue to iterate.

I also highly suggest you enroll in classes and read the plethora of papers and book to learn more about experimentation.

You can start your educational journey here:

https://community.jmp.com/t5/Learning-Center/Design-of-Experiments-Introduction-Kit/ta-p/280499

https://www.jmp.com/en_us/online-statistics-course.html

Re: some experiments failed, what next can be done

VarunK — Mon, 21 Aug 2023 16:02:59 GMT

Thank you statman.