What do I do if I forgot to add main effect terms during DoE design?

[#1]

While designing a study to optimize process parameters, I added the interaction terms but not the main effects terms by mistake. Now that I've already collected the data required. What shold I do to get the most out of the data?

Thanks!

[#2]

Hi @zj2000,

Welcome in the Community !

You may have deleted the main effects from your model, but you'll be able to estimate them with your design (even if "less-optimally"): You have supposed interactions and quadratic effects for each of your 7 factors (so each factor has 3 levels, min-medium-max) in the model, and you have enough degree of freedoms to estimate them.

• There are 44 independent experiments in this design, representing 43 degree of freedoms (one is used to estimate the intercept). A model containing main effects (7 terms to estimate), 2-factors interactions (21 terms to estimate) and quadratic effects (7 terms to estimate) would require 35 degree of freedoms (as seen in the capture "Analysis of Variance" from "Fit Model" with random data for one response), so you have enough DF to estimate all these terms (and 8 to estimate model error) :

You can check the power to estimate main effects (and other effects of the model) by using the Evaluate Design platform (menu DoE, Design Diagnostics, Evaluate Design), specify your factors and responses, and add main effects in your supposed model. No error message is displayed, so it is possible to estimate main effects, and you'll obtain this window :

Even if you deleted the main effects from the model in the design generation, the principle of Effect Heredity ensures that if higher order terms are present in the design (like the quadratic terms or 2-factors interactions you have added in the model), lower order terms should be present and estimable in the model (like main effects). Note that the mistake you have realized would be a great suggestion to add in the JMP Wish List about the design creation: Provide a warning (or block ?) if a user specify a model with higher order effects without adding lower order effects.

Looking at the previous snapshot, you can see that main effects still have higher power than quadratic terms even if they were not added during design generation. Most of the main effects could have an higher power if added in the model from the design generation (for example T 2, 100, ...), but you should be able to have a correct analysis and model at the end.
You can compare the design you generated with a design with the same number of runs but with main effects added (see table attached "Custom Design-Process2" to do the designs comparison) to see where this change may impact the results and analysis (lower power for main effects, higher prediction variance at the edges of the experimental space, higher standard error for estimates, ...).

Hope this answer will help you (and appease you), 

[#3]

Hi Victor,

Thank you so much for your reply! I added the 7 main effects terms in the "Construct Model Effects" box and analyzed the data and try to identify optimal process parameters by using the Optimize Desirability function in Prediction Profiler. Would you think it'll work? I am attaching my file with data.

Thanks again!

Jason

[#4]

Hi @zj2000,

I don't know what is your goal, neither your acceptance criterion/threshold for a model, nor what you expect to find with this design. Also depending on your industry/area of research, a specific metric value (like R² = 0,4 or a specific RMSE value for the model) can be considered (sufficiently) good or bad.

In your case with the D2v response values, it is possible to identify possible important factors (to be confirmed or not with your domain expertise). I don't know if you have JMP or JMP Pro, but I'll mainly use JMP possibilities to be sure :

• If you want to find possible important factors for this response, you can use an exploratory tool like the Predictor Screening (in Analyze, Screening, Predictor Screening) : Based on a Bootstrap forest, it is an helpful tool to rank and see what could be the most influential factors for your response : 

In your case, it seems that V2 and 100 (and perhaps T1) may be the most influential factors on the response D2 v.

• You can also build a model with all the assumed main effects, interactions and quadratic effects from the design, and start refining/reducing your model. With all terms in the model (even if some are not statistically significant), the R² value is around 0,5 but the model is not statistically significant to explain response variability, and R² adjusted is around 0, so this complete model would need some refinement to be useful.
  By testing several models (with JMP/JMP Pro) and removing terms that do not bring valuable information to explain the response, we can have a smaller and statistically significant model (with only terms V2, V2xV2 and 100), but the R²/R² adjusted values are quite low, even if close (0,21 - 0,15).
  Does this sound reasonable ? Were you expecting this ? How can you explain this situation and is it ok with your domain expertise ?

It seems that your response is quite "noisy" and this may explain partly why it is difficult to have high R² values and to find statistically significant terms. 

Maybe it could be interesting in a following step to augment the design on the factors of interest (to be confirmed from the analysis by domain expertise), maybe add other possible factors, and use replication to be able to estimate noise and take it into consideration in the modeling.

I have attached the datatable with the scripts corresponding to the analysis I have mentioned (Predictor Screening, Full model, Reduced model).

[#5]

Hi Victor,

Could you try this one?

Thanks,

Jason

[#6]

[#7]

[#8]

I tried to optimize 7 parameters for maximal respose. I got this when I used the "Maximize Desirability" function. My questions now are: 1. Is this the right way to get optimal parameters? 2. Why did not it push the lines toward the left further for NP and V2 to get even higher D2 v?

Thanks,

[#9 · Accepted solution]

Hi Jason,

The response values for D2v in this new table seem to be a lot easier to work with to build an efficient and useful model.

May I ask how/why the values have changed between previous table and this one ? Measurement recording error, or are the values synthetic data to train yourself ?

By trying the different steps I mentioned before, you can have a (quite satisfying and statistically significant) full or reduced model, explaining a large portion of the response variability. You have to check now that this model is useful and in accordance with your domain expertise, or if the model helps you discover something useful.

1. Yes, using "Maximize desirability" helps you find optimal settings for your factors according to the model you have found.

2. You may have to check that the desirability function is the right one for your use case ; here you have set up the target for D2v as "Match Target" for a value between 0 and 100, so JMP draw a curve with an optimum around 50. If you want to change the desirability and objective, click on the red triangle next to the Profiler, click on "Optimization and Desirability", and then "Set Desirability". You'll be able to change the goal for the response D2v (Minimize, Maximize, Match Target) and specify the different values boundaries for high, medium and low values and their related desirabilities (Desirability Profiling and Optimization) : 

Hope this answer will help you,

[#10]

Hi Victor,

Your expanation wa svery helpful. Thanks a lot!

I made a design with those main effects terms included and analyzed the data with that design. That's the one I uploaded first. In the second dataset, the data was added to the original design with defect.