How can I add extra runs into the designed runs from Definitive Screening Design?

[#1]

Dear JMP experts, 

I am Chau Dang, and new to JMP. I used JMP to make design for my experiments. I selected a Definitive Screening Design (DSD) with 6 variables to create 17 runs for my experimental set. During the time of conducting the experiments, I made 7 extra runs (not designed by DSD) with the same variables. In the end, I want to analyze the results from 17 runs (from DSD) and 7 extra runs (fake factors?) together. However, when I add 7 extra runs into the table of 17 runs, the model (Fit Definitive Screening) does not understand and provides no result. I have tried to use "Analysis --> Fit model" for total of 24 runs; however, only significant parameters are determined but the second order was not considered.

It would be nice if you could suggest me solutions to solve this problem. I dont want to waste 7 extra runs. 

Thank you so much for your great support!

I am waiting for your comments.

Best regards

Chau Dang

[#2 · Accepted solution]

Hi @danghuyenchau89,

Welcome in the Community !

Your 7 runs are replicates of original runs in your 17-runs design. Adding replicates to a Definitive Screening Design destroys the specific foldover structure of the DSD, and so you can't use "Fit DSD" platform anymore (as it requires a foldover structure for the analysis). See Definitive screening Design - Replicates topic and answer by @Mark_Bailey. It's recommended to use the "Extra Runs" option in the DSD platform if you have enough experimental budget to increase sample size and help select terms in the model : Effective Model Selection for DSDs

You can still use the "Fit Model" platform, specifying all main effects, 2-factors interactions and quadratic effects of your factors, and use a variable selection method, like the Stepwise personality of the "Fit Model" platform (with JMP) or use Generalized Regression models (with JMP Pro) in the same platform. You might also be interested in these discussions :
How to treat replicates and failed runs in a definitive screening design.

Definitive Screening Design Questions (about the use and impact of replicates)

Please find attached your datatable with scripts to launch Stepwise platform and a model created with "Fit Model".

I hope this answer will help you,

[#3]

Dear Victor,

hank you so much for your answer! I found it very useful.

I followed your instructions and obtained the results I was aiming for. What I realized is that the significant parameter interactions need to be selected in the "Fit Model" based on those shown in the "Fit Definitive Screening". Otherwise, all parameter interactions will be considered, which can lead to different results for the p-values and R-squared (Rsp) of the model.

Another question that comes to mind relates to my experiments, where I converted biomass into biochar. The solid yield and heating value of the biochar are well predicted by the model. However, there is one indirect response (e.g., ash composition of biochar), which is analyzed by combusting the biochar at 550°C to determine the ash composition. This indirect response shows a very high RMSE (Root Mean Square Error) of 2930 in the model.

How should I interpret this high RMSE? Can I conclude that this model is not suitable for predicting this indirect response?

Best regards

[#4]

Sorry @danghuyenchau89, but there is not enough information about the model, response and context of this response measurement to help you interpret/conclude about this specific value of RMSE.

Some questions to help you :

• What is the standard deviation of this measurement ? Do you already have some tests or a value indicating the typical dispersion/uncertainty of these measurements ? This value can help compare the prediction performance of your model to the measurement uncertainty. Depending on how closely related the values are, you can have a better understanding about how reliable and useful the model can be.
• Are all possible factors linked to this indirect response in the DoE study ? Maybe some other factors not taken into consideration in the DoE can be helpful to explain and predict this response ?
• How controlled and stable are operating conditions ? Particularly, how controlled is the temperature setting for combustion ? Can you measure the temperature during the experiment to account for it in the analysis (maybe as a covariate) ?
• For the modeling part, did you check the distribution of your response and choose an adequate one (for example in the Generalized Regression platform) ? Did you check the residuals ? A non-adequate distribution can have serious impact on the modeling, breaking the assumptions of linear regression and inflating the error of the model. In your previous example, choosing a LogNormal distribution for GenReg modeling or applying a Box-Cox transformation can improve the RMSE (same terms in the two models, with Box-Cox trasnformation on the right, the RMSE is reduced from 1623 to 1308):
  
• Also as I don't have information about your modeling, there might be several things to check : what are your model selection criteria and metrics (R², R² adjusted, RMSE, p-values, Information criterion, ...), what are the terms introduced in the model, etc...  

I'm sure there may be many other considerations to take care, but these few quick questions can help you.