@SamGardner
Instead of writing you a PM I thought it also might be more helpful for other JMP users if we keep the discussion public. I also google very frequently questions I have and I'm always happy if I find a post in the community
I spent now some time with prediction profiler and you guys have done a really great job there. This feature is really great to tackle QbD or PAR requests from authorities such as the FDA or EMA. However, at the moment is see two big problems why I cannot use the feature.
1. As I already mentioned in my last post, I'm afraid that the model uncertainty in the simulation is underestimated. @John_Sall explained at the end of this video how to incorporate the model uncertainty within PredFormula. I could reproduce his example. But, the used RMSE of the model does not consider the correlation structure within the model. You anticipate that the model uncertainty across the whole investigated design space (lets call the design matrix X) is constant. This however is in general not true, since most of the models are not completely orthogonal regarding their input-parameters. See below a short equation how the uncertainty of an estimated coefficient is dependent on the design correlation structure.
Since JMP allows easily to save the PredSE formula of each model, where the correlation structure is considered. I wonder why the formula is not used for the simulation within the DesignSpace profiler? Instead of using a constant SD, simply connect the PredSE formula here and the issue should be solved.
2. The other point is that by using the RMSE of the model as the SD for the simulation within the DesignSpace Profiler, the uncertainty in the model mean prediction is neglected. From my understanding the simulation anticipates that the mean prediction of the model has no error and only considers the uncertainty around this model prediction. To make my point a bit clearer, see the following formula for the calculation of a prediction interval (just copy/pasted from a paper):
At the moment you are only considering the first part of the equation but not the second part (i.e. the standard error).
To sum up my points. I think at the present status, the DesignSpace profiler underestimates the true variability and thus allow to wide ranges for the input-parameters. I think both points could easily be fixed and I would much appreciate your feedback (also from @Phil_Kay if he has some time during his train trip )