Leveraging JMP for QbD-Driven Lentiviral Vector Process Development, Part 2 (2025-US-30MP-2515)

Welcome, JMP Community, and thank you for joining our talk. My name is Emmett George, and I am joined here today by Martin Demel. I am a Senior Bioprocess Engineer with Miltenyi Biotec, based out of Gaithersburg, Maryland. Martin is a Principal JMP Systems Engineer with JMP Statistical Discovery, based out of Munich, Germany.

Today, we will represent our approach to using design of experiments to accelerate process development for lentiviral vector manufacturing. This methodology is currently being successfully applied in multiple customer projects and internal programs within in our Process Development Department in Gaithersburg, and was developed in collaboration with Thomas Little Consulting.

In addition to presenting our established data analysis strategy, we will also introduce some complementary insights that from our perspectives, are particularly helpful. The presentation will be divided into two parts. First, a general introduction to the topic, followed by a demonstration of how we use JMP software to evaluate design of experiments.

Before we dive in, I would like to briefly highlight the core competencies we offer as a biotechnology company. Miltenyi Bioindustry operates as a CDMO for cell and gene therapy and as a division of Miltenyi Biotec. In addition to supporting client projects, we also apply our own expertise to develop our own CAR T-cell and T-cell Receptor therapies.

This gives us deep practical experience in JMP-compliant manufacturing of lentiviral vectors and cell products. Our lentiviral vector manufacturing platform is comprehensive. Beyond standard production, our Process Development or PD team provides targeted optimization and specialized characterization studies to support commercialization efforts.

We also offer a standardized analytical platform that can be expanded with additional assays and support assay and tech transfer to clients' internal cell manufacturing facilities. Our regulatory experts provide guided throughout this entire process. Our headquarters is in Bergisch Gladbach, Germany, with facilities for lentiviral vector development and cell processing. We also have manufacturing sites in Gaithersburg, San Jose, and Shanghai.

Our main JMP-compliant vector production facility in Germany is located in Teterow. Large-scale JMP vector production is also supported in Gaithersburg, Maryland, the US, which additionally houses our US process development operations. We use a third generation lentiviral vector packaging system built around the highest safety standards.

Our processes are serum-free, suspension-based, and fully scalable, and capable of being produced in either US or Germany. As a global manufacturer, we draw on extensive experience from producing hundreds of JMP batches using our platform process.

This slide provides an overview of the product life cycle for a cell or gene therapy product as it progresses towards commercialization. The first part of this product life cycle is understanding the product. You begin by gathering knowledge about both your product and the process to be developed. The goal at this stage is to define a set of quality attributes that accurately reflect the product's critical quality characteristics. At this point, you're also going to consider the process parameters that go into the entire process.

The second part of the product development lifecycle is process development and process characterization. Following this process, moves through the phases of process development and process characterization with a multitude of three risk assessments.

Following this, we get into the design space establishment, the third part of the product life cycle, where we ultimately lead to the establishment of both the control strategy and the design space where one can ensure that product quality in terms of safety and efficacy is guaranteed.

The final part of the product life cycle or stage four is Process Performance Qualification or PPQ, also known as the overarching Process Validation Phase. However, this should not be viewed as a final one-time qualification step. Instead, it marks the beginning of what we call continuing process verification or CPV, an ongoing effort to ensure the process remains in a state of control over time.

Today, Martin and I will focus specifically on the development of the design space, the methods we use to define it, and how to identify Critical Process Parameters and establish Proven Acceptable Ranges. A Critical Process Parameter is a process parameter whose variability has an impact on a critical quality attribute and therefore should be controlled or monitored to ensure the process produces the desired quality.

The proven acceptable range is defined as a characterized range of a process parameter for which operation within this range, while keeping other parameters constant, will result in a production of a material meeting relevant quality criteria. Let's begin with defining what is the design space. Shown schematically on the left, the white area represents the in-spec-region, while the shaded region or the shaded area represents the out-of-spec-region.

While not a regulatory requirement for process validation, regulatory authorities strongly recommend defining your design space. It is strongly encouraged and supported by regulatory agencies such as the US Food and Drug Administration, as well as the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use, otherwise known as ICH. The concepts of the design space are outlined in ICH guidelines such as Q8(R2), Q9, and Q10, which provide guidance on pharmaceutical development, quality risk management, and pharmaceutical quality systems, respectively.

Additionally, the FDA's Process Validation Guidance Document emphasizes the importance of understanding and defining the design space as part of the process validation lifecycle. Also outlined in ICH Q8 (Revision 2), the design space is a multidimensional combination of input variables that assure product quality. It is developed through a systematic approach, optimizing critical material attributes and process parameters using tools like design of experiments and JMP software.

The design space provides both flexibility and control by identifying operational limits and potential failure modes, ultimately supporting consistent high-quality production to consistently meet the quality standards and product specifications set out.

How do we generate the design space? We begin with a low-level risk assessment which evaluates potentially influential process parameters. We will list all those process parameters and associate them with specific responses or quality attributes. Then we assess the degree to which parameter variability might affect those responses. If a parameter has a high assumed impact or if there is no prior knowledge to suggest otherwise, it received a high risk score, such in this case as a six or a nine.

Conversely, well-understood parameters with minimal expected influence receive low risk scores such as a zero or three. This helps exclude non-influential parameters from the DoE study. We apply the same rationale to higher-order terms, what we call the interactions in quadratics. Scientific understanding and prior data are key in deciding which to include for study.

Ultimately, this enables us to define a tailored DoE design. Every model term investigated requires an additional experimental run, so an informed risk-based design helps balance scope and feasibility. We build our DoE studies using the custom DoE platform and JMP software. Once designed, the wet lab performs the experiments and collects the data, and the dry lab develops a statistical model describing the relationship between the input variables or factors and the output variables or the responses.

This model is the basis for identifying Critical Process Parameters. Again, parameters whose variation significantly impacts critical quality attributes. We emphasize that risk assessment alone is not sufficient to identify your Critical Process Parameters. Measurable data from design of experiments is required. DoE here is clearly the method of choice as it helps to influence and isolate the influence of every factor and interaction on the critical responses.

To define CPPs or Critical Process Parameters, we examine your scaled estimates, which is otherwise known as the effect size of each factor in the study, as well as the tolerance range between your upper and lower specification limits. A factor is considered critical if it shifts the mean of the response by more than 20% of the tolerance in the design of experiment study. I'll repeat that again. A factor is classified as a Critical Process Parameter if the mean shifts more than 20% of the tolerance range as shown here in the schematic on the bottom left.

Once your Critical Process Parameters or CPPs are identified, the design space is used to define process parameter ranges. Because the design space is dynamic and sensitive to factor settings, we use Monte Carlo simulations to model batch to batch variability and explore the robustness of all your models from the responses one put together.

These simulations account for three key sources of variation. You have model uncertainty, you have factor variability, and then you have residual variation, also known as the Root Mean Square Error, or RMSE from things like your analytical methods that you use to test the responses from your dynamic experiment study. Using this information, JMP simulates thousands of virtual experiments to estimate your process capability. For example, if we observe a defect rate above 33%, this clearly signals the need to tighten parameter limits, as shown in the schematic here on the screen.

In this example, for Factor D, we see a correlation to response A, especially at the higher end of the factor range. There is a higher portion of in-spec runs. We use a script similar to JMP's Design Space Profiler, which originated with JMP 17, to visualize simulated data and failure rates. Green dots, as shown here in the scatter plot matrix, represent your in-specification runs, while the red dots represent your out-of-specification runs.

This visual tool clearly shows how each factor influences your process performance, especially at the range extremes. By manually adjusting your factory limits, as you can see on the left, we've adjusted each of the six factory limits, we refined what we call the Proven Acceptable Ranges or your PARs. In one case, this adjustment reduced your out-of-spec results from the previous 33% that we saw to less than 0. 1%, about 0.05%, a substantial improvement.

These Proven Acceptable Ranges form the basis for robust reproducible manufacturing. We also define Normal Operating Ranges or your NORs, which are tighter limits based on historical performance, equipment capability, and operator variability, always within the bounds of the parts. I'll say that again, NORs are almost always within the bounds of the parts.

At this point, we're going to jump into our JMP software, and we're going to walk through a demonstration of our design of experiments as well as the evaluation of those design of experiments. The example we'll go through today is shown through the slide deck. This example has five responses, six factors. Through the low-level risk assessment that we put together, we then have our six factors, otherwise known as main effects. Then we have a multitude of the interactions and quadratics that were included from in this study.

Through the low-level risk assessment, there were some alias terms or some interactions that were specifically not included due to low influence or low risk. We came out to a DoE design of approximately 16 runs. This is how we just use our custom design. It's the D-optimal design platform in JMP. Once we have performed this in the wet lab, we come back to now the DoE evaluation phase. Here we perform what we call a data quality check at the very beginning before we even build out our fit models. We look at the residual plots here, residual by predicted plot, as well as the studentized residuals.

Here we're looking to see are the residuals randomly scattered around zero, indicating a normal distribution, which is the case in this example. Otherwise, specific patterns in the residuals would indicate the presence of an unknown active effect. We check for outliers using the studentized residuals and the Bonferroni limits. As these limits provide a more conservative approach, reducing the risk of falsely detecting an out-of-trend point, otherwise known as an outlier.

Now, outlier handling. How do we detect an outlier going through our analysis phase? Even when an outlier is detected outside of the Bonferroni limits, so that would be the red limit shown here, this does not necessarily mean that we're going to automatically exclude this data point from the data set. It is recommended to not remove outliers without some scientific rationale, whether that be operator error, method deviation, et cetera, in place. If unsure, perform model comparisons with and without the suspected outlier, supported by justification on why the model without the outlier is more representative.

Now that we've gone through the data quality check, we're actually going to put together our fit model, and this is through model regression. We use what we call backward elimination, stepwise regression, guided by the principles of effect hierarchy, removing non-significant terms from our model. We only remove main effects, if not included in higher order terms. Again, the interaction is in quadratics.

How do we know if they're not included in the higher order terms? You're going to look for this caret symbol here to the right of the P-value column. Anything with a caret symbol or any main effect with a caret symbol means that there's an interaction or a quadratic above it, or which means it's more significant, that's included in that higher order term. Our threshold for removing main effects interactions and quadratics that are insignificant or non-significant from this fit model platform is a P-value here of greater than 0.1, while keeping our P-value for the entire model statistically significant, so less than or equal to 0.05.

Here you see this interaction factor C by factor F. This can be removed from the model. You see factor C as the main effect is also insignificant, does not have the caret symbol, so it's not included in a more significant term, higher order term. Then we come down, and again, I mentioned P-value of 0.1 as our threshold when removing main effects interactions in quadratics. We do have this a boundary condition, factor D by factor F interaction.

Here, again, it's wet on that 0.1 value threshold. What we would do is we would go ahead and remove this guy. If the RMSE increases, so again, the Root Mean Square to Error, when we remove this interaction, then this exclusion would not be warranted because there's some unknown variation that's still present in the model.

However, if the RMSE did decrease when we remove this interaction from the model, then this term removal would be warranted. We're going to go ahead and remove this. However, the RMSE increased by about five units. We're going to go ahead and leave that back in the model. This is our final-fit model.

The RMSE, basically here in conclusion, is used to evaluate model robustness during model term removal. Once our fit models put together, we're going to go up to the red hotspot for the response. You're going to go down to Save Columns, and you're going to save the prediction formula as well as the standard error prediction formula for response. This is going to be used later on to define our Proven Acceptable Ranges or our PORs. But first we're going to go in, and we're going to identify our Critical Process Parameters for the response.

From here, we have a script that was developed with our statistical consultant, as I mentioned previously, called Critical Process Parameters. You're going to click on that script. It's going to open a window. You're going to click on the fit model that you currently have open, the significant fit model that we put together, and you're going to click Open tables or run. My apologies. We do need to have the scaled estimates open. That's one thing that I did forget. Scaled estimates must be open.

As I mentioned previously, these are the half effects. The half effects are used to calculate our Critical Process Parameters. We're going to go back to the Critical Process Parameter script. Now we're going to hit Open tables. Here, you get the initial part of our calculation where you're looking at lower specification limits and upper specification limits where you can basically input these if you have these predefined specification limits for the response.

Specification limits are normally predefined. However, for example, one, we're estimating a unit operation within a process, so some intermediate process step. Limits for this intermediate step might not exist. In such cases, we would consider the at-scale data and use the minimum and maximum values from these batches which pass the final product specifications.

In this case, if this is also not available, we do have a default approach to defining the specification limits of our response, where we use 80% of the DoE process mean for the lower specification limit and 150% of the DoE process mean for the upper specification limit. In this example, we do have a specification limit in place, which in this case is 350 for the upper specification limit. We're going to go ahead and input that here. This is because the response goal of response E is to minimize.

Then we jump back to our script screen. Again, as I mentioned earlier, we use a 20% mean shift to define our CPP acceptance criteria. We have that default approach already entered here in row 3. Now we're going to click on Run. Here the CPPs are then calculated. How do we define the threshold for the CPPs? Again, it's greater than 20% of the tolerance range or 20% of your mean. We're going to look at our model terms here with a mean shift of greater than 20%. This percentage can be adapted again to one's requirements. One might want to use 10%, 30%, but we use the standard default approach of 20%.

The script performs the CPP calculations with a specific multiplier to determine the full effect. Your scaled estimates, also otherwise known as your half effect, are multiplied by a multiplier of either one or two. Continuous main effects and interactions use a multiplier of two, quadratics and categorical factors use a multiplier of one. Then this, so the scale estimates times the multiplier calculates the full effect. Then the full effect is divided by your margin or your tolerance. The absolute value of that is then calculated into a percentage, and that's what's output here.

Then again, if we're greater than 20% of the tolerance range mean shift, you're now looking We're looking at a Critical Process Parameter. If we're less than 20%, we're now looking at a non-Critical Process Parameter. That's how we declare our model term CPPs versus not. This fit model prediction formula, standard error of the prediction formula CPP exercise is performed for all responses in the design of experiment. Then here we would come to the Profiler under Graph. This is where we use the Profiler tool. I already have this prefilled. Once we perform that exercise, we come to the Profiler tool to define our Proven Acceptable Ranges for the process parameters.

As you can see here, some prefilling has occurred on this Profiler. The Simulator tool was opened under the Prediction Profiler hotspot. To show the defect rates, as you can see here, you would then right-click on Columns for your Simulator tool. Go ahead and add in the PPM because we do use PPM in our simulation. Then we also added the respected RMSE values, which is the random noise of the responses as we saw previously.

We then define our factor distributions. You can use triangular, normal, truncated, et cetera, to simulate your process. A triangular distribution was selected here with the respective factor set points from the at scale process located at the peak with the lower and upper ends or the extremes representing the extremes that we studied design. If the defect rates are too high, which in this case, as we also previously saw in our example, was about 33% out-of-spec. We will refine our ranges using the Design Space Profiler, again, with the advent of JMP 17 and now JMP 18.

Previously with JMP 15 and 16, we did have a built-in script using the scatter plot matrix called Edge of Failure. This was basically a tool that displayed the overall design space and response-specific defect rates in real-time, supporting informed decision-making. This is where we'll finalize the Proven Acceptable Ranges, and then all those Proven Acceptable Ranges will then be reviewed with the manufacturing teams for operational feasibility. Now at this point, I'm going to hand it over to Martin to demonstrate the Design Space Profiler in JMP 18.

Thanks, Emmett. Let me just share my screen. I'm starting basically where in the slides where it's 9 and 10 of the presentation where we had this Profiler, basically. I open up the Profiler here once again, what Emmett just showed you.

If you want to understand what's going on in the Profiler, you probably know it already, but you have your responses on the left, you have your factors on the right, on the bottom, and you have the simulations here as well with that. You see, of course, the defect rate, if you have spec limit set for the different responses.

You can optimize that whole thing by using optimize and desirability and maximize desirability. I have done it already here. Then you get maybe to an optimal response here or optimal process setting. However, for the sake of finding proof of acceptable ranges, this might not be the right thing to do because optimization tends to go to the extremes in many cases. If we have a setting at the edge of your design space, that's typically not what you want to have for your proof of acceptable ranges because we don't know anything about on the other side here.

That's something which doesn't really help us here. How can we deal with that? Basically, there's the design space Profiler which can help you with that. The Design Space Profiler makes it quite easy to get your in-spec portion to a higher level by just pressing those move inwards thing. We have the in-spec portion on the left. We have our factor settings here on the bottom again. We see the limits. The current limits of your factor settings, and we see the in-spec portion of your responses and the overall in-spec portion.

Now, let's just move inward and see what happens. We see that there has been changed something on this line. This line represents the lower limit for factor D. This is the upper limit for factor D. We can see on the trace line that this is the effect on the in-spec portion when we change the lower limit to a higher value, it will also result in a higher in-spec portion. That's how you can interpret the thing.

JMP tries to find out what limit change will have the most on your in-spec portion and optimize that further and further and further, and we can move along with that. We can do that as long as we get to a desired in-spec portion. Let's ask for maybe 99% or something. Then we are finally on your limits.

Now we see the in-spec portion is quite high for all the responses. We can send the midpoints to the Profiler. We can also send the limits to the Profiler. Here you have options what limits you want to do. The normal with limits at 3 Sigma is often used in my eyes. But for better visualization, and as I want to be very defensive and conservative in my response. I select the uniform because I also want to be at the edge of the path in the same amount. Or have the simulation in the same amount in the edges of the Proven Acceptable Ranges than within.

Now, we see already what effect it had. It's a great result, but we see that this automatic approach resulted in a very tiny settings or proof of acceptable range for factor D. This may or may not be an issue. Sometimes you can control that type of factor, but often cases this is not really the case. How can we move forward with that?

We also see that some others we can basically work as we like with them. Then your subject matter expertise comes into place where you say, "We may want to reduce something because it's the price of the material or the cost of the energy or whatever it is has a higher impact, so we want to reduce it." If this is quite not important, not critical, then we can set it as like in this whole range.

Now, to illustrate how we can deal with that and get wider ranges, I start with a new report. Of course, you don't need to do that, but for this demo, I found it more useful. Here we start again with the setting and at the Design Space Profiler with all the new settings here or the general settings here. You have options to get around that and see what's going on in this spec, in this design range. That's what you can do with the make and connect random data table. If you open up that, you can also embed the factor space scatter plots and the response scatter plots here and put it in. That will create a random simulated data table.

You can think about additional experience based on your model. They will tell you about the factor space. You see always the factor. Here we see factor D. You see in green and red, similar to what Emmett showed you before, the things which are in-spec and the things which are not in-spec. For the responses, we see the same scatter plot here. We can already imagine that higher settings of D are related to more green and lower values of D are more related to more red.

That's why the algorithm tried to get into this direction because that's very obvious to go in this range. However, you need to understand and not just find the best in-spec portion because that may result in tiny limits. You want to understand what's going on. If we do that and move around, you see already what effect it has on the in-spec portion for a certain factor combination. We see that there is a lot of red going out, so we ignore that, and we get more of the greens in the ratio, at least.

We see why it tends to do this moving inward using the factor setting D. However, we want to see what can we do to eliminate that and understand really what effect it has. What happens if we change the factor E setting from the lower? We see that it has not really an effect, basically. It takes a little bit red here on the bottom, but it also takes a lot of the green within the range here.

It didn't really help too much, at least not for everything. The same is if I, for example, change the upper limit here, we see that it takes away more of the greenish, so we don't want to move that for it. You can follow the traces if you like. If we, for example, go for the factor F and start to say, "We want to bring that to 28, for example, so we have a better setting here," you see how this has an effect on the red here, but also on the other responses and a lot in the lower part here of response D and B range.

That's already quite good. We can do that same for some other factors, for example, we can take the factor C if we want to and see what this does with your effect. This is gaining understanding and not just finding the best in-spec portion. We can change the settings here. Like, for example, do this further and further, and we can see how this has an impact on your ranges and how you can change it so that you are in a good spot of the greenish without even touching the factor D setting, and we can do the rest with the factor D setting and get to a wide range at the end.

I just open up the final results for this study so that you can see that again. We have here the green areas where we are with the factor settings, but also see the response settings, the mainly green stuff here. We can bring that back to the Profiler and see that we have a very low defect rate, but way wider ranges of factor D by reducing the ranges for the other factors.

Finally, we can take a look at the edge of failure plot again in a scatter plot matrix. I just adjusted the scatter matrix image showed a little bit because I like to use the non-parametric density in addition to the dots because it gives me a better feeling where are the most dense regions, especially when I think about data points which are very close to the spec limits, I want to see if there's any rad hotspot here or is it just one single data point? That's basically how you can do that with the design Space Profiler in JMP and the scatter plot matrix. With that, I hand over back to Emmett to wrap up the session.

Thank you, Martin. Martin and I hope that this presentation has demonstrated how we have established a robust data-driven approach to identifying your CPPs or Critical Process Parameters, as well as defining the Proven Acceptable Ranges or PARs using DoE and the Design Space Profiler. We began with a risk-based design tailored to specific process needs. We defined our CPPs based on their actual impact to CQAs, and we established Proven Acceptable Ranges with process simulations and the Design Space Profiler in JMP 18.

This methodology has been successfully applied to our internal products, and Miltenyi Biotec supported a client's successful BLA submission last year. This approval was the first time the FDA approved a T-cell Receptor or TCR therapy for solid tumors. Thank you very much for your attention. Martin and I will now take questions.