Thanks to JMP DOE for the Next Level of Assay Development

Hello, everyone. I'm Swetlana Berger, and I'm a mathematician and physicist by training with a PhD in quantitative genetics. I fell in love with JMP from my first working day in the R&D department at Novartis vaccines. Working nonclinical statistics supporting bioassay development and maintenance, I appreciated the flexibility and intuitive approach of JMP. Currently, I'm working at UCB supporting all statistical aspects in biomarker and immunogenicity, or anti-drug antibody assay development and validation.

UCB is a global biopharmaceutical company dedicated to transforming the lives of people living with severe diseases. At UCB, we are not just about developing medicine. We are about making a real difference. We are committed to making a positive impact on our patients and the planet.

Now let's talk about the therapeutic areas we focus on. UCB has a deep heritage in urology, providing a range of solutions for people living with neurological disorders, including epilepsy and Parkinson's disease, for example. We are committed to expanding our leadership and capabilities into new frontiers with substantial unmet needs, for example, in Alzheimer's disease and systemic lupus.

In addition to neurology, they are also dedicated to immunology. At UCB, they embrace the possibility of creating a world free from the burden of immune mediated inflammatory diseases, such as osteoporosis, rheumatoid arthritis, myasthenia gravis, and Crohn's disease, for example.

UCB operates in 36 countries and deploys around 9,000 people. Our commitment to research and development is evident, as we allocate 30% of our revenue to support these efforts. I am a part of a larger team focused on early development and gene therapy statistics.

Today, I would like to introduce you to the strategies put in place to establish the use of design of experiments, or DoE method, as the gold standard in biomarker and immunogenicity assay development at UCB. DoE is a method that allows us to plan and analyze experiments in a structured way. It helps us to understand and optimize an experimental system by investigating all possible influential factors in a single comprehensive experiment, providing additional information such as interactions, for example.

Working in bioassay development means working in a non-GxP environment, where the involvement of a statistician is not mandatory. This is an opposite to the clinical environment, where all the analysis and steps are highly regulated, and GxP, which could be, for example, GCP, that good clinical practices, or GLP, good laboratory practices, are the key.

In our case, in particular for biomarker bioassays, we are not regulated. The assumption is that the lab scientists should be able to handle the statistical questions. Of course, all the scientists had one or two statistical lectures during the studies, but of course, it's not comparable with the package of experience and knowledge we statisticians could bring in this environment.

This could lead also to limitations and drawbacks due to the lack of innovative statistical methods. The added value from the involvement of a statistician from the early stage on, such as increased efficiency, time savings, delivery of first time right results, and also development for fit-for-purpose assay developed really accounting and targeting the intended use, needs to be demonstrated to developers to get involved.

After a while, we realized that the best strategy to get earlier and deeper involved in assay development was to work in pilot projects. Developers who participated in pilot projects act as multipliers to the rest of the developer team, sharing those successes and sharing those learnings and experience from collaborations with statisticians. I should say the strategy worked out. The response was huge.

The increased need for statistical support, not only for DoE, but also for DoE planning and analysis, led to an increase in workload for the statistical team. We were literally flooded with requests for collaboration. How do we handle this workload increase with our existing limited resources?

Now let us take a step back and review the DoE process and pilot processes. Initially, lab scientists review the entire experiment, all the parameters, and decide on objectives and critical parameters they would like to optimize. They discuss with the statistics team the possible factors and responses to be included in the DoE.

In this slide, the colors in the scheme represents the players involved. Yellow stays for lab scientists and blue for statisticians, just for your information. This will play a role. In the next step, statisticians generate possible experimental designs for optimization DoE. Although the statistical effort is highlighted in blue here on this step, we usually have some iterations of planning and discussions with lab scientists, adjustments to designs, and further discussions. For the next step, and we are in the executive step, at this stage, experiments planned and discussed in the previous steps are implemented and performed in the lab. This is the data generation step, followed by lab scientists only.

However, in case for some reasons the DoE experiment could not be performed exactly as planned and we need to adjust and to change some aspects, we could have intermediate discussion between statisticians and lab scientists at any time.

At the end of the experimental phase, lab scientists compile the data for statistical analysis in a predefined format and transfer data to statistician for analysis. Next step in the phase of the DoE evaluations, statisticians work on data analysis, generating outputs and summaries for all responses of interest.

Since we are in the situation where different assays and different type of assay have also different area of interest, one is more focused on the low range of measurement, requiring a higher sensitivity of the assay, while another one is more focused on specificity on the readout generated on the whole range. This means there is no default strategy for the analysis. In each specific case, in the first instance, we try different types of evaluation applying to the data to find the most appropriate, the most informative one.

Finally, identified methods are applied to all responses. This means from case to case that the number of responses could vary between 5 and 10, but it could be also more depending on the intended use of the assay. So we have, say, 10 similar analysis applied to the different responses at the end, which are summarized in a huge final table with all the outcomes. This table is then reviewed in the last step.

The last step in the DoE experiment is the most exciting one. The lab scientists, together with statisticians, are spending hours and hours reviewing and interpreting the analysis outcomes, giving weight to the result observed based on their extensive background knowledge and accounting for factors outside of the statistician's scope. For example, an optimal condition should be also evaluated with aspect to the feasibility. Feasibility in the lab is accounting for working hours of analysts, and limitation of assay as well, and assay readers, of course.

As you might have guessed, all steps from planning to analysis and summaries are performed using JMP. This is the key in our case. Since handling JMP compared to other statistical software is very intuitive and user-friendly, it is an attractive tool for beginners, and this was the key in the situation we found us.

We decided to start a new strategy to deal with the increased workload by outsourcing many steps to lab scientists, allowing them to work on DoE in a semi autonomous way. Bioassay stat team is providing consultations at all stages on demand. But to achieve that this strategy works, we definitely need a solid training on JMP-based DoE planning and statistical evaluations.

This is what we have done for that. The bioassay statistics team created a series of four training sessions conducted over 4 days with sufficient time in between. As you see here displayed on the slides, we started on day 1 with an introduction, going first through the statistical theory and DoE principles, followed by JMP basics. Really the very, very basics, how we open the file, how we close the file, saving, the simple manipulation of the data, the simple visualizations. Then we also touched the topic of preparation of DoE experiment, how to structure this discussion of experimental conditions to be prepared and to extract the important factors to be optimized.

The second day, we focused on the planning aspects. First, we provided some statistical theory again on the classification of TV designs, followed by the practical planning via JMP platform. We just went step by step through different options and showed how a DoE could be planned by means of JMP.

On the third and fourth day, we spent time on statistical analysis. Since the topic is that huge, we split the analysis in 2 days. The first day was focusing on the descriptive analysis and visualizations, like showing also how to do the scatter plots, histograms, box plots, distributions, and so on. Then on the last day of the training, we focused on the statistical interference analysis, showing how to build the main effect models, how to account for interaction, how to model interaction, what is the simple linear regression, how we can apply this.

At the end, finally, we decided on a fifth session, which is not displayed here, but it's easily explained. We wanted to have a session 2 months later after the training, which showed all the steps of the analysis, which we see now from session 1-4, but performed following the new strategy on a practical example. This was the first case of DoE performed following the new strategy.

At the end, as we delivered this fifth session, the session was partially delivered again by lab scientists, and they shared their learnings, experiences. At the end, it was the most helpful part, I think, like the final closing part of the training. I would recommend also to do the fifth session as well after you went through all the details of how to.

Now I'm already at the end of my presentation, and I would love to face the why and how. Why we should apply the DoE? We know that DoE provides a robust conclusion that serves business decisions. This allows us to make data-driven decisions at very early stage of development, saving time and money. We know that DoE doesn't necessarily involve less experiments than one factor at one time approach, which is very often used in the labs, and which we try now to replace by DoE. But this is the short-term perspective.

However, we know that the DoE provides a higher quality of information. They have a better precision. They are depicting the whole picture. They have more information, not only the better, but also that just the content is much, much higher. We'd rather go provide right the first time and having less rework and less costly drawbacks in assay development.

To the question of how, what we learned is that in first step, it showed in pilot studies some few positive examples of collaboration of developer and statisticians. Then we found a way to introduce use of DoE on a large scale, enabling developers to manage the whole flow in semi autonomy, consulting statistical team when needed.

In particular, we invested additional time in training of so-called key users, who covered the first level of question at the end in developers' teams, giving us the opportunity getting involved at the level of more complex problems. We learned that JMP-based DoE-specific training is absolutely necessary. There is no way how people could learn themselves with no training, just by trial and failure. But in addition, it's a more general training on JMP is very beneficial, and we would recommend to do it as well.

Now I would like to thank you for your attention, and I hope this helps. I'm excited also to share more about our journey and achievements during the conference. Thank you very much.