Summarizing Adverse Events in Clinical Trials (2020-US-30MP-638)
Kelci Miclaus, Senior Manager Advanced Analytics R&D, JMP Life Sciences, SAS
Reporting, tracking and analyzing adverse events occurring to patients is critical in the safety assessment of a clinical trial. More and more, pharmaceutical companies and the regulatory agencies to whom they submit new drug applications are using JMP Clinical to help in this assessment. Typical biometric analysis programming teams may create pages and pages of static tables, listings and figures for medical monitors and reviewers. This leads to inefficiencies when the doctors that understand medical impacts of the occurrence of certain events can not directly interact with adverse event summaries. Yet even simple count and frequency distributions of adverse events are not always so simple to create. In this presentation we focus on key reports in JMP Clinical to compute adverse event counts, frequencies, incidence, incidence rates and time to event occurrence. The out of the box reports in JMP Clinical allow fully dynamic adverse event analysis to look easy even while performing complex computations that rely heavily on JMP formulas, data filters, custom-scripted column switchers and virtually joined tables.
Auto-generated transcript...
Speaker | Transcript |
| Kelci J. Miclaus | Hello and welcome to JMP Discovery Online. Today I'll be talking about summarizing adverse event summaries and clinical trial analysis. |
| I am the Senior Manager in the advanced analytics group for the JMP Life Sciences division here at SAS, and we work heavily with customers using genomic and clinical data in their research. | |
| So before I go through the summarizing and the details around using JMP with adverse event analyses, I want to introduce the JMP Clinical software which our team creates. | |
| JMP Clinical is one of the family of products that includes now five official products as well as add ins, which can extend JMP to really allow you to have as many types of vertical applications or extensions of JMP as you want. | |
| My development team supports JMP Genomics and JMP Clinical. JMP Genomics and JMP Clinical are respectively vertical applications that are customized, built on top of JMP, that are used for genomic research and clinical trial research. | |
| And today I'll be talking about how we've created reviews and analyses in JMP Clinical for | |
| pharmaceutical industries that are doing clinical trials safety and early efficacy analysis. | |
| The original purpose of JMP Clinical and the instigation of this product actually came through assistance to the FDA, which is a heavy JMP user | |
| And their CDER group, the Center for Drug Evaluation and Research. Their medical reviewers were commonly using JMP to help review drugs submissions. | |
| And they love it. They're very accomplished with it. One of the things they found though is that certain repetitive actions, especially on very standard clinical data could be pretty painful. | |
| Example here is the idea of something called a shift plot which is for laboratory measurements | |
| where you compare the trial average of a laboratory of versus the baseline against treatment groups. In order to create this, it took at least eight to 10 steps | |
| within the JMP interface of opening up the data, normalizing the data, subsetting it out into baseline versus trial, doing statistics, respectively, for those groups merging it back in, then splitting that data by lab tests so you could make this type of plot for each lab. | |
| And that's not even to get to the number of steps within Graph Builder to build it. | |
| So JMP clearly can do it, but what we wanted to do is solve their pain at this very standard type of clinical data | |
| with a one-click lab shift plots, for example. In fact, we wanted to create clinical reviews in our infrastructure that we call the review builder | |
| that are one-click standardized reproducible reviews for many of the highly common standard analyses and visualizations that are required or expected in clinical trial research to evaluate drug safety and efficacy. | |
| So JMP Clinical has evolved since that first instigation of creating a custom application for a shift plot | |
| into a full-service clinical...clinical trial analysis software that covers medical monitoring and clinical data science, medical writing teams, biometrics and biostatistics, as well as data management around the study data involved with clinical trial collection. | |
| This goes for both safety and efficacy but also operational integrity or operational anomalies that might be found in the collection of clinical data as well. | |
| Some of the key features around JMP Clinical that we find to be | |
| especially useful for those that are using the JMP interface for any types of analyses | |
| are things like virtual joins. So we have an idea of a global review subject filter, which I'll show you during the demonstrations for adverse events, | |
| that really allow you to integrate and link the demography information or the demographics about our subjects on a clinical trial to all of the clinical domain data that's collected. | |
| And this architecture, which is enabled by virtual joins within the JMP interface with row state synchronization, | |
| allow you to really have instantaneous interactive reviews with very little to no data manipulation across all the types of analyses you might be doing in a clinical trial data analysis. | |
| Another new feature we've added to the software that also leverages some of the power of the JMP data filter, as well as | |
| creation of JMP indicator columns, is this ability to, while you're interactively reviewing clinical trial data, | |
| find interesting signals that say, in this example, the screenshot shown is subjects that had a serious adverse event while on the clinical trial, find those interesting signals, | |
| and quite immediately, create an indicator flag that is stored in metadata with your study in JMP Clinical | |
| that's available for all other types of analyses you might do. So you can say, I want to look now at my laboratory results for patients that had a serious adverse event versus those that didn't to see if there's also anomalies that might be related to an adverse event severity occurrence. | |
| Another feature that I'll also be showing with JMP Cclinical and the demonstration around adverse event analysis is the | |
| JMP Clinical API that we've built into the system. One of the most difficult things of providing and creating and developing a vertical application | |
| that has out-of-the box one-click reports is that | |
| you get 90% of the way there and then the customer might say, oh, well, I really wanted to tweak it, or I really wanted to look at it this way, or I need to change the way the data view shows up. | |
| So one of the things we've been working hard on in our development team is using JMP scripting JSL | |
| to surface an API into the clinical review, to have control over the objects and the displays and the dashboards and the analyses and even the data sets that go into our clinical reviews. So I'll also be showing some of that in the adverse event analysis. | |
| So let's back up a little bit and go into the meat of adverse events and clinical trials now that we have an overview of JMP Clinical. | |
| There's really two kind of key ways of thinking of this. There's that safety review aspect of a clinical trial where that's typically counts and percentages of the adverse events that might occur. | |
| And a lot of the medical doctors, monitors, or reviewers often use this data to understand medical anomalies, you know, a certain adverse event starts showing up | |
| more commonly, with one of the treatments that could have medical implications. | |
| There's also the statistical signal detection, the idea of statistically assessing our adverse events occurring at an unusual rate | |
| in one of the treatment groups versus the other. So here, for example, is a traditional static table that you see in many of the types of research or submissions or communications around a clinical trial adverse event analysis. | |
| Basically it's a static table with counts percents and if it is more statistically oriented, you'll see things like confidence intervals and p values as well | |
| around things like odds ratios or a relative risks or rate differences. | |
| Another way of viewing this can also be visually instead of with a tabular format so signal detection, looking at say odds ratio or the, the risk difference | |
| might use the Graph Builder in this case to show the results of a statistical analysis of the incidence of certain adverse events and how they differ between treatment groups, for example. | |
| So those are two examples. And in fact, from the work we've done and the customers we've worked with around how they view and have to analyze adverse events, | |
| the JMP Clinical system now offers several common adverse event analyses from simple counts and percentages to incidence rates or occurrences | |
| into statistical metrics such as risk difference, relative risk, odds ratio, including some exposure adjusted time to event analyses. | |
| We can also get a lot more complex with the types of models we fit and really go into mixed or Bayesian models as well in finding certain signals with our adverse event differences. | |
| And also we use this data heavily in reviewing just the medical data in either a medical writing narrative or patient profile. | |
| So now I'm going to jump right into JMP Clinical with a review that I've built around many of these common analyses. | |
| So one of the things you'll notice about JMP Clinical is it doesn't exactly look like JMP, but it is. | |
| It's a combined integrated solution that has a lot of custom JSL scripting to build our own types of interfaces. So our starter window here lays out studies, reviews, and settings, for example. | |
| And I already have a review built | |
| here | |
| that is using our example nicardapine data. This is data that's shipped with the product. It's also available in the JMP sample library. | |
| It's a real clinical trial, looking at subarachnoid hemorrhage. It was with about 900 patients. | |
| And so what this first tab of our review is looking at is just the distribution of demographic features of those patients, how many were males versus females, their race breakdowns, | |
| what treatment group they were given, their sites that the data was taken from, etc. So this is very common, just as the first step of understanding your clinical data for a clinical trial. | |
| You'll notice here we have a report navigator that shows the rest of the types of analyses that are available to us in this built review. | |
| I'm going to walk through each of these tabs, just quickly to show you all the different flavors of ways we can look at adverse events with the clinical trial data set. | |
| Now, the typical way data is collected with clinical trials is an international standard called CDISC format, | |
| which typically means that we have a very stacked data set format. Here we can see it, where we have multiple records for each subject indicating the different adverse events that might have occurred over time. | |
| This data is going to be paired with the demography data, which is one row per each subject as seen here in this demographic. So we have about 900 patients and you'll see in this first report, we have about 5,000 or 5,500 records of different adverse events that occurred. | |
| So this is probably the most commonly used reports by many of the medical monitors and medical reviewers that are assessing adverse event signals. | |
| What we have here is basically a dashboard that combines a Graph Builder counts plot with an accompanying table, as they are used to seeing these kind of tables. | |
| Now the real value of JMP is its interactivity and that dynamic link directly to your data so that you can select anywhere in the data and see it in both places. Or more powerfully, you can control your views with | |
| column switchers. | |
| Now here we can actually switch from looking at distribution of treatments to sex versus race. You'll notice with race, if we remember, we had quite a few that were | |
| white in this study, so this isn't a great plot when we look at it by percent or by counts, so we might normalize and show percents instead. | |
| And we can also just decide to look at the overall holistic counts of adverse events as well. | |
| Another part of using this as this column switcher is the ability to you know categorize what kind of events those were. Was it a serious adverse event? What was the severity of it? | |
| Was the outcome that they are when they recovered from it or not? What was causing it? Was it related to study drug? | |
| All of these are questions that medical reviews will often ask to find interesting or anomalous signals with adverse events in their occurrences. | |
| Now one of the things you might have already noticed in this dashboard is that I have a control group as column switcher here that's actually controlling both my graph and my table. So when I switched to severity, this table switches as well. | |
| This was done with a lot of custom JSL scripting specifically to our purposes, but | |
| I'll tell you a secret, in 16 the developer for column switcher is going to allow us to have this type of flexibility so you can tie multiple platform objects into the same columns switcher to drive a complex analysis. | |
| I'm going to come back to this occurrence plot, even though it looks simple. Here's another instance of it that's actually looking at overall occurrence where certain adverse events might have occurred multiple times to the same subject. | |
| I'm going to come back to these but kind of quickly go through the rest of the analyses and these reviews before coming back to some of the complexities of the simple graph builder and tabulate distribution reports. | |
| The next section in our review here is an adverse event incident screen. So here we're making that progression from just looking at counts and frequencies or possibly incidence rates | |
| into more statistical framework of testing for the difference in incidence of certain adverse events in one treatment group for another. | |
| And here we are representing that with a volcano plot. So we can see actually that phlebitis, hypotension and isothenuria occur much more often in our treatment group, those that were treated with nicardipine, versus those on placebo. | |
| So we can actually select those and drill into a very common view for adverse events, which is our relative risk for a cell plot as well, | |
| which is lots of lot of times still easier to read when you're only looking at those interesting signals that have possibly clinical or statistical significant differences. | |
| Sometimes clinical trials take a long time. Sometimes they're on them for a few weeks, like this study was only a few weeks, but sometimes they're on them for years. | |
| So sometimes it's interesting to think of adverse event incidents differences as the trial progresses. We have this capability as well within | |
| the incidence screen report where you can actually chunk up the study day, study days into sections to see how the incidents of adverse events change over time. | |
| And a good way to demonstrate that might be with an exploding volcano plot here that shows how those signals change across the progression of the study. | |
| So another powerful idea with this, especially as you have longer clinical trials or more complex clinical trials, | |
| is instead of looking at just direct incidence among subjects you can consider their time to event or their exposure adjusted rate at which those adverse events are occurring. | |
| And that's what we offer within our time to event analyses, which once again, shown in a volcano plot looking here using a Kaplan Meier test at | |
| differences in the time to event of certain events that occur on a clinical trial. One of the nice things here is that you can select these events and drill down into the JMP | |
| survival platform to get the full details for each of the adverse events that had perhaps different time to event outcomes between the treatment groups. | |
| Another flavor of time to event is often called an incidence density ratio, which is the idea of exposure adjusted incidence density. Basically the difference here is instead of using some of the more traditional proportional hazards or Kaplan Meier analyses, this is more like a | |
| a poisson style distribution that's adjusted for how long they've actually been exposed to a drug. And once again here we can look at those top signals | |
| and drill down to the analogous report within JMP using a generalized linear model for that specific type of model with an adverse event signal detection. | |
| And we actually even offer some really complex Bayesian analyses. So one of the things with with this type of data is typically adverse events | |
| exist within certain body systems or classes...organ classes. And so there is a lot of posts...or prior knowledge that we can impose into these models. And so some of our customers, their biometrics teams decide to use pretty sophisticated models when looking at their adverse events. | |
| So, so far we've walked from what I would say consider pretty simplistic distribution views of the data | |
| into distributions and just count plots of adverse events into very complex statistical analyses. | |
| I'm going to come back now, back to what is that considered simple count and frequency information and I want to spend some time here showing the power of JMP interactivity that we have. | |
| As you recall one of the differences here is that this table is a stacked table that has all of the occurrences of our adverse events for each subject, | |
| and our demography table, which we know we have 900 subjects, is separate. So what we wanted was not a static graph, | |
| like we have here, or what we would have in a typical report in a PDF form, but we wanted to be able to interactively explore our data | |
| and look at subgroups of our data and see how those percentages would change. Now, the difficulty is that the percent calculation needs to come from the subject count in a different table. | |
| So we've actually done this by formula...like creating column formulas to dynamically control recalculation of percents upon selection, either within categorizing events | |
| or, more powerfully, using our review subject filter tool. So here for example, we're looking at all subjects by treatment. | |
| Perhaps serious versus not serious adverse events, but we can use this global data filter which affects each of the subject level reports in our review | |
| and instantaneously change our demography groups and change our percentages to be | |
| interactive to this type of subgroup exploration. So here, now we can actually subgroup down to white females and see what their adverse event percentage and talents are, or perhaps you want to go more granular and understand for each site, how their data is changing for different sites. | |
| So what we really have here is instead of a submission package or a clinical analysis where the biometrics team | |
| hands 70 different plots and tables to the medical reviewer to go through, sift through, they have the power to create hundreds of different tables and different subsets and different graphics, all in one interface. | |
| In fact, you can really filter down into those interesting categories. So if they were looking say at serious adverse events | |
| and they wanted to know serious adverse events that were related to drug treatment very quickly, now we got down to a very small subset from our 900 patients to about | |
| nine patients that experienced serious adverse events that were considered related to the treatment. So as a medical reviewer this is a place where Ithen might want to understand all of the clinical details about these patients. | |
| And very quickly, I can use one of our action buttons from the report to drill down to what's called a kind of a complete patient profile. | |
| So here we see all of the information now, instead of at a summary level, at a subject individual level of everything that occurred to this patient over time, | |
| including when they had serious adverse events occur and their laboratory or vital measurements that were taken alongside of that. | |
| One of the other main uses of our JMP Clinical system along with this medical review, medical monitor is medical writing teams. So another way of looking at this instead of visually in a graphic or even in a table which these are patient profile tables, you can actually go up here | |
| and generate an automated narrative. | |
| So here we're going to actually launch | |
| to our adverse event narrative generation. Again, one of the benefits and values of our JMP Clinical being a vertical application relying on standard data | |
| is that we get to know all the data and the way it is formatted up up up front, just by being pointed to the study. So what we can do here is actually run this narrative | |
| that is going to write us | |
| the actual story of each of those adverse events that occurred. | |
| And this is going to open up a Word doc | |
| that has all of the details for this subject, their demography, their medical history, and then each of the adverse events and the outcomes or other issues around those adverse events. | |
| And we can do this for one patient at a time or we can actually even do this for all 900 patients at a time and include | |
| more complex details like laboratory measurements, vitals, either a baseline or before. And so, medical reviewers find this incredibly valuable be able to standardly take data sources and not make errors in a data transfer from a numeric table to an actual narrative. | |
| So I think just with that you can really see some of the power of these distribution views, these | |
| count plots that allow you to drill into very granular levels of the data. This ability to use subject filters to | |
| look either within the entire population of your patients on a clinical trial or within relevant subgroups that you may have found. | |
| Now one thing about the way our global filter works through our virtual joins is this is only information that's typically showing the information about the demography. | |
| One of the other custom tools that we've scripted into this system is that ability to say, select all subjects with a serious adverse event. | |
| And we can either derive a population flag and then use that in further analyses | |
| or we can even throw that subject's filter set to our global filter and now we're only looking at serious...at a subject who had a serious adverse event, which was about...almost 300 patients on the clinical trial had a serious adverse event. | |
| Now, even this report, you'll see is actually filtered. So the second report is a different type of aspect of a distribution of adverse events that was new in our latest version which is incidence rates. | |
| And here, the idea is instead of normalizing or dividing to calculate a percent by the number of subjects who had an event. If you are going with ongoing trials or long trials or study trials across different countries that have different timing startup times, | |
| you might want to actually look at the rate at which adverse events occur. And so that's what this is calculating. So in this case, we're actually subset down to any subjects that had a serious adverse event. And we can see the rate of occurrence | |
| in patient years. So for example, this very first one, see, has about a rate of 86 occurrences in every 10 patient years on placebo versus 71 occurrences | |
| In nicardipine. So this was actually one which this was to treat subarachnoid hemorrhage, intracranial pressure increasing likely would happen if you're not being treated with an active drug. | |
| These percents are also completely dynamic, these these incidence rates. So once again, these are all being done by JMP formulas that feed into the table automatically | |
| that respect different populations as they're selected by this global filter. So we can look just within say the USA and see the rates and how they change, including the normalized patient years based on the patients that are from just the USA, for example. | |
| So even though these reports look pretty simple, the complexity of JSL coding that goes beyond building this into a dashboard is basically what our team does all day. | |
| We try to do this so that you have a dashboard that helps you explore the data as you know, easily without all of these manipulations that could get very complex. | |
| Now the last thing I wanted to show is the idea of this custom | |
| report or customized report. So this is a great place to show it too, because we're looking here at adverse events incidence rates. And so we're looking by each event. | |
| And we have the count, or you can also change that to that incidence rate of how often it occurs by patient year. And then an alternative view might be really wanting to see these occurrences of adverse events across time. | |
| And so I want to show that really quick with our clinical API. So the data table here is fully available to you. | |
| One of the things I need to do first off is just create a numeric date variable, which we have a little widget for doing that in the data table, and I'm going to turn that into a numeric date. Now you'll notice now this has a new column at the end | |
| of the numeric start date time of the adverse event. You'll also notice here is where all that power comes from the formulas. These are all actually formulas that are dynamically regenerated based on populations for creating these views. | |
| So now that we have a numeric date with this data, now we might want to augment this analysis to include a new type of plot. | |
| And I have a script to do that. One of the things I'm going to do right off the bat is just create a couple extra columns in our data set for month and year. | |
| And then this next bit of JSL is our clinical API calls. And I'm not going to go into the details of this except for that it's a way of hooking ourselves into | |
| the clinical review and gaining access to the sections. So when I run this code, | |
| it's actually going to insert a new section into my clinical review. And here now, I have a new view of looking at the adverse events as they occurred across year by month for all of the subjects in my clinical trial. | |
| So one of the powers, again, even with this custom view is that this table by being still virtually joined to our main group can still fully | |
| respond to that virtual join global subject filter. And so just with a little bit of custom API JSL code, we can take these very standard out-of-the-box reports and customize them with our own types of analyses as well. | |
| So I know that was quite a lot of an overview of both JMP Clinical but, as well as the types of clinical adverse event analyses that the system can do and that are common for those working in the drug industry or pharma industry for clinical trials, | |
| but I hope you found this section valuable and interesting even if you don't work in the pharma area. One of the best | |
| examples of what JMP Clinical is is just an extreme extension and the power of JSL to create an incredibly custom applications. | |
| So maybe you aren't working with adverse events, but you see some things here that can inspire you to create custom dashboards or custom add ins for your own types of analyses within JMP. | |
| Thank you. |
