Hello, everyone. I'm Holly Oxley. I am an Application Scientist and Manager at Exact Sciences, and I'm here today to share with you an investigation where we use JMP to determine root cause.

In a lot of our work, investigations are a big part of what my team does. In that, we realize that investigations are a little bit like un-built puzzles. What I mean by that is they're made up of component parts.

They vary in complexity. They require looking at both small elements and large elements. Pieces can typically be grouped and related to derive additional information. They will typically have some pattern that is discoverable. They almost always require some level of experimentation. At the end, the work is a work of data art.

For this specific scenario, I'll be going over how these puzzle pieces fit together and how JMP helped us accomplish the understanding of what was going on, which led to a true root cause of the issue.

First, we have component parts and varying complexity, which I put akin to the number of puzzle pieces. Data for investigations comes from a ton of sources within our laboratories, and that can include things like reagent data, instrumentation data, environmental data, operator observations, and trending data.

These data sets and investigations themselves can also vary in complexity. My team uses JMP for result outputs as small as a single run, all the way up to five plus million lines of data with over 200 columns. The use of JMP imports range from single files file imports all the way up to full SQL query files connected directly to databases.

The case study in this specific one is a smaller one, but it is the case of increased invalid sample rate. In looking at those small and large elements, we see that there are different shaped pieces.

Some of those small elements include things like specific date ranges, run data, and re-agent information, where some of the larger elements include things like site comparisons, if I'm running at multiple labs, knowledge of other issues that are occurring within the organization or within current investigations, historical trending, that's a big one for us. We have a lot of data, that historical trending is very important. Then instrumentation and overall sample impact.

Analysis of our data through JMP includes, but is clearly not limited to, multiple file imports, distributions, tabulation tables, fit Y by X tables, variability charts, Graph Builder, recoding, time and date modifications, and formula creations. We also employ scripting wherever applicable to make our jobs and our data analysis as efficient as possible.

In this case, the lab escalated an issue where we noted an increase in samples returning invalid results due to insufficient sample qualities being detected in the first few steps of the run. We group or relate elements of that puzzle to understand and determine what additional information we may be able to derive that will help us in this investigation.

In doing that look, we did group together different pieces of, I guess, control data, calibration data, instrumentation data, and came to the conclusion that none of those specific things were the cause of the current issue.

Once these pieces were ruled out, we were allowed to evaluate other potential connections, such as an influx of issues with consumables that we had noted in other investigations, and therefore it allowed us to begin things like lot trace back to understand where used.

Instrumentation maintenance events didn't preclude the same instrument from having another issue shortly after the completion of the original maintenance event, and so because of that, we were able to determine that instrumentation could be a contributing factor, but not likely the root cause.

When we evaluated the smaller elements like calibration and control, failure data or reagent failure information, we did note that the trending of the data that we had didn't show any increase in invalid rates from a single run. It was more on the single sample level.

Reagent information showed us that there were no changes to the specific reagents that a single site was using, and we were seeing the issue at two sites that were using different sets of reagents.

Once we got to that point, how we ruled out instrumentation was showing discernible patterns. What we were seeing was that despite a large quantity of the instrumentation being tagged out at times due to this issue, they were fixing them and they that are not coming back into service with any better performance than we had seen prior to the fixes.

This led us to looking further into the patterns. What we were able to do is use JMP Graph Builder and the Map Shape feature to allow for visualization of where the invalid samples were occurring. In this small picture, you can see that there are red and blue dots within each of the wells of the 96 well plate.

You can see that there's a bit of a pattern. The red dots are valid samples and the blue dots are invalid samples. You can see that on the right side of the plate, there are clearly a lot more blue dots than the red dots. They also are a little bit following certain columns, too. It's not just the rows, but also the columns. My team then requested that the next high sample invalid occurrence be reported immediately.

Knowledge of the instrumentation combined with this data pattern led us to consumable material lot trending, which we had already started the lot trace back based on another investigation that was occurring. Of course, there's always some level of experimentation.

In this case, a single instrument was selected to attempt to reproduce the issue in the laboratory because in our hands, we were unable to produce the issue with materials from the side of the investigating team. We went into the laboratory to use their materials.

The intermittent nature of the problem was able to be reproduced with approximately 20 of the consumable parts that we had a question on. Experimentation did show a distinct, visually identifiable cause during the instrumentation placement of the consumable in question. Then that was confirmed subsequently with the data that we ran on those plates during the troubleshooting.

Once this was all done, we were able to finally determine that our Deep Well Plate was the true issue behind the invalid rate uptick that we were seeing. The manufacturer then subsequently confirmed that the plates that we had acquired and kept as part of this investigation were actually outside of the specification tolerance ranges as defined by the supplier, which in the hands of our laboratory partners, resulted in inappropriate seeding of the plate on the instrumentation, thus the observed clustering of the samples generating the invalid results.

As a whole, of course, the art of simplicity is a puzzle of complexity, and that's exactly what we saw in this very specific situation. Since we used JMP to be able to do this investigation, I just wanted to share just a couple of tips on what we saw, how we determine we could use this tool. There are multiple map types found within the JMP community. You can even create your own. I included a post here on how to create your own type of layout.

The second post is actually the Well format that we used to do this investigation. It was the Plate96Welsl-Name.jmp file, which can be found in the associated link below.

That ends my slideshow. Let me quick grab the stuff for the actual demonstration. How I used this when I was actually performing this analysis. This is an example data set of what I have used to do this determination with the map.

You can see here that there's a plate ID, a Well number, and then a specific column called Well, and then sample status and the setup serial number. The setup serial number is associated with the instrumentation itself. The sample status is pretty clear that's the valid or invalid result.

Then well and well number, because of the way that our software is set up, I had to create a new column for well in order to use the map feature that I wanted to use. In doing so I labeled just what our order in the 96 Well Plate would be and gave that the A1 through H12 designation in this Well column, which is what is required for use by the actual map.

Then once you… oops, apologies. Once you're in this specific column, and you created that information, then you can actually go to the column info. Then my specific modeling is character and It's nominal for this example, but then you go to the Column Properties drop down, and then you'll find Map Role. Select Map Role.

You have an option here of shape, name, use, or shape, name, definition. I will stay with shape, name use, and then I locate my specific file that I would like to pull from, which will be… okay, hold on, 96 Well Plate. Then I will define the column that I named Well as that specific column I want to apply this map shape to. Then I click okay.

Then from there, I will go into Graph Builder, and I will drag that new column that I just characterized for the shape into Map shape. Now you can see that I have a 96 Well layout. From there, for this specific analysis, I've dragged sample status into the color and serial or set up serial number into the page. Now I can see over the number of runs that I have representation from this specific serial number, differing populations are showing up of valid or invalid results.

Then from there, I just wanted to look by run. I applied a local data filter, which then included sample set up, or status rather, and I just selected the two items that I really wanted to see.

Then within the Map itself, I went to Map Shapes and changed it to points so that when I had runs that were done more than once on an instrument, I could see all of the representative data that was represented in that specific table.

You can see here on the top, this is a single run, and you can see that patterning of the invalids. On the second example here, there is a different patterning of invalids and also multiple runs. This is an example of a pre and a post test where we knew that there was a problem. We tested prior to maintenance and then retested after maintenance.

You can see that the population didn't shift a lot. There was a little bit of movement, but both of these scenarios show us two things about how the plate was seated. In this scenario, the plate was likely seated on our instrumentation so that the right side of the plate consumable was actually tipped, causing this specific error to occur.

Then on the second example, the pre-and-post maintenance, not really showing any change, but continuing to show that that plate placement was extremely important, where you're seeing those wells reproduce in the same type of area even after a maintenance event.

Once we were able to really see this and root cause it to that consumable, then we had grounds to reach out to our supplier and have a discussion with them, which then did return that information that I presented in the last slide with their dimensions being out of specification. That's all I have here. Thank you all for sitting and having fun with Graph Builder, and I hope this was helpful.