Hello, everybody. Thank you for coming to our talk today. The title of our talk is Hop to It: Crafting Paper Frogs with Easy DOE. I'm Ryan Lekivetz, Manager of the DOE and Reliability Team at JMP.

I'm Rory Lekivetz from Northwoods Elementary School.

Yeah. And I just happen to be your dad. Yes. You see in the title, you see something crafting paper frogs, if you're like me, a natural question might be, "What is a paper frog?" Let's take a look. Here we have a video. What is this a video of?

It's me making a paper frog. The paper frog is an origami frog, which you fold, and then you can jump it.

Yeah. Do you usually make them this big?

No.

No. I think we had just wanted to show, to get a better sense of what they look like. Now where did you learn how to make paper frogs?

At school.

At school. Full disclosure, I've never made a paper frog myself. She's really the paper frog maker in the house. But you can see after you made this paper frog, what's the hope that you can make it jump as such. But I think we were hoping that we could make it jump better than that. Is that

I'll also say once she learned how to make these paper frogs in school, a lot of times whenever we had, like, a spare piece of paper around, it would somehow turn into a paper frog or plastic frog as it may be. What was our purpose though of this? Again, I've never made a paper frog, but I've seen lots of them at home. What was the purpose of this study? What did you want to do?

I wanted to see what factors could make frogs jump farther.

Mm-hmm. When she had said that, my challenge to her was, "Can you build a frog that jumps far?" And that was a dad pun intended. Yes. Now, for those of you who don't know what Easy DOE is, Rory was actually one of the first beta testers of Easy DOE when we had first introduced it. But this is just a figure that actually comes from junk documentation.

These are the steps or the phases that you would usually go through when you're designing an experiment. We have the describe phase, and that's really where we're talking about what are our responses and factors. The specify, where we have some underlying model that we're assuming. Create the design, collect data, and then ultimately fit a model to that data we've collected and use that to predict whether it be to optimize or to predict process performance. But if you think about what have we traditionally done in JMP?

What I like about Easy DOE is it actually going to combine these steps into one platform, design and analysis. If you think of the way we traditionally do DOE in JMP, you use one of the DOE platforms to go through this describe, specify, and design, But at that point, you just create a data table, and there's this real separation between this design and collect phase. Easy DOE tries to have this all within one platform. So really, when we were building this, it's to help novice users uncomfortable with custom design.

Maybe let's say if you go to DOE, you look at custom design, maybe you start to be overwhelmed with everything that's going on in there. Easy DOE, you see we have it just under that DOE menu thread under custom and augment. But when we launch it, you can see there's this much friendlier, Add a Continuous Factor. Let's add some extra words with hints. A continuous factor can take any numeric value. You see these hints. For example, temperature can be between 30 and 50. An Easy DOE throughout this platform, you're going to see these hints and additional details throughout.

But one thing that while we were designing this, we really wanted to remain true to our DOE principles at JMP. Really viewing this as a bridge to custom design. While we want to help our users and Easy DOE, the idea is that can we help them that when they're ready to use a custom designer when Easy DOE doesn't fulfill their needs, it shouldn't be a huge leap to go to custom design. In particular, for those that are marked comfortable doing a design experiment, flexible mode actually has a more similar feel to custom design.

If we take a look here, maybe I'll add some factors, but if we look at flexible mode, things like the model look a lot more familiar than they might in something like guided mode. We'll see these different tabs as we go throughout, but if you remember when I said about all that workflow being within one platform, so you can see over here, we have these tabs at the top that really reflect these different steps as part of the DOE process. Now one thing I'll mention, you see, when I close this Easy DOE file, you see it actually has saved… Do I want to save the changes to this JMP DOE?

For now, I'll say no, but one of the nice things with Easy DOE, if I wanted to, I could save that JMP DOE file. Instead of just saving, let's say, a DOE dialog or a DOE JS script, or a data table from JMP, now I can share this JMP DOE file. So I can share that with colleagues. Whether it be for in that early design phase or after I've went about collecting the data.

Now, when it came to these paper frogs, what did we actually want to do for our design? Our first step, when we go into Easy DOE we have that defined phase. We have our responses and our factors. What was the response to it? What did you-

I wanted to measure how far the frog would have jumped.

How far the frog jumped? Yeah. I think I will talk about it a little bit later how… This is not immediately obvious how are you going to measure how far the frog jumped? That was a little bit tricky, wasn't it? At first, yeah? But that's ultimately what you were interested in is how far a frog can jump.

But the interesting thing too, if you actually do a search for experiments with paper frogs, it's not as easy as one might think. You do find some science fair experiments, but they're usually only changing one thing, aren't they? Maybe like type of paper or something like that. But what were the factors that you decided on?

Paper type, like, cardstock and decorative. I thought that because for different weights, it might change how far we go.

Okay. How about the size?

I thought if it was bigger it might go not as far.

We saw that in the video. That one was almost too big, wasn't it? It's harder to make it jump. What about this weight? What was that with the penny?

If I added a penny, or I didn't put any penny on it, I wanted to see if it was heavier if it would go as far.

Okay. Yeah. What does this mean, fold?

When you're folding the frogs, if you just do it normally, then that's what the normal is. If you do it really precisely, that's what I did in the precise.

Is that like I think you put extra force on the folds?

Yeah.

I'll say when watching her do it, that might make a difference because there's almost a certain springiness. When it's folded too precise, maybe it doesn't have any jump in it. What was going on with the surface here?

The surface, I was wondering if it was a smooth or a bumpy, if it would affect how far it goes.

Okay. Those were the factors. What did you do? I think you were the one who entered these again, right? If we looked. Here are the factors that you put in here, in an Easy DOE. Now one thing because you had done an Easy DOE before, did it seem any different this time? Did it a little bit?

Mm-hmm.

Okay. I wasn't sure if it would be a parent to Rory or not. Because one thing you notice at the top of Easy DOE, you can add your factors via these buttons here. Add a continuous factor, a discreet numeric factor, add a categorical factor. But if you look here, some of my comments, I thought it was much faster this time. The first time we had done it with paper airplanes, and I think for every factor, we were going through, and it was a discussion, "Okay, what type is it going to be?" I think here that you had recognized pretty quickly what the different types were for most of these.

But I do want to point out, if you have access to JMP 18, Easy DOE now has a newer look to the way that that factor list is. In particular, last time, so let's say if Rory decided afterward well, maybe if size… If first she had it as categorical, and she decided, oh, maybe it's actually continuous or discreet numeric because size actually has a number associated with it, she would have to go and delete all those other factors and then add it back in if she had a certain order that she wanted.

Now in JMP 18, you see here, I can switch. Let's say, even if I wanted weight. Maybe if I had instead of a penny maybe we had some little weights that we could assign, like, 1 gram, 2 grams, things like that. You see now we can quickly go with that to continuous or categorical at any point. Let's say if I wanted to change my order, maybe I'm going to move force up the list. Do it when I'm trying to do this on the laptop. There we go. You can see we can also drag and drop within this new factor list.

Before, if you wanted to try and change the order things we're showing up, and it was a lot of work within DOE. Now we can do this quite easily. You can see here we have these convenient buttons as well. We can add, remove rows, undo, which is particularly nice. I think once you get a better sense as to what's going on with the rules, I don't even think you… Did you use these buttons or not as much?

Yeah. I think that once you get the hang of it, you can just start using this really convenient control here. It's the same. You'll see the same thing in the response table. Now, do you remember? We had to talk about this a little bit. Why was your goal maximize?

Because we're just seeing how far it goes. It's not like we're trying to get to a target or like.

Or bigger is better in this sense. I think you wanted to make it jump. I will admit we had talked about did we want to have a target to see if we can make the frog jump on it. Maybe that's something interesting for the future.

But I think for here for a 9-year-old, it's much cooler to make a frog jump as far as it can. Is that right for your friends at school? Yeah. That's, you know, again, and just this new factor list. This looks fairly similar to what would have been in JMP 17, but if you have access to JMP 18, I think that this new factor list really is more forgiving. Especially in something like Easy DOE, or if you want to change your mind, it's much easier to do so.

Okay. That was our Define. Now we had all of this. Now we can either use this navigation control at the bottom or we went to the model. I'd say this was still a little bit tricky. Here at the model. This is where I think if you're using Easy DOE, you still have to have a sense as to what interactions are. Do you remember? Now do you remember what is an interaction?

An interaction is where, two of the factors interact with each other.

What would be an example? What did you think might happen if you looked at your factors here?

I thought there would be something with fold and paper type.

Okay. Is that because is one of them easier to fold?

Yeah.

Which one was easier to fold?

Cardstock.

Okay. Maybe then there was that idea that there could be something going on with interactions. But if we look here… When we looked at that model, it was like, well, the main effects are just finding what's important, but that two-factor interactions. What did that mean that number runs?

I had to make 30.

Right. She had to make 30 frogs. Was 30 frogs too many or not?

I could do it. It was a lot of flow.

Yeah. Well, I think that when you'd practice, a normal one, you could sometimes do in about 30 seconds.

Yeah.

I think you just made them while watching TV, maybe, is that right? But so we discussed, and so 30 seemed reasonable. It wasn't terrible. And it did seem like there was that interest in the interactions. That is ultimately what we did go with. But so let's say… Now we had our 30 frogs. All that we did there we just hit that next button. This was telling you which frogs you had to make. Then I'll admit what we actually did. We printed that out, and then it was a matter of going and collecting that data. Do you want to talk about… How did you go about collecting the data on this one?

We randomly pulled a frog out of the box. Then we took a piece of tape and at the end of the nose where it was, we jumped it, and then we took the distance between where the nose was and where the nose was after it jumped. Then we did that three times, and then fixed the maximum absolute value.

Yeah. I'll say, this was interesting because I didn't want to have too much interference on what you might want to do for that distance. Because one might think you might go to the closest point to where that nose had started. There was actually also a question. You noticed that maximum absolute, and you see I said the question mark. Do you remember what is an absolute value?

It's how far it is from zero.

Because did you have some that were negative?

Yes.

What does that mean if it's negative?

It went backwards.

Yeah. I'll admit the first few that we pulled out. Daddy was worried that her frog experiment wasn't going to go very well. Because some of them weren't jumping really at all, were they? They've jumped up in the air and stay maybe 1 centimeter or 2 centimeters, and then some that were going negative. But I think ultimately… You can see you did have some that actually went pretty far, didn't they? What do you think?

Yeah.

Yeah. Now that we have that data entry, what do we do after we have that data collected? What's the next tab that we have? Oops. I guess you can't see it there. Do you remember what we did after we collected?

We rolled in the distances and went to analyze.

Yeah. You can see here, I guess I've actually already filled out that analysis. Let's see if it's filled out. It might be filled out in the other one too. Let's see. What did you do for the analyze? I think when you come through, it has all of these in here. Did that look like something that you wanted here? No. Do you remember what did you do?

I went to best model.

The best model? Okay. What did you see? When we hit that best model, what did it look like was important to you?

It looked like cardstock in light was important.

Okay, yeah. That cardstock in force. Yeah. Or the paper in force. Yeah. Maybe the weight possibly.

Yeah.

Now this was interesting, because it didn't look like that was that important, this force light. Now this is also something to mention for JMP 18. I don't know if you remember this last time, but in the JMP 17, If you removed an effect… All of the effects not in the model were just centered right at zero. Let's see what happens if I take all that interaction? Now if you notice, what happens in JMP 18 is that it actually tells me what would happen if I add this effect back in next. I don't know that this was obvious to you, but in the last version of JMP, if I had removed this, suddenly, all of my confidence intervals would have been larger. Then it would have been hard if I didn't pay attention to what I had removed.

One of the nice things I like about JMP 18, I can go back and put that in right away. I'll also mention you'll notice if I remove force, after hearing a lot of feedback from our more experienced users, so Easy DOA now does obey affect heredity as well. That's a discussion for another day as to what heredity means. Now I should actually… I didn't neglect to mention. After we had collected that data, I thought it was interesting. So before we did anything else, before we even entered the data, you remember I had asked you what your initial impression was. Do you remember? This is what I wrote down. The what? Paper.

Paper seems to be important. Then I think there was something going on with decorative and normal force. Seem to be really well together.

Okay. Which we said, and when I asked you, so that maybe is something like an interaction. Then you said maybe penny is better than no penny. Or was the other way around? I don't know if I wrote that down or not. But that's what I had on my piece of paper. But I'll say, from this, if we go to that predict, did it look like that the… Here it looked like that no penny was better. Did it seem like you were right though? Was there something going on? Let's see. If we move that force from light to normal, does it look like there's that idea of an interaction?

Yeah.

Yeah, it does, right? It seems like depending on what's happening with decorative or cardstock, then something's happening with that force. You actually did pick up that there might have been… Now do you remember? What if we wanted to make a really good frog? What did you do?

I clicked optimize.

Yeah, optimize, yeah. Admittedly, we actually had to have a discussion. I think in the future versions, we may even put some text there to say, "These will be the best settings that you have for your factors." But if we click that, that's what JMP was suggesting to us. But so I'll say, that was what Easy DOE was telling Rory. I should actually mention here too. We had talked about the idea of outliers. I guess we didn't highlight that, did we? In this model, when we looked at those residuals, you actually already had some idea of what outliers were, right? What did we notice here?

It was an outlier in row 7 and row 9.

We actually did go back to, so I say this row 9, looked okay. I think when we went back, but what did we notice about row 7?

There was actually a positive maximum.

Yeah. There was a positive value.

Yeah.

This was like that absolute. This one just happened to go back really far. I think that when we went in and changed then things had changed a little bit too. I think we still had one of them there. There was another negative. I think that if I went back, right. I think we actually changed some of those absolute ones to the more positive. That was truly like the maximum. I'll say in that one, when we hit the best model, there were actually a few more effects that looked important to you, weren't there?

Yes.

It's still perhaps another one. But I think regardless, so at the end of the day, we were still curious. The interesting thing was that even after we had changed those outliers, it actually still predicted the same, didn't it? Like, your best frog was still what it had. In particular, did you actually have that frog in your? I think not for hardwood. Was that right? I guess maybe a bit of a spoiler alert, but when you tried that frog, did it work?

Yeah.

If we were in-person, we'd like to show you that that's… Actually what? Like maybe 40 or 50 centimeters, which I didn't expect you can make a paper frog jump that far. Did you until the experiment? No.

Now I will also mention. Dad thought, "Okay, I'm a statistician. Surely, I can beat Rory's model, or I'm going to use my statistical tools to do something." For instance, I tried to stack it. I thought, "Okay, maybe is there some kind of time effect? I can use a random effects model to say these frogs are going to be random effects." And look at that. We thought maybe there was a time trend.

Well, say one of the things I did find interesting, you can say, "Well, maybe I'm also going to go in. I'll take the average of those three instead." But I'll say, ultimately, Daddy's results ended up pretty much the same thing that you've had. I couldn't do any better. But one thing I did, so how did I get this? Well, from that data, this is where I just exported that data. I was using the same thing that Rory was, and then I just added my own columns for those distances. I just took that original sheet that we had. I mean, that was how I got this Rory one, distance 1, 2, and 3.

But one thing I liked about exporting that data, if I have it here. I just was able to do something like a Graph Builder. If you remember, Rory had said, well, maybe she thought paper was going to be important. This is very simple, if we just load up Graph Builder, with paper versus the main distance, and then I could have done it with distance 1, 2, 3, I could have done the max. But you can see, I just added a column switcher here to Graph Builder, so now we can see. It does look like there is something going on with that cardstock and decorative.

Size, maybe, maybe not. Wait, yeah. It looks like the no penny. The fold, I think it was less obvious. Maybe a little bit better. Force, maybe more variability going on there. There's something going on with that force. Surface, you can see a little bit. But I think you can see that distinction here. There is definitely something going on with the paper.

The other thing I'd like about loading this up in something like Graph Builder actually, maybe let's just start it from here. Let's just say if we took that distance. Okay. If you remember that Rory had said, well, she thought there was something going on with cardstock and decorative. Why don't we just bring that up into here? There's different ways we can look at that. But it looks like here's, so maybe she is right that we can see it for the light. That light force, maybe not a huge difference between the cardstock and decorative. But it looks like once you have the normal one, then it does seem like there is a difference.

I thought this was a nice way to, let's just bring up Graph Builder directly from that exported data. Now I'll say here, maybe maximum actually wasn't so bad. I think about it from what a 9-year-old wants. Here, I was looking at the mean. In some of my trials I had standard deviation. I wanted to make this as robust as I could. But at the end of the day, she wants to be able to go to school and show her friends that she has this frog that can jump far. Even if it fails sometimes, that's probably okay. I don't think you only get one chance to show them how far it can jump. Is that right? Maybe actually her picking the maximum really wasn't so bad.

But let's just say, what if I actually did want to bring that mean distance in? Maybe I've done a whole bunch of work. Maybe I've removed my own outliers. Maybe I've transformed that response, or I've done some work on the outside. Now I could try to bring that back in to Rory's Easy DOE, but what can I do if I just had that data table I was working with? If I launch a new Easy DOE, I'll say this is a little bit harder to find, but under that factor table, if I click the red triangle, there's the option here for low design.

Here, I'm just going to pick the factors that I have, those were the factors that Rory had, and so you can see that Easy DOE has loaded in that same design that you used. But let's say instead of using Rory's response, or maybe I want to do my own, I'm just going to go to Load Response, and take that mean response in there. In this way, maybe I can do some other work on the outside and then use Easy DOE to load in. I'll call it the new design, but not really. It's just that maybe if the person using Easy DOE has sent that file off or sent that data table off to somebody else, I can still easily load it into Easy DOE to do my own work with it.

Again, ultimately, the predict. I think we had the same predict in the end. Were you pretty happy with your predicted frog? I'll say, I think even when Daddy tried it, I think I was able to get mine to go farther than yours. Using your predicted one. Maybe not.

Yeah. With that, that's our presentation on that Easy DOE. Just some final thoughts. Safe from my own, maybe sometimes let's keep it simple. Trust the process. Just doing a design experiment, you're learning a lot. Again, maybe if dad spent some extra time, I could find a better model or more going on. But that keeping is simple really worked. Really that idea of trusting the process that even sometimes just doing a design experiment and looking at the data really does tell you a lot.

That double randomized, that was one thing in hindsight. Perhaps I should have kept track of the order that we hopped them. Because we didn't use the original order. I think that was your numbering system. But sometimes we couldn't really tell if you were just getting better or correcting yourself over time. We couldn't really measure a time frame, which I don't think you were worried too much about. But as a statistician, Daddy was worried that you were learning how to hop paper frogs better as you went. Because you were the hopper the entire time, weren't you?

What would you do differently next time? Is Dad can go first here. I found forced too subjective. I think it was hard to tell the light versus normal. At least when I was watching Rory, sometimes that light, it was a little flick that was hard to do. Again, thinking more about that time effect, maybe is there some learning going on? Even those three repetitions within a frog, I think it might be nicer to do this as a split plot. I think it's easier to do this in batches to do a few frogs at one time, especially like that location effect.

Then maybe that response because here we just measured from nose to nose, perhaps next time, it might be nice to try to get it straight down. Like if we had a ruler that was straight down in a straight line, can we get it to go straight, or maybe that target? How about you? What would you do differently next time?

Add another paper type.

Add another paper type. Anything else you do differently?

Maybe to truncate your target.

Target, that sounds fun. I think that would be pretty cool, wouldn't it? You think you're still up for another the DOE at some point? Yeah. Maybe a cooking one, though. So stay tuned. Maybe we've talked about making chocolates at some point.

But with that, thank you for your time. Thank you for joining us. If you have any comments or questions, leave them below. Thank you.