Easy DOE: Easy Enough for a Seven-Year-Old? (2022-US-30MP-1163)

Hello. Thank you for joining us today.

I'm Ryan Lekivetz, Manager of the DOE and Reliability team at JMP.

And I'm Rory Lekivetz, rising second- grade student.

Today , we're going to talk to you about Easy DOE.

I posed the question easy enough for a seven- year- old.

Now, for a lot of you watching, you may not have heard of Easy DOE before.

It's one of our new platforms in JMP 17,

and so it's base JMP so you don't need a pro to use it.

The idea with Easy DOE is that we're going to have one file,

one workflow that's going to contain both the design and analysis.

If you're familiar with doing design experiments and JMP,

you're used to going under the DOE menu,

creating a designed experiment, and then making a data table from there.

And then you would do all your data table on that analysis.

And there was that separation between the design and analysis point.

The idea with Easy DOE is that

we're trying to aid novice users through the entire workflow.

And so, unlike custom design, you're going to see a lot more hints

and different defaults that are set to try to aid those users.

And you're going to see that both on the design and the analysis side.

In addition, you'll also see there isa flexible mode

for those who are more comfortable with DOE.

That's all safe.

My purpose in this talk was really to see,

is Easy DOE going to be easy enough to use?

Well, I said seven- year- old,

but I guess you're more like seven and a half now, is that right?

The idea here was to let Rory do the steering throughout.

That I wanted her to be the one using Easy DOE, putting everything in.

I wanted to have as little input as possible,

even when it came to decisions about what to do with the design,

and things like that.

Of course, I still did need to give her an introduction to DOE.

I mean, if you've seen in our DOE documentation,

this figure might look familiar.

So we went through the different phases.

We talked about what's the difference between a goal,

a response and the factors.

Specify.

That's where we talked about what's going on with the model?

And in particular, what's the difference between a main effect and an interaction?

Why would you care about one or another?

And we said, well, once we have the design,

we actually have to go about collecting some data.

Then we have to fit, then we have to do something with the model.

We want to find out what's important

and how can we use that model to do something further.

I mean, you can imagine we spent maybe about half an hour

just talking about some of those different things.

I'll say, well, why don't we use some classic experiments ?

My suggestions were, why don't we try the old paper helicopter experiment?

O n our shelf we have the statap ult or the classic catapult experiment.

And so what did Rory say?

No.

Rory's idea was actually to do paper airplanes.

S he'd started to to try out flying some paper airplanes.

S he said, well, I want to do a DOE with paper airplanes.

I said, well, that's great, let's see what we can do.

Luckily, she knew the classic paper airplane

which is what you'll see is called a Dart.

But we found a website that had different instructions

for that different types of paper airplanes you could make.

And on top of that, it had suggestions

for what you could do to try to make your plane fly better.

So thankfully then, instead of her having to try to figure out

what are some of her factors and levels going to be,

this website had some really nice suggestions for that.

Before we get into this, Easy DOE, as I said, it is that new platform,

and you're just going to find it under the DOE menu.

Underneath custom and augment design, there's Easy DOE right there.

Now, what you'll see with Easy DOE is that idea

of going through that workflow is going to be done via tabs.

A s soon as you launch Easy DOE,

you'll see there's that guided and flexible mode.

We're just talking about guided mode here.

But the idea is we're going to go

through these different steps by clicking on the tabs.

There's the Define,

and then we're going to go to the Model, Design, et cetera.

One way to do that is to click on the tabs, the tabs one at a time.

And at the bottom of Easy DOE,

you'll also find a set of navigation controls

so that will take you forward and backwards between the different tabs.

And so the idea of our talk here,

we're going to go through these different tabs

and both of us are going to give observations.

Rory is going to give her thoughts first on those different tabs

and then followed by my own.

Think of it more as a teacher in my point of view,

and Rory's was more as the novice user, trying Easy DOE for the first time.

Let's start with the Define tab.

On this one, we had to type in the different factors and levels

that we were going to do for a paper airplane.

None of my factors had numbers, so I chose categorical.

The different levels and factors that I had type,

and then the levels were dart and lock bottom.

And then there was paper, and the levels were regular and construction.

In throwing force, the levels for that one was hard and light.

And paperclip was paper clip and no paper clip.

The response that we had was distance.

And for the distance, I wanted to see

what could make the paper airplane go the farthest

and our JMP goal was maximize.

Just to mention.

When you go into Easy DOE, you're going to find right now.

If you take a look at that screenshot from before.

So currently, we have three different types of factors:

continuous, discrete, numeric, and categorical factor types.

I'll say I think she was able to identify

that she needed the categorical for all of those.

Of course, now, she did need confirmation.

I mean, when she said categorical,

she looked back to me to see, "Am I doing that correctly?"

S he was able to identify the factor types and actually enter her level names,

sorry, her factor names and her levels.

Now, I will admit though, she's used to using a touch screen.

And so there did come a point

instead of her trying to click into the little box for levels

or the factor name and to do a double click,

there did come a point where I told her

you can just use the tab button to make your life easier.

A gain, I didn't want to have a whole lot of input.

S he picked the four factors and levels

and I think I had told her to pick three or four was a good number.

But I will say so if you think back to that paper clip.

She had mentioned it was the paper clip or no paper clip,

that one took a little bit of time.

I think maybe if we would have went paper clip yes or no,

it might have been easier.

But I think for her, it was.

Well, how do we actually define a paper clip factor

that also has a paper clip level?

But I think once she had that idea, okay, it was just kind of a yes or no,

it's in there or not, we were able to get through it okay.

Moving on from the Define tab.

The Model.

For the Model tab,

we picked the one which we thought would be the best.

We decided that Main Effects and two- factor interactions

were the ones that we want the number.

So we picked the one that had Main Effects and two- factor interactions.

The number of runs meant we had to make 16 airplanes

which didn't seem too bad.

Now, I'll say in hindsight for my own view on the Model tab,

I mean this required the most hand-holding of all the tabs.

And this was really more because of trying to explain the difference

if we just went with main effects versus interactions.

But again, so this is a seven and a half- year- old

who's never taken any statistics course,

has never done any kind of modeling before.

If you have somebody who's familiar with the idea of main effects and interactions,

then I think that tab wouldn't be nearly as bad.

Now, I will say too, though , it was nice.

The paper airplane website that we were using,

it spelled out the idea

of what interactions really mean with those factors.

I think so you would see things where it said:

while some of the airplanes will work better with a paper clip,

or you'll see some airplanes are better when they're throwing hard,

while others need that lighter touch.

In some sense, that actually gave us

a natural point to start talking about interactions.

Now, also, say, when I think of from my own perspective,

it's also something that we can improve upon.

From a US, I want to think about,

well, how can we distinguish between these choices?

We did have the hints in there as well.

But when I think of, if I wasn't there to do the hand-holding,

how might she'll come up with that decision?

Now when we went to that Model tab,

and our next tab that we clicked on was...

Design.

There's not really a lot I can say about this one,

but I thought it was interesting that it was showing

what kind of paper airplanes that you were going to make.

And then we had the hard work of making planes and flying them.

Yeah, I think that probably took the most time, but yeah.

I don't have too much to add on this one.

I think, though, it was nice to have that design displayed

just to give her that sense of,

well, what did that really mean when we put in those 16 runs?

When we have those factors, what does that mean at the end of the day?

I think from this, then she could really get that sense.

Okay, then my plane one, it's the lock bottom with regular paper.

I'm going to throw it lightly and put a paper clip on it.

What was the next tab that we went to?

Data Entry.

So full of a data entry.

We didn't really have to do that much in this one.

So we just put the distance of the different kind of airplanes flew.

So that was what we measured with our measuring tape of stuff.

Is that right?

Now, I'll say with this one, if you look, there were these factor plots.

This is just for the main effects here,

but this actually was telling her what she had said as soon as she was done.

She see those factor plots at the bottom,

and she said the lock bottom is not the best.

I think she might have used something stronger than that,

but yeah, so the lock bottom was not good for her.

Also, this was a good teaching moment about randomization.

So the run order does come out randomized.

I did have to warn her when we had all the paper airplanes with us outside,

I said, well, you don't want to

throw all the dart type first, followed by the long bottom.

I said, because you might get better as you were throwing

or it might start to storm, it might get windier.

Now, I say some of these results.

There were times where the hard- throwing force needed a bit of practice.

I'll admit there were a few there that probably were based on more than one,

because if you had a crash, almost immediate crash landings.

But I'll say it did seem straightforward for her

to be able to enter the data right from there.

I mean, I didn't have to say anything; she knew.

So when you come in to hear it,

the response column just had missing values.

And so she had that intuition,

well, this is where I need to go to put in the data.

I also mentioned here, you'll see this export data and load response.

That export data is if you actually

just want to create a data table with all your stuff.

S ometimes that will be useful if you want to go through

what you're typically thinking of with your JMP workflow.

If you just want to JMP data table,

that's what that export data button is going to do.

Likewise, load response, if you've actually just recorded

your responses in a different data table, you can do that.

Now that we had our data, what did we have to do?

-What was our next step? -Analyze.

For the analyze, I already knew that the dotted ones

weren't the most important ones.

I figured out I just needed to click them to get rid of them.

I found out dart type was one of the most important ones.

Now, I'll say here.

The analyze I had thought was going to be the hardest one to explain.

Now, I'll say it was surprisingly easy and effective.

It was almost she had clicked on the tab and just started doing her own thing,

and I didn't really need to say an awful lot.

Now, you'll notice here,

so when you come into that analyze tab, when Rory saw it, it was the full model,

but she saw a lot of the terms that were not significant.

There were a lot that were dashed and close to that zero.

And so what she did was just remove those ones one at a time.

And then I also see there is a best model button

that's based on some type of a forward selection.

Now, I'll say the best model may actually have more terms,

but at the end, you can see

those extra terms really weren't even significant.

S ometimes I couldn't even argue with the results.

Perhaps, one might even argue that the model that Rory picked was better

because it was simpler and there wasn't a huge difference between the two.

Now, one of the nice things with this Easy DOE platform,

this analyzed for the guided users,

there's this idea of adding and removing terms easily.

And so to add and remove these terms,

all that you do is you go to those confidence intervals in there

and a click will either add it or remove it

depending on if it's currently in the model.

I highly recommend trying that out when you get your hands on 17.

Perhaps, there was something to be said.

The best model, when we'll see in a minute with the profiler,

it makes it a little bit more interesting, of course,

if you have some additional terms in there.

Rory's model, with only the three terms,

the profiler was less interesting to try to explain things.

But again, for being a first time DOE,

a model like that didn't disappoint me at all.

Now that we had a model, what do we actually do with that?

-So what was our next step? -Predict.

The Predict tab, this tab tells you what the best paper airplane was.

If you click on the levels that you think are the worst,

it shows what might happen with that airplane.

Likewise with that Model tab,

if you have a user who's used the profiler before in something like Fit model,

of course, that'll be a lot easier.

But I'll also say the profiler is intuitive in general.

I think it was easy for her to pick up on

once she started playing around with that profiler.

And then just a little bit of an idea,

a discussion as to what that actually meant.

But I think the profiler,

that's the nice thing with the Easy DOE, the profiler is already intuitive.

It was also a good teaching moment when it came to interactions.

If you go back, if you look at that model,

we actually had an interaction between the type and the type of paper.

Paper and type had an interaction and so then we could talk about

well, what happens when we change type, then look at what happens with that paper?

Again, this is where those extra model terms.

In Rory's model, that hard and light had a zero effect.

I was saying it didn't really matter what you did for hard or light.

The best model, it had a small effect,

so you could say, well, it's not going to make a big difference.

But for some of these, it was saying that it was.

But again, I think the Predict tab, she seemed to do well in that.

And the optimized did need some explanation,

I think just because that's a new word in her vocabulary.

But I think that she had that sense as to,

okay, these were the settings that were going to be the best.

I think that was what we had to talk to you about today.

I did just have a few final questions for you, if that's okay.

What was your favorite part of the experiment?

Flying the airplane.

What was your least favorite part?

Getting hard flying the airplane.

The air was about 100 degrees Fahrenheit when we were flying the airplanes,

but we didn't have a lot of choice.

We had a lot of storms coming up to it, didn't we?

If you were to tell somebody, what was the most important factor?

What's the most important thing if you wanted to make the paper airplane?

Maybe play type, like, with a dart.

Okay.

Was there anything else that was important?

Nothing you can really think of. Okay.

I think we found the construction paper a little bit,

and I think it did actually say the no paper clip.

I think you were saying that might be because of the weight.

Bul I'll say yeah.

If we tried it again,

I think you had the paper clip at the back part of the wing.

And so we thought about maybe at the nose if we were to try that again.

But let's say, what do you want to do for your next experiment?

Should we do another one? What would you want to do?

Statapult.

A statapult? Yeah, that does look fun.

Ane now, I didn't ask you to answer this in any way, did I?

But was Easy DOE easy to use?

Yes.

It was. Okay.

I'm glad to hear.

I think that's everything that we have left today.

So thank you for your time.

And please post any questions in the community forums below

if you'd like to ask either of us anything.

Thank you.