Hello, my name is Michael Hecht, and I am here to talk today about the workflow builder, which is a new feature in JMP 17 that allows you to record operations that you do within JMP and then play them back to recreate those operations.
If you attended the plenary session this morning, then you saw Mandy Chamber's demo of workflow builder, which gave a good overview. Mandy also gave a talk this morning that goes into more detail on the workflow builder user interface.
I'm going to start talking about a more advanced feature of the workflow builder called the reference manager. This is a feature that allows you to take a workflow and manage how references to data tables and columns within those tables are mapped and resolved. It allows you to make workflows that are more generic to be used with any data table.
Let's get started. I'm going to start by creating a new workflow which is under File, New and there's New Workflow. You see the little tag that says this is a new item for version 17. When I choose this, I get this untitled workflow builder window. The panel in the center lists the steps of my workflow and as you can see, they're empty.
I'm just going to click the record button, which is this button with the big red dot. You see it changes appearance to show that recording is in progress. Then I will do some operations in JMP and have them be recorded.
First, I'll go I'm going to go to the File menu and I'm going to open up a data table. We see Big Class here opened and if we look in the workflow builder, we see that a step was added to do that same operation.
Now I'm going to go to the Analyze menu and I'll do a Fit Y by X to do a one- way analysis using age as my X factor. For the response, I'm going to create a transform column by right clicking and choosing formula. I'm going to use weight divided by the height squared, which is the formula for body mass index.
If we were using metric units of kilograms over centimeter squared, then that would be sufficient. But Big Class has its measurements in imperial units of inches and pounds. We have to multiply this by a scaling factor to get a standardized BMI.
Now I have my formula correctly, I'm going to rename my transform column 'cause I don't want to use the default name, I'm going to call it BMI. Click okay. Click okay. Now BMI is in my list of columns that I can use to create the report. I'm going to add that as my response. Click okay.
Here's my one- way analysis of variants. You notice, though, that it did not yet add this step to the workflow builder, but there is a little note at the top saying, hey, I see you launched a platform. A s soon as you finish with the analysis and close the window, I'll add it to the workflow.
That's so the workflow builder can capture any changes you might make to the analysis. For example, I'm going to turn on means ANOVA here to get the means diamonds. Now when I close the window, you see the step gets added, report snapshot, Big Class, Fit Y by X of BMI by age.
Great. I am done recording. I'm going to click the button again to stop recording. Then I'm going to turn, switch to presentation mode for this workflow. Like the tool tip says, that removes some of the editing controls. Namely it removes the record button so that I don't accidentally hit it again, and it takes away this activity log at the bottom.
I'm going to rewind my workflow to the beginning, which closes windows that were associated with it. Then I'm going to click the Run button to replay it just to make sure it does what I wanted to do. I click Run and it opens Big Class. Then here's the analysis just like that we had it before.
That's great. I want to look at these different steps in the workflow. You can see that behind each one is some JSL that shows up in the tool tips. But I can open up this step settings panel to see more details. I'll click on the first step and we see that there's some metadata information and then the JSL to run it, which is just an open command and there's a path name to the file.
When I click on the second one, I see that we have JSL that sends a one- way message to this reference to the bigc lass. Jmp data table. Inside the one- way, we see references to weight and height and age. There's a reference to BMI, but BMI is computed as a transform column right there.
That all looks good. But when I run this workflow, I don't always want to run it on Big Class. I might want to run it on a different data table. I'd like the user of the workflow to be able to choose that data table. I really don't need this open data table step at all. I'm going to select it and click the trash can icon here to remove it from the workflow.
Well, now what's going to happen when I run my workflow? Let's give it a try. I'll click the Run button and it immediately prompts me to choose a data table. The data table Big Class, it can't find it anywhere. It's not in the list of tables that are currently open in JMP. It's asking me, do I want to go find it and open it or maybe open a different table?
It says that anywhere that Big Class is referenced, it will use the table that I opened. It has a list here, but there's only one item in it . I'm going to select that and click okay. Now it's prompting me to go ahead and open a table. Well, I have Big Class right here, so I'm going to drag that in and click Open, and it runs just like before.
Well, that's great. Let me rewind and run it again, but this time I want to choose a data table that's not Big Class. Let's see what happens. I'll click okay, and I want to use this table Football. Football is data about a college football team playing American football as opposed to rest of the world football.
I'm going to open that and there it is. But you see, I'm immediately prompted, hey, this data table does not have a column named age. What would you like to use instead? I'm going to choose position and I'll click okay, and I get my one- way analysis.
Notice that it did not prompt me for height and weight because those columns already exist in the data table, so it just uses them directly. T hese positions are now the categories that it's using for the one- way analysis, and they're all the abbreviations for different positions in American football.
You have your defensive back, defensive lineman, full back, half back. I don't know what IB is, but kicker, offensive back, offensive lineman, quarterback, tight end. You see the wide receivers have a nice little group here with low BMI because they have to be fast.
That's all cool. But I noticed in this football data table right below position, there's a second variable called position 2. I f we look at what that is, it's a different categorization of the data. Position divides the data into 11 categories, but position 2 divides it into 7.
It might be interesting to run my workflow using position 2 for comparison. I'm going to rewind this and I'm going to run it again, but this time I'll choose position 2. Well, wait a minute, it didn't even give me a chance to choose the variable. It just went ahead and used position.
In fact, it didn't ask me what data table to use. It decided Football was already open, so it could just use that. Somehow the workflow is remembering my choices from the previous run. If I want to choose a different variable, I have to somehow prevent that from happening.
Let's take a look at this workflow. We see that there's an option on the red triangle menu that says References and has a sub menu, allow replacement of references. Well, that's already checked and the tool tip says it allows prompting for tables and column references that it can't find.
Well, that's exactly what happened the first time we ran it. But then when we ran it again, it reused the replacements that already had. But here's the second option, Clear Replacements. The tool tip here says that it clears the previous replacement choices, which is what we want. Let's do that, and then we'll rewind this and run it again.
Okay. Now it's prompting me for a data table again. Because Football is one of the tables already opened, it appears in the list here. I can just pick it, click okay, and now it's prompting me for age again. This is great. I can pick position 2 and click okay. You can see the seven categories that are defined by position two here.
Okay, well, let's go back to position 1 now. Rewind this and run it again. Again, it didn't give me a chance to choose position 2. I don't really want to have to clear the replace ments every time that I run this workflow. What I need is some way to control which replace ments are remembered. I see this third option here that says Manage. P ool tip says it manages the table and column references that can be replaced at runtime. Let's see what we can do here.
This brings up a window called the Reference Manager. At the top is this check box, allow replacement of references, which is pretty much the same as that first sub menu choice. Just like that, when it's checked. Then there's a button reset all replacement choices, which sounds the same as the clear replacement menu item.
Then we have a list of table references. There's only one item in the list. That's because my workflow only has the one reference to Big Class. If this were a more complicated workflow that accessed multiple tables, then all the table references it used would appear in this list. You select one of them and then you have details that you can change.
I can see that I can add a custom prompt rather than the big long prompt. I'm going to use something a little simpler. How about please choose a data table. I see that this mode is set to prompt if necessary. It's necessary when it can't find Big Class or it can't find what you see Big Class is mapped to here Football. But what I want is for it to prompt every time I run the workflow, so I want to choose each run.
Then down here, we have a list of the column references that the workflow uses from this data table. We see BMI, which is the transform column. That should never be prompted before because we're computing it within our workflow, so I can change this one to never.
You see age is remembering its mapping to position and it prompts if necessary. We want to change that to prompt on each run. I'm going to also give a custom prompt for it about to please select a category column. I'm going to copy that so I can reuse it.
Height and weight are both referenced here and as we saw, it can pick those up automatically if they exist in the data table. That's good. I think if necessary, it's doing what we want. Let's leave it at that, but I'll give it a better prompt. We'll change this one. The same thing for weight, we'll change here.
I think we have all of our settings the way we like it, but we still want to clear these mappings to the Football data table. I'm going to click button and hopefully, those will go away. All right, t hat's good. I'll click okay.
Let's rewind this and try it again. It's letting me choose, and I can go back to position. You saw the new prompts in both of those dialogs, the custom prompts that I chose.
That all works well. In fact, I can even... Let's see, let's go open our Big C lass table. If I run the workflow now, you'll notice that even though the workflow was originally recorded using Big Class, because I set the prompt to each run, it's prompting me now instead of just using Big Class directly. But I'm going to choose Big Class. Now, even though Big Class has an age column, it's asking me to select a category so I can choose something different like sex. Then you get one way of BMI by sex.
This is all doing what we want. I am going to save this workflow from the File menu, Save, I'll give it a name of BMI and it automatically gets an extension of . jmpflow, J-M-P F-L-O-W. We'll save it to the desktop and there it is. You see it has this little org chart looking icon. If I get info on it from the finder, we can see that the hidden extension .jmpflow is right there.
This is great. I could distribute this to my colleagues and they would open it on their installation of JMP. Then when they run it, it will prompt them for a data table, it'll prompt them for a category column, and it'll produce the report.
That's fine. That's a great way to do it. I think I'm going to take it one step further, though. I would like to package this within an add- in. An add- in lets you customize items on the JMP toolbar and menu bar. For example, you can see I have an add- ins menu here already with some items in it, but I'm going to create a new add- in to put my own command there for BMI.
Let's go to the File menu and we'll create a new add- in and this add -in builder dialog appears. Let's give it a name. We'll call it BMI Report. Add-ins need a unique identifier, which is just a string, but we use this reverse DN S system that you take your company's website, like ours is Jmp.com and reverse it, s o we'll use com.j mp.
I am the only hashed at JMP, so I'll put that in here, and I'm going to give it something unique for this specific ad- in that I'm creating. We'll call it BMI-R eport. I'm going to select all that and copy it.
This is version 1 of my add- in. As I said, the workflows are a new feature of JMP 17, so I'd like to set the minimum JMP version to 17. Unfortunately, it looks like we forgot to add that as a possibility to this menu. I'll just do the best I can and choose 16, and hopefully, we will get that corrected for JMP 17.1 coming next month.
I want to add a menu item to the add- ins menu. I'll click add command and we'll name it BMI Report. I'll even give it a tool tip, create a one- way analysis of body mass index by the chosen category. That's pretty good.
Now I do need to add some JSL here, but it's pretty simple. All I want to do is open bmi.j mp flow. However, I'd like to embed my BMI workflow within the add- in that I'm creating, which means I need to tell the open command that it comes from this add- in's home directory, which I can get to with the path variable dollar add- in_ home. I have to give it the add- in's unique ID as well in parentheses and put a slash there for the directory separator. That looks good.
Now I need to actually add my workflow as a file embedded in my add- in. I'll go to additional files and add it there. That I believe is everything. Let's close this, save changes. It gives me a default name of BMI Report. We'll save it on my desktop. There it is. It should have this workflow embedded in it.
I'm going to close it here and I'm going to put this workflow that I built in the trash, empty the trash. When we save an add- in, JMP automatically installs it as well. If I come over to JMP and look at my add- ins menu, now we have BMI Report. When I choose that, it should open the workflow that's embedded within the add- in.
There it is. We can run it. We get our custom prompts before, we can choose a table, we can choose a category. I am going to redo the analysis here so that I have a copy of this that's not under the control of the workflow and will stay around even when I rewind the workflow. Let's rewind it. I'll run it again and let's choose position two.
You can compare the two reports and let's see, we can look at things like, this O category under position two that corresponds to the full back, the half back and the quarterbacks, so it's the offense. L is the defensive line and the offensive line, so that's the linesman.
Anyway, that pretty much concludes the items I want to cover in this talk. I direct you again to Mandy's presentation this morning, Navigating Your Data Workflow: Workflow Builder Grants Your Wishes for Data Cleanup, for a great overview of of the rest of the workflow builder UI. I believe at this time we are going to take live Q&A. Thank you very much.