SHAP Analyses and SHAP Graph Builder

[#1]

Hi All,

I have searched in the JMP community about SHAP Values and I am wondering if someone can direct me to some resources in how to evaluate SHAP values using JMP Pro 17. I understand how to populate the SHAP values (from the prediction profiler), but creating the graphs are a bit confusing at the point. I used the graph builder to build the graph, but I am wondering if there are videos or guides on the types of analyses that I can do with SHAP values and if there's an automatic way to create graphs from the SHAP platform.

Thank you for taking the time to get back to me.

Sincerely,

Nisha

[#2]

Shapley values are available from the Profiler.

[#3]

Thank you @Mark_Bailey ! I did take a look at the guide before, but it didn't provide any detail on the process needed to create different types of graphs and how to evaluate/interpret those graphs, which is what I am looking for. Thank you for sharing the link.

[#4 · Accepted solution]

When you add shapley values to your data table you will also get a table script which creates a graph for you

You can change the type if wish to (stacked bar chart (default), stacked line chart, stacked area chart all work on some level).

You could also stack the data and visualize that. If your data doesn't have unique identifier for each row, you might want to create one and then stack the shap columns and keep that unique column. After you have stacked your data, you can create a bit different looking plots, for example

Link Mark provided has some information about the shapley values in JMP and some sources to check out.

There is also Shapley Values: Explaining Individual Predicted Probabilities and  

[#5]

Thank you for getting back to me jthi  @jthi . I took a bit of time to play around with the prediction profiler to calculate the shap values on the data table and create a graph. However, I am running into the following problems and wondering if you are able to either give some more insight or share resource that can help answer my questions below:

1) I am evaluating a total of three models, one of them is the boosted tree with smote plus tomek sampling method.

Please see the attached subset of the data (the data is large so I am attaching a subset of it). I ran the shapley values on the prediction profiler, but I did not see the newly calculated shapley values on the data sheet. I also selected to add shapley script to data table, but I did not see it added to the data table. I am wondering if this is possible to do  on the boosted tree? I was able to do this on generalized regression but I also didn't have to create the k fold cross validation for the generalized regression, I did need to do that for boosted tree. I assume with shapley values as an option on the boosted tree, I should be able to see the calculated shapley values on the data table. Some insight will be helpful in trying to figure out why I can't see the calculated shapley values on the data table.

2. How do I go about changing the type of graph outputed? For instance, how do I go about changing the type of graph you presented as an output above to dotted graph? What would I select to change from line graph to dotted graph on the right?

        TO

In the (video) link that you provided, it shows the outputs for the graphs, but they don't show the steps in how they got to those outputs. I am wondering if you can shed some light on how to go about changing the types of graphs, what would I select? Is it the control panel and then select the different types of graphs? When I do this, it changes the outcome totally. Please see attached for a subset of the data table. You mentioned "You can change the type if wish to (stacked bar chart (default), stacked line chart, stacked area chart all work on some level)," where do I go to change the type of stacked graph from the default bar chart to line chart?

Thank you for taking the time to provide some guidance.

Thank you,

Nisha

[#6 · Accepted solution]

I'm not sure which outputs you mean for graphs. Second plot you provided looks like it could be from shap python package Welcome to the SHAP documentation — SHAP latest documentation and not from JMP (you can get quite similar in JMP but it does require some work). SHAP documentation does also have additional examples you can use ideas how you could plot your data. 

After you have your shapley values in your graph, you can start using them in any way you find useful in JMP. Usually I think stacking them is best option, you can find stack platfrom from Tables menu Stack Columns in Data Tables (jmp.com). Stack the shapley value columns (+ feature value and row identifier if you want to) and then start plotting. I would still keep the other format as both of these have their own pros and cons.

[#7]

Hi Jarmo @jthi ,

You are right about the second plot, it is from shap python package, but I checked out the SHAP documentation that you recommended for additional examples and found it to be helpful in visualizing the data. Thank you for providing the direction in creating the graphs using the stack columns (this is new to me and I explored more to learn about it).

Thank you,

Nisha

[#8]

I found the following website to provide examples of interpreting different types of SHAP plots (though not all inclusive) and may be of help to others:

https://www.aidancooper.co.uk/a-non-technical-guide-to-interpreting-shap-analyses/