Subscribe Bookmark
KarenC

Super User

Joined:

Feb 10, 2013

Why model visualization is integral to model building

Model visualization? Data visualization has gained traction in the past few years, with numerous interesting books and talks focusing on improving our data visualization skills. JMP’s own Xan Gregg recently spoke about data visualization on Analytically Speaking). Model visualization is simply applying data visualization to models.

When I can “see” the model, my confidence and my ability to share and explain the model also increases. While the Profiler is one model visualization tool in JMP not to be overlooked, here I will look at a simple data plot for a specific type of model as an illustration of model visualization.

One area that I work in is medical diagnostics (e.g., a blood tests for some condition). We often start with a large number (say 1,000) of potential model factors (often referred to as markers, as in a biological marker for a disease, condition or state) and then work to reduce this number to a small number (say 10) factors to build an algorithm (model) that can be implemented on a piece of lab equipment.

Throughout this process, we have candidate models to evaluate. ROC curves and the area under the ROC curve (AUC) are one standard way to evaluate such diagnostic models. However, while these are helpful, they don’t tell the whole story. Different-shaped curves can have similar AUC, and often one area of the curve (say where sensitivity is high) may be of more clinical interest than the whole curve.

 

Roc_JMP_6

In addition to ROC curves, I have found that a plot of the disease state by the model outcome using data points as well as violin plots (as shown below) aids in the evaluation and understanding of the model. This plot provides a sense of how well the model can separate the data and provides an immediate feel for how shifting the cut-off (the value used to distinguish positive from negative) impacts the diagnostic performance.

Such a plot is also useful in understanding whether the diagnostic test may be more useful as a three-way test where you can identify with confidence negatives and positive subjects but have subjects in the middle for whom additional information is warranted before a clinical judgment is made.

JMP_model_Viz_2

 

Model visualization is an integral part of my model building strategy. Do you have favorite model visualization?

2 Comments
Community Member

Simon Walton wrote:

A great post! Visualising a model can be really powerful, and it's great that so many researchers are warming to the idea. Vis can be useful not just in understanding the model, but also in creating/refining the model too as you've alluded to. Being able to tune parameters and visualise the results interactively is somewhat the holy grail. Our vis group wrote a paper about using visual analytics for model development (in a cybersecurity context) last year at IEEE VizSec: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=7312767. The visualisation of individual models wasn't the focus as such - more the overall process of looking after the models in an operational environment.

Community Member

DATA IN MOTION – Model Visualization wrote:

[…] JMP Blog Why model visualization is integral to model building […]

Article Tags