Venn Diagram

[#1]

This add-in constructs Venn diagrams with either constant or proportional areas.  The former accommodate up to five variables and the latter are limited only by the distribution of the counts corresponding to the variables.

The proportional-area diagrams call methods kindly provided by Lee Wilkinson and documented in his paper “Exact and Approximate Area-proportional Circular Venn and Euler Diagrams”, available at http://www.cs.uic.edu/~wilkinson/Publications/venneuler.pdf  The corresponding computations are performed in Java, so you must have Java installed on your machine to run them.

The input can be either a single table with 0-1 variables or multiple tables with lists of items.  For details, click Add-Ins -> Venn Diagram -> Help after installing the add-in.

An example screenshot is below.

[rossmiller]

is there a way to format the data labels?  e.g. adding a comma to the number, changing .10 to 10%, etc, or hiding the labels so i can paste in my own?

[russ_wolfinger]

Basic editing is available by expanding the "Labels" outline box on the right hand side and making changes in the boxes within.  Please let us know if you need more advanced formatting.

[russ_wolfinger]

Ah sorry, that previous comment applies only to the labels for each Venn circle, not to the data counts inside the circles.   To format those counts I guess some functionality would need to be added.

[nsadot]

This is really a great tool. Wish it had a recall button

[32103940]

great addin! ditto on the Recall button

[russ_wolfinger]

Hi Venn Friends,

Just uploaded an update, featuring

- Automatic recall of specs in the dialog

- New dialog field that lets you specify a variable with a JMP format, and this format then gets applied to the counts

- New radio button in the display window lets you switch from counts to proportions, allows you to display percentages using the aforementioned formatter

- New checkbox "Density Points" provides a visual display of count density in the diagram

- Example data is now a part of the menu

Please give these a whirl and let me know if you have any problems or ideas for improvement! 

[MKostura]

Just would like to ask about a functionality quesiton with the Venn Diagram Add-In.  It seems that it is not linked to changes that might occur in row states (eg Hide and Exclude).  If I hide and exclude row those still are being counted as part of the Venn Diagram.  Is there something that I may be overlooking in setting up the Venn Diagram or might this be bug?

[russ_wolfinger]

Good catch MKostura--just uploaded an update that now honors excluded rows.  

[MKostura]

Thanks.  Nice first step.  But making this Venn Diagram addin completely and dynamically  interactive with row selection would greatly increase utility and uptake by the community.   Right now the fix allows for hidden/excluded rows to function but only as a refresh.  The user has to start over to get the updated diagram.  Connecting to the local data filter would help immensely. 

This would be an excellent addition to the Graph Builder graph type menu.

[irinastl]

Can I build VENN diagram in JMP 13Pro?

[Jeff_Perkinson]

Hi @irinastl,

Yes, this add-in works with JMP Pro, version 13 to build Venn diagrams.

Go to the top of this post and download the .jmpaddin file and open it in JMP Pro 13. You'll find a Venn Diagram menu at the bottom of your Add-Ins menu.

[abh1]

This is a great add-in. However, when I check the box for "Proportional Areas" I get the following error in a JMP Alert window: invalid subscript (must be number or list of numbers) in access or evaluation of 'cN0[idx]' , cN0[/*###*/idx] The Java version of the proportional diagram is produced, but it never gets into a JMP output window. I have tried this in both JMP Pro 13 and 14, and tried the SAS supplied java.exe as well as the java.com supplied version. All give the same error. Non-proportional diagrams work fine and look wonderful. Since the proportional diagrams are partially produced, I can resize and move all the circles to match that, but all that editing gets cumbersome pretty quickly. Is there a simple fix for this problem?

[John_K]

Does this only work on Win version of JMP ?

[abh1]

No, it works in JMP for Mac too. I think all JMP add-ins work anywhere. But this one also give an error in JMP for Mac when checking the box for proportional areas, although it is a different error message than in Windows.

[John_K]

Russ - thanks for all the recent help making this Add-In even better.

Now that the hard stuff is done... :)  I also echo MKostrua's comments above:

"But making this Venn Diagram addin completely and dynamically  interactive with row selection would greatly increase utility and uptake by the community.   Right now the fix allows for hidden/excluded rows to function but only as a refresh.  The user has to start over to get the updated diagram.  Connecting to the local data filter would help immensely.  This would be an excellent addition to the Graph Builder graph type menu."

[AC_Repka]

Hi Russ,

Love the venn diagram tool. I was wondering if an update that allows for display of shared object names (instead of just object counts) in the body of the graph would be possible?

Thanks

[russ_wolfinger]

Hi All,  Just in time for Santa's sleigh, attached is an update that should fix proportional area running Java on the Mac, as well as the cN0[idx] error.

@MKostura @John_K  there are some tricky issues in implementing interactivity from the JMP table back to the graph.  For example, exclusion/inclusion would trigger recalc of proportional areas, which can take 5-10 seconds.  Maybe not too bad for basic use cases but if you are making such changes frequently could get clumsy.   More difficult is how to display partial selection in the diagram itself.    For example, if only some of the rows of a Venn sector are selected in the table, how do we visually cue this?   My colleague Drew Foglia (now also a contributor) has been investigating a little and has the idea of water-fill shading from the bottom up in each sector.   Anyone seen any research along these lines?

@AC_Repka  Thanks, I'm not quite following.  You want to completely hide the object counts and instead show a list of states that define a particular sector inside each?   Wouldn't this get completely unmanageable for medium and larger data sets?  I may be completely missing the idea.  Do you have an example picture?

[galactus3000]

when doing proportional venn diagram with Single Table with 0-1 variables, I'm getting the error meansX[i] and i've traced it to around line 1240 of the VennDiagram.jsl code

at first i thought it was a typo and should have been meansX[idx], but meansX only his 8 elements, so upon show(i), I found that i was 256.

the error seems to occur, in my case, when idx=16, so i guess the if statement conditional is not met until then

// fill in any missing ones using average of circle centers
        cN0 = cN;
        show(two_to_nc);
        show(nc);
        show(i);
        for(idx=1, idx<=two_to_nc, idx++,
            if(cN0[idx]==0 & counts[idx]>0,
                cX[idx] += meansX[i];
                cY[idx] += meansY[i];
                cN[idx]++;
            );
        );

view log shows

meansX = [61.44375, 45.68125, 65.83125, 46.16875, 58.4375, 43.08125, 63.8, 62.90625];
meansY = [56.925, 41.56875, 47.0125, 50.83125, 56.51875, 36.85625, 50.9125, 52.375];
two_to_nc = 256;
nc = 8;
i = 256;
invalid subscript (must be number or list of numbers) in access or evaluation of 'meansX[i]' , meansX[/*###*/i]

[galactus3000]

the script does not crash (but resulting Venn diagram is ugly) if I replace meansX[i] with meansX[8] and same with meansY[i] to meansY[8]

[galactus3000]

guessing that meansX[i] should be meansX[ceiling(log(idx,2))] ?

[russ_wolfinger]

Just pushed a fix for this; apologies for the trouble.   Please download the updated attachment and reinstall the add-in.

[Smn]

Hi Russ and all,

Thanks a lot for this! Essential tool!

clarifications:
*Density points: can you give more methodological info on this (i.e. for reference in scientific articles)? BTW each time I run the same dataset the position of the density points changes.
*Format variable: how can this be used? Any "manual" to read?

Previous points/issues:
* Russ, I think @AC_Repka meant exactly that. For example it would be great for smaller size datasets. So, instead of displayed numbers or proportions, for each area, we would have the names - for ex., if I have "3" COLORS in a certain area of te Venn diagram, instead of displaying the number ("3"), we would choose to display their 3 names (for ex. "blue, red, green").
* @MKostura and @John_K Yes good points.
*Proportional areas: I just tried the new addin, and now it works (JMP 14).

Suggestions/ideas:
*Variables: the function "BY" would be very useful.
*Adding the "Keep dialog open" option would be very useful.

Thanks!

[AC_Repka]

Sorry I missed your message in the holiday scuffle. Yes, I think an option
for counts and/or list of object names would be very helpful. It may not be
great for large data sets, but I think it could be up to the user to decide
what is a reasonable way to display the data. Something along the lines of
this:
[image: image.png]

Thank you for your reply,
AC

[caseylott]

Thank you to @russ_wolfinger for this super cool add-in! I was wondering if it could be updated to either: 1) label POINT values when density points are displayed, 2) select records in the table associated with a single point (rather than all the points in a sector) or 3) display information about individual points (sample number, species, trait, etc.) as a hover action  (or any other method that would show multiple attributes for any given point)? 

Any of these options would help answer questions about individual samples that comprise any given sector. Right now, when I have a sector with many points and I select one (which is actually just selecting the sector) I have to search through all the selected records in the table and guess which record applies to a specific point of interest in the graph.

Thanks again for this tool. Casey  

[russ_wolfinger]

Hi Vennatics,

Just uploaded a new version with the following updates:

1. Before running proportional-area Venns, you must now define the environment variable $JAVA_HOME to point to a path that contains java.exe.  This applies for both Windows and Mac.  You may need to restart JMP after doing this for it to take effect in the add-in.

2. New checkbox in the dialog to leave it open. 

3. Several minor tweaks to output elements.

4. New checkbox "Data Points" in the output window. The idea is to create randomly jittered  X-Y coordinates for all rows in the input table and display them inside their appropriate Venn sector.  (“Density Points” is an older feature with a similar idea, but it plots fewer points and they are not linked to the table.)   New columns named VennSector, VennX, and VennY are added to the table so you can save the table and reuse the points or create coordinates of your own.  Here is an example:

In order to be able to show the points dynamically we switched the base platform from a Graph Box to Graph Builder.   There is now the usual two-way interactivity between the points and table, and consequently between any other graphs derived from the table.   You should even be able to create multiple, interactively-linked Venns if you run the add-in from scratch each time.  Labels on points should also work.

Enjoy and please keep the great feedback and ideas coming!

[caseylott]

@russ_wolfinger Thank you, Russ! This is a great addition and the interactivity opens up a lot of new doors for an analysis I'm doing. I really appreciate this add-in.

Casey

[Skilng]

@russ_wolfinger or any others- I had it working for proportional on another PC, but seem unable to get it to work on my new PC. Have installed java, and set the correct environmental variable $JAVA_HOME to the path for java.exe. But I when run with proportional I always get the ERROR about the environmental variable. Have restarted and reinstalled both java and JMP, and confirmed the environmental variable is functional.

Any advice? I am on windows 10 pro. Appreciate it, and the add-in itself of course!

[Skilng]

I think I found the problem- the environmental variable needs to be definded as "JAVA_HOME" not "$JAVA_HOME" as appears in the warning. Perhaps I had tried this on my old instal and forgot.

[russ_wolfinger]

Thanks @Skilng, just uploaded an update that changes the error message and doc to say "JAVA_HOME" instead of "$JAVA_HOME".

[Skilng]

@russ_wolfinger Hi the global variable name change was made, but now I get an error whenever running the latest add-in (was some other more critical code changed?)-

[ablavoie]

@russ_wolfinger I'm seeing the same error as @Skilng - any chance there is a fix for this? This looks like a great tool, thank you so much!

[russ_wolfinger]

Apologies @Skilng and @ablavoie , I forgot to include one of the updated files in the jmpaddin zip file.  Please give the new attachment a try.  In the dialog, you should see optional fields allowing for specification of X and Y coordinates of the points in case you want to use results from a previous run or roll your own.   

[RossS]

Hi I am getting the JAVA error above related to proportional diagrams, whenever I run with more than 2 groups.

How do I go about fixing this.

I am windows 10.

Thanks

[russ_wolfinger]

@RossS,  Need more details.  Have you defined JAVA_HOME?

[RossS]

I don't think so. I don't know how or where to I define JAVA_HOME?

[russ_wolfinger]

Click Add-Ins > Venn Diagram > Help to open a doc that contains instructions on how to define a JAVA_HOME environment variable.   We did it this way to avoid more complicated problems that can occur without it.

[russ_wolfinger]

Venn Friends,

Drew Foglia (@drewfoglia) has kindly just done some nice streamlining work that alleviates the need to define JAVA_HOME before fitting proportional area diagrams.   The updated add-in at the top should now find java automagically on Windows or Mac--just download and drag it onto your JMP window to reinstall.   As always, let us know if you encounter any problems or have ideas for improvements.   Wanted to also mention that integrated Venn functionality is available within JMP Genomics and JMP Clinical.   

[drewfoglia]

Thanks, Russ. Note, if for some reason JMP cannot find a Java runtime on your OS (Windows or Mac) you should still be able to set JAVA_HOME and that runtime will be used. Thanks!!

[RossS]

Hi,

I am getting the following error in the log when I try to run proportional venns

counts = [34, 0, 9, 3, 4, 0, 11, 39];
"Using installed JRE from: C:\Program Files (x86)\Common Files\Oracle\Java\javapath\java.exe
C:\ProgramData\Oracle\Java\javapath\java.exe
C:\Windows\System32\java.exe"
ERROR: Venn Diagram proportional area calculations did not complete successfully. Check the cmd.exe window for details and/or try different variables.

[russ_wolfinger]

Hi @RossS    I created a mock table based on the counts you posted and it runs okay for me (see picture below).   Are there any clues in the cmd.exe window as the ERROR message indicates?   Have you set JAVA_HOME as in the notes above?   Does the same thing happen with the example data that comes with the add-in?   Are you on Windows or Mac?

[ezorlo]

I very much appreciate this Add in!

I used it recently on non-proportional circles setting for some environmental data and it generated the most beautiful graph I've ever seen.

See below

#nature is lit

[rossmiller212]

Hi Russ, can you have the labels formatted based on the source data?  I am using % of total as an example and it shows .026 versus 2.6%

[russ_wolfinger]

@rossmiller212  The Format Variable in the dialog lets you specify any format, including percentages.

[rossmiller212]

thanks for the quick reply @russ_wolfinger 

where is that in the dialog?

[russ_wolfinger]

@rossmiller212  It's in the initial dialog:  Add-Ins > Venn Diagram > Single Table with 0-1 Variables.  

[rossmiller212]

got it, thanks Russ!

[caseylott]

@russ_wolfinger 

Hi Russ,

I just tried running the add-in on a new computer. I followed instructions to set the JAVA_HOME location and I formatted my data table the same way as your example data called "Single Table with six 0-1 variables". I then set up the tool dialog as follows: with all of my categories entered as 0-1 variables with the proportional areas box checked (see below).  When I hit ok I keep getting the following error: not subscriptable value. Do you have any idea why this might be happening and what I could do to fix it?

Casey

[russ_wolfinger]

Hi @caseylott   Thanks for reporting this problem.   Just relooking at the code it appears there is a hidden limit of ten 0-1 variables.  You're the first person I've encountered that has tried more than that.   I just increased the limit to 50 and updated the add-in--please download and reinstall it.   The proportional area Java algorithm increases in complexity with number of variables and it is doubtful it will produce meaningful results in any reasonable amount of time for more than 50.   Looks like you are trying around 100.   If you don't mind, please try sets of 50 or less and let me know if you think pushing beyond this is worth a try.    It might be preferable to use two-way Hierarchical Clustering or some form of dimension reduction like PCA to discover patterns in these 0-1 variables.   

P.S.  We have a new Multivariate Embedding platform coming in JMP 17 that will have T-SNE and UMAP.   T-SNE is available currently with JMP Pro Early Adopter 6, which you can request at jmp.com/earlyadopter.  

[caseylott]

Hi Russ,

Excellent and thanks! I’ll report back tomorrow. It’s kind of a bizarre boundary pushing application for a systematic literature review where I’m comparing which references were returned from unique searches of 65 different information sources. I can pare this down to 50 no problem. Thanks for making the change. I’m interested to see how this goes.

Casey

[Nazarkovsky]

Dear Russ,

Yesterday I installed the add-in and JAVA following the path to indicate the variable - tried both JAVA_HOME and $JAVA_HOM. But the same error occurs repeatedly - after restarting both PC and my JMP Pro 16. 

Could you please give some advice? Windows 10.