Dynamic bivariate binning

[#1]

UPDATE: I have modified the original add-in posted here because of a change that occurred between JMP 13 and JMP 14 that made it unusable with JMP versions <14. I have also added the option to have a spline fit through the bin means as well as added a different way of exporting the bin statistics. Happy improved binning! Yves

Here is a JMP add-in to do what I have called dynamic bivariate binning. It is useful to explore very noisy data by simply aggregating the X and corresponding Y data into bins of various sizes. The binning can be done uniformly along the X axis or dictated by the density distribution of the X data. It produces in a bivariate plot of bin averages with facultative error bars in both directions and some other options. It also provides dynamically changing least-squares statistics. Once you have chosen the desired level of binning, you can export the bin statistics to a new JMP file. Happy binning. Yves

[anaheilman]

Pretty cool addin!! Thanks for sharing it

[marxx]

Thank you very much yvesprairie... Would you be able to help, or anyone else who has successfully used the add-in?

I am trying the addin now but am not getting any window or results or interactive GUI after putting continuous variables in both X and Y fields of the initial pop-up window and clicking OK. The initial pop-up window with X and Y variable entry fields disappears but nothing shows up in it's place and no columns are created. I am on Windows 7 Enterprise running JMP 13.2.1 64bit.

Thanks you in advance, marxx

[yvesprairie]

HI marxx,

I haven't tested it on many platforms but it is not using anything fancy so I don't see why it would not work on JMP13 on Windows (I used JMP14 Pro on a Mac). One thing you could do is open the embedded JSL file and simply run it as a script. You might get more info in the log window as to what is not working. I include here a little screen capture video showing what it is supposed to do.

Let me know if I can help you more.

Yves

[marxx]

 Thank you for the reply yvesprairie, 

I will watch the video in a bit when it becomes available (looks like it's loading now) and try to run the emebedded JSL directly, then circle back here. And, sorry I forgot to mention I am actually running Pro as well. 

Thanks again for the followup, looks like a very cool add in... I use quite a bit of bivariate continuous fits and am eager to get the samples binned in this manner.

Thanks! marxx 

[yvesprairie]

It is uploaded now. If you can think of little features you would find useful, let me know. Yves

[marxx]

Thank you yvesprairie, will do! :-)

[marxx]

That video was awesome, very explanatory. Thank you much yvesprairie.

To make this comparable, instead of using my data I used the "Growth" sample data that comes with JMP, and I followed your suggestion to use the extracted JSL script directly by copy-pasting it into the Growth Data Set instead of running as an addin.

Unfortunately, still nothing happened when I clicked "OK" after the first dialogue where we enter the X and Y variables. 

I tried it in debugging mode, and this is the error message that popped up:

Matrices dimensions do not conform, 72 by 1 and 72 by 2 in access or evaluation of 'Subtract' , ym -  /*###*/axm * lsbetas/*###*/

I tried to google this error and couldn't find much other than it appears that the matrices are of different sizes, but since there are the same number of rows for the only two columns of data in the Growth Sample Data I couldn;t understand how this could be.

Any help you could provide would be greatly appreciated.

Thanks! marxx

[yvesprairie]

Hi Marxx,

This is strange. It is as if the subtraction is evaluated before the matrix multiplication, which it shouldn't.

Look for line 219 and add a set of parenthesis as in

lsres = ym - (axm * lsbetas);

It turns out tht this line is not really needed anyway and is a relict of a previous script. You  could also simply delete lines 219, 220 and 225 (or put // at the beginning of each). Try that and let me know.

Yves

[marxx]

Hi Yves, 

Thanks very much, I was able to "//" those lines (although they were like row #188 or so in the extracted script) and get to the next window.

It shows the original data ... However, there seem to be two (independent?) issues that prevent from using the program further.

In the first issue, no bins show up in the initial presentaiton of the plot when the window opens.

When I try to slide the "Binning adjustment" slider I get the following JMP Alert pop-up error message:
"y must have only 1 column"

Please note that I can click at a point on the slider scale other than where the diamond is at, and after I click OK on the error message pop up described above, it will show the bin points. I can click elsewhere on the slider and (after the error message appears and is dismissed with an OK) it will adjust the binning, but it will not slide.

The second thing seems to be that the GUI will not plot "OLS fit line" of either original or binned data. I can check the boxes and the checks will appear but no line of fit appears. However, please note- the "Binned fit" R-square and Number of Obs does change, but no line appears in the plot.

The log throws this message when the OLS fit line is checked:

invalid subscript (must be number or list of numbers) 51002 times in access or evaluation of 'Subscript' , betas[/*###*/2]

Would you have any tips on progressing through these?

Thanks much, marxx

[mujahida]

I have had a try on this addin, it is great, but it cannot fit equation/function of spline and lines. Can it be improve this function?

[yvesprairie]

Hello Mujahida,

First of all, sorry for the delay in responding. I don't quite understand what is happening because the function works well for me. What version of JMP are you using? 

Best regards, Yves

[mujahida]

Hi!  yvesprairie

yes, it can work well, for output report, I can write following function:

y=-7.1819+0.60695x

y=-5.1756+0.54369x

but, What's the spline's function/formula?

[yvesprairie]

Hi mujahida,

In response to your question, the spline function is not a simple mathematical function, it involves a series of coefficients  that change along the X according to certain intervals. It is not practical to export a matrix of coefficients.

Regards, Yves