Subscribe Bookmark RSS Feed

Bubble Plot 'crosses levels' error

gv

Community Trekker

Joined:

Jul 28, 2014

Hi,

I am trying to create an animated bubble plot and I am running into some problems.

On each row I have a 'state', 'step', 'time' and 'id'. The ID is unique for a set of rows (with step 1 until 10, with varying states).

My rows of data have a variable 'time', ie. some IDs have different times than other IDs. In some cases time also overlaps, ie. one 'ID' has multiple 'Steps' or 'states' with the same Time.

When I bin the time column into a number of larger intervals, the bubble plot starts and works, but only IF I make the bins large enough. However, the following error is displayed in the log:

ID column 'singleID' crosses levels with categorical axis column 'Step'. Change ID to 'Step'?

ID column 'singleID' crosses levels with categorical axis column 'State'. Change ID to 'State'?

When I do not bin time in a number of intervals, the bubble plot completely does not start and JMP prompts me to replace the ID with the other columns. When I decline, the bubble plot is not launched and  the following error is displayed:

ID column 'singleID' crosses levels with categorical axis column 'Step'. Change ID to 'Step'?

ID column 'singleID' crosses levels with categorical axis column 'State'. Change ID to 'State'?

DisplayBox[]

Unable to allocate enough memory in access or evaluation of 'Bubble Plot' , Bubble Plot(

Launch Dialog,

X( :Step ),

Y( :State ),

Time( :binTimeCentred ),

Coloring( :Stop condition ),

ID( :singleID )

)

My apologies for not including an example, preparing an example that I could post online would take some time. Therefore, I was hoping if anyone had some thoughts on how to fix this problem based on the description alone.

Best,

1 ACCEPTED SOLUTION

Accepted Solutions
Solution

Hi gv,

Thanks for the data and sample script - it's very helpful to see the case you are looking at.  On the issues:

  1. Memory - in order to do the animation, Bubble Plot will store the position of each bubble at each unique time.  Out of your 250k rows, I see 83k non-missing time values, and over 5k of them are unique.  Since each bubble only persists for 10 steps, the 5k time values require quite a bit of memory (7.7GB on my machine compared to 1.5GB in your binned example).
  2. The binning makes complete sense given the above.  No aggregation is done across ID's, so you will not lose any bubbles, but you will lose the ability to see some state transitions that occur very quickly - within the bin size.
  3. There is no way to avoid the "crosses levels" dialog for interactive use, but when run from a script the message will simply be written to the log, with no interruption.

If the time animation is not that important, other platforms might be helpful.  This example uses a Graph Builder Line element with Overlay role to show all of the transitions at once:

Graph Builder(
     Show Control Panel( 0 ),

     Variables( X( :Step ), Y( :State ), Overlay( :UniqueID ) ),

     Elements(

          Line( X, Y, Legend( 3 ), Row order( 0 ), Summary Statistic( "Mean" ) )

     ),

);

8386_GBLineStates.png

Within this graph, if you drag-select any part of a line it will select all rows with the same ID.

If you reshape the data to include all states transitions in a single row:

Data Table( "bubble plot test data" ) << Split(

      Split By( :Step ),

      Split( :State ),

      Group( :UniqueID )

)

then you can use the Parallel Plot platform to see a similar view.  In this platform, you can right-click > Customize to set the Polyline transparency to something like 0.01.  This will give you a shaded view where more common transitions are darker, and less common transitions are faded.  If you select a segment in any line, it will draw highlighted:

8393_PPStates.png

Parallel Plot(

      Scale Uniformly( 0 ),

      Center at zero( 0 ),

      Y(

            :Name( "1" ),

            :Name( "2" ),

            :Name( "3" ),

            :Name( "4" ),

            :Name( "5" ),

            :Name( "6" ),

            :Name( "7" ),

            :Name( "8" ),
      :Name( "9" ),

            :Name( "10" )

      ),

      SendToReport(

            Dispatch(

                  {},

                  "Parallel Coord",

                  FrameBox,

                  {Frame Size( 633, 420 ), DispatchSeg(

                        PolylineSeg( 1 ),

                        {Transparency( 0.01 )}

                  )}

            )

      )

);

I hope these ideas are helpful!

-Dan

7 REPLIES
gv

Community Trekker

Joined:

Jul 28, 2014

Hello,

I would still be interested in any suggestions you might have. Maybe, one of the JMP pro's could comment on the significance of the 'Crosses levels' error, since it is not mentioned in any of the available documentation. Also their appears to be a limit to the amount of data that can be used in a bubble plot, since I also receive an 'could not allocate enough memory', or could this be caused by some sort of error in my input data?  I have unfortunately been unable to reproduce the error with the sample data used in the bubble plot example: (Example of a Dynamic Bubble Plot).

Best,

danschikore

Staff

Joined:

Sep 21, 2012

Hi gv,

The 'Crosses levels' error is simply a warning, because in many cases it is not what the customer expected.

When you have a categorical axis, and two rows with the same ID map to different values on the axis, the question is where to put the bubble. If you decline to change variables, it should still work, but it is more difficult to read values from the graph.  It may not be clear if a bubble on X="North Carolina" really represents observations from North Carolina or if the bubble represents an aggregation of two bubbles with X="Virginia" and X="South Carolina".  The "averaging" is simply based on the value orders of the categorical column, so any combinations are possible.

I believe your memory issue relates to the (number of IDs) x (number of times).  If you have lots of IDs that only have data for a small subset of times, the storage needs may be larger than expected.  If you can share a dataset or comment further on the characteristics of the data we can confirm that this is the problem.

-Dan

gv

Community Trekker

Joined:

Jul 28, 2014

Hi Dan,

Thank you for your reply and explanation. I do have multiple rows for each id (:uniqueID), but these correspond to both different X-axis values (:Step) and time values (:starttimestate or (:binTimeCentred). I am lost why this would be considered a problem since the point of the ID is to follow one bubble across the other parameters right? But perhaps I am missing something. I have included some data. I am tied to using the 32-bit version (11.1.1) by the way so that is probably not going to help with memory issues as well.

My data describes a 'chain' (sequence) of states (:State) that occurs during a replication of a simulation (:Key), multiple chains can exist for one :key. The states follow each other in a number of steps (:Step) in what it is called a chain. Each new step in a chain is state which is different from the preceding state, and occurs at a moment in time (:StartTimestate). This creates about 255 different combinations of steps (:ID). But, in total about 25000 chains are observed (:uniqueID). Each chain always has 10 steps but data might be missing.

What I am trying to achieve is a visualization of how the chains behave in the 'state' and 'time' dimension. I would like to show how a bubble of each chain (:UniqueID) moves through its steps (x-axis) over time, and how its state (y-axis) changes. The result would thus be a bubble chart in which the bubbles always move from left to right (:step), but behave differently moving up and down (:state). I believe this would be a fun way to visualize patterns in a rather large multidimensional data set, the script will be used in a user interface, so I would rather not have warnings popping up. When Using the :starttimestate variable as time would be great, but this triggers the 'memory' error. Strangly using (rather large) time intervals (:binttimecentred) does work, I say strangly because this does not change the number of IDs..

Any thoughts are welcome, also on how to suppress the 'crosses levels' error

Thanks again!

Script:

data table("bubble plot test data") << Bubble Plot(

X( :Step ),

Y( :State ),

Time( :binTimeCentred ),

Coloring( :Stop condition ),

ID( :UniqueID ),

Bubble Size( 15.24 ),

Time Index( 2.5 ),

Orient Shapes( 1 ),

Label( "None" ),

Trail Lines( "Selected" ),

Set Shape( "Triangle" ),

Title Position( 8, 5.11 ),

SendToReport(

  Dispatch(

  {},

  "1",

  ScaleBox,

  {Min( -0.5 ), Max( 9.5 ), Inc( 1 ), Minor Ticks( 0 )}

  ),

  Dispatch(

  {},

  "2",

  ScaleBox,

  {Min( -0.5 ), Max( 10.5 ), Inc( 1 ), Minor Ticks( 0 )}

  )

)

);

Solution

Hi gv,

Thanks for the data and sample script - it's very helpful to see the case you are looking at.  On the issues:

  1. Memory - in order to do the animation, Bubble Plot will store the position of each bubble at each unique time.  Out of your 250k rows, I see 83k non-missing time values, and over 5k of them are unique.  Since each bubble only persists for 10 steps, the 5k time values require quite a bit of memory (7.7GB on my machine compared to 1.5GB in your binned example).
  2. The binning makes complete sense given the above.  No aggregation is done across ID's, so you will not lose any bubbles, but you will lose the ability to see some state transitions that occur very quickly - within the bin size.
  3. There is no way to avoid the "crosses levels" dialog for interactive use, but when run from a script the message will simply be written to the log, with no interruption.

If the time animation is not that important, other platforms might be helpful.  This example uses a Graph Builder Line element with Overlay role to show all of the transitions at once:

Graph Builder(
     Show Control Panel( 0 ),

     Variables( X( :Step ), Y( :State ), Overlay( :UniqueID ) ),

     Elements(

          Line( X, Y, Legend( 3 ), Row order( 0 ), Summary Statistic( "Mean" ) )

     ),

);

8386_GBLineStates.png

Within this graph, if you drag-select any part of a line it will select all rows with the same ID.

If you reshape the data to include all states transitions in a single row:

Data Table( "bubble plot test data" ) << Split(

      Split By( :Step ),

      Split( :State ),

      Group( :UniqueID )

)

then you can use the Parallel Plot platform to see a similar view.  In this platform, you can right-click > Customize to set the Polyline transparency to something like 0.01.  This will give you a shaded view where more common transitions are darker, and less common transitions are faded.  If you select a segment in any line, it will draw highlighted:

8393_PPStates.png

Parallel Plot(

      Scale Uniformly( 0 ),

      Center at zero( 0 ),

      Y(

            :Name( "1" ),

            :Name( "2" ),

            :Name( "3" ),

            :Name( "4" ),

            :Name( "5" ),

            :Name( "6" ),

            :Name( "7" ),

            :Name( "8" ),
      :Name( "9" ),

            :Name( "10" )

      ),

      SendToReport(

            Dispatch(

                  {},

                  "Parallel Coord",

                  FrameBox,

                  {Frame Size( 633, 420 ), DispatchSeg(

                        PolylineSeg( 1 ),

                        {Transparency( 0.01 )}

                  )}

            )

      )

);

I hope these ideas are helpful!

-Dan

gv

Community Trekker

Joined:

Jul 28, 2014

Thank you Dan for you elaborate answer. Your alternative approaches are interesting and your explanation on memory usage in relation to binning time has been very helpful. It is good to know that binning is a solution in this situation, although it does mean losing the visualization of some transitions.

Thanks!

danschikore

Staff

Joined:

Sep 21, 2012

An interesting way to summarize the data is to add a "previous state" column:

Data Table( "bubble plot test data" ) << New Column( "PrevState",

       Data Type( Character ),

       Formula( Left( Right( :statepath, 4 ), 2 ) ),

       Set Property( Value Labels, Arg( :State << Get Property( Value Labels ), 1 ) ),

       Set Property( Value Ordering, :State << Get Property( Value Ordering ) )

);

With this added column, a Graph Builder heatmap will show the frequency of each transition:

Graph Builder(

       Show Control Panel( 0 ),

       Variables( X( :PrevState ), Y( :State ) ),

       Elements( Heatmap( X, Y, Legend( 3 ) ) ),

       Local Data Filter(

              Location( {642, 61} ),

                        Mode( Select( 0 ), Show( 1 ), Include( 1 ) ),

              Add Filter(

                     columns( :State ),

                     Where( :State == {"01", "02", "03", "10", "11", "12", "20", "21", "22", "30"} ),

                     Display( :State, Size( 204, 174 ), List Display )

              )

       )

);

I used a Local Data Filter to exclude the transitions to a "00" (missing) state.  When you pause over a cell, the hover tooltip will show the count of matching transitions:

8404_GBHeatMap.png

This view could be used in combination with other platforms to select and highlight certain transitions.

gv

Community Trekker

Joined:

Jul 28, 2014

Interesting way of visualizing the transitions Dan. It would also be powerful to put the last state of each chain on the x-axis. From other analyses I have found out that some states will never lead (or will always lead) to a certain end state. Visualizations like this indeed work great in combination with transferring selections to other platforms.

I have been experimenting with ways to visualize all observed chains at once. I have included a column in the example dataset that contains the state of a row and all preceding states in its chain (:statepath), I have also assigned colors based on the state of the row, and made :state the row label. This information can be used to create something that resembles a tree structure of all possible chains. I use a stacked bar graph to display this information. Each :step is a bar, each segment of the bar a :state in that 'branch' of the tree. You can read the graph from left to right. Each segment of a bar in one steps splits up to become multiple segments in the next bar (:step). Changing the number of 'levels in view' on the top x-axis allows you to walk through the graph.

Feel free to provide feedback.

8392_pastedImage_0.png

Graph Builder(

            Size( 1142, 591 ),

            Show Control Panel( 0 ),

            Show Legend( 0 ),

            Variables( Y( :Key ), Group X( :Step ), Overlay( :statepath ) ),

            Elements(

                  Bar(

                        Y,

                        Legend( 5 ),

                        Bar Style( "Stacked" ),

                        Summary Statistic( "N" ),

                        Label( "Row Label" )

                  )

            ));