topic Re: Bad Stacking? in Discussions

Bad Splitting?

BHarris — Thu, 10 Jul 2025 17:57:38 GMT

Suppose I have this table:

Category	Item	Level
A	1	delta
A	2	foxtrot
A	3	hotel
B	1	juliet
B	3	lima

Note that Category=B, Item=2 is missing.

If I try to "split" this table with "Split By" = Category and "Split Columns" = Level, and "Keep All", it returns this:

Item	A	B
1	delta	juliet
2	foxtrot	lima
3	hotel

Note that on the newly Split table, it shows that Item=2,Category=B is "lima", but that's incorrect -- that cell should be blank, and "lima" should be under Item=3,Category=B.

Is this user error, or is this a bug in JMP?

Re: Bad Stacking?

Chris_Kirchberg — Tue, 22 Apr 2025 17:45:14 GMT

Hi @BHarris, try using Item in group. The result will be what you would expect.

Best,

Re: Bad Stacking?

BHarris — Tue, 22 Apr 2025 17:49:02 GMT

Yes, that does seem to work.

Can you explain what's going on? I'd like to understand why the Group step was necessary.

Re: Bad Stacking?

Chris_Kirchberg — Tue, 22 Apr 2025 18:11:16 GMT

Sure, Splitting without the group is doing it in a literal fashion by row and is not aware that Item has a missing component for category. Group forces level to separate out by Item for each category by taking a look at Item first then Category.

From online help (https://www.jmp.com/support/help/en/18.2/?os=mac&source=application#page/jmp/split-columns-in-data-tables.shtml:(

Group

Specifies a Group variable when you want your data to be split within each group of the selected variable. Each group results in a row in the output table. You must also specify the required variables, Split By, and Split Columns.

Note: If the variable that you want to group by contains unequal groups or is in a random order, specifying it as the Group variable ensures that your data is restructured properly, and any missing values are assigned in the appropriate places.

Otherwise you would have to put in an extra row and define Category and Item but leave Level blank for that row.

Hope that helps.

Re: Bad Stacking?

BHarris — Wed, 07 May 2025 18:03:04 GMT

The more I think about this, the less it makes sense. I don't understand why split would behave like that under any circumstance -- almost like the developers simply nested some loops and hoped that the data would fall in the right spots...

I'm left wondering if (a) I'm still too dumb to understand the underlying rationale for this behavior, (b) this was implemented this way to offer high performance (fast splits on large tables) where the inputs are full-factorial and the user is expected to know that assumption is being made, or (c) it's really just a bug that needs to be submitted/fixed.

(I won't be offended if it's (a). ;) )

Re: Bad Stacking?

hogi — Sat, 10 May 2025 19:50:44 GMT

Can you elaborate on your thoughts?

I would argue in the opposite direction:
If the input data is well structured, you can Split, without taking any care.

If the input data is not well structured - but has some grouping information in addition, there is always the possibility to use this grouping information to tell JMP what it should do.

What doesn't work:
the user knows the grouping information (there is the column) but doesn't tell JMP (via the GUI).
Then it's no wonder that JMP doesn't use the grouping information.

[there are other cases where JMP applies some guesses | auto correction - and the user wonders about the creativity].

Re: Bad Stacking?

BHarris — Wed, 07 May 2025 22:03:03 GMT

What's your take on the data set provided in the original question? Is it "well structured"?

And what's your take on JMPs current behavior when splitting that table?

My understanding is that "Split By" columns are those with values that you want to end up in the new table's column headers, and "Split Columns" are the columns whose values you want to end up in those new columns' cells. I still don't understand what "grouping" means in this context.

Re: Bad Stacking?

txnelson — Wed, 07 May 2025 22:16:51 GMT

Your initial data table is "well structured".

The Grouping concept can be thought about in terms as to what rows in the input data table are to be grouped together to define a specific output row.

Re: Bad Stacking?

hogi — Sat, 10 May 2025 19:52:01 GMT

Another way to explain:

Your columns "Category" and "Level" (alone) are not well structured - one needs some additional information like

@BHarris wrote:
Note that Category=B, Item=2 is missing.

to inform others about the fact: between row #5 and row #6 something is missing.

Case #1:
The user doesn't tell JMP about the existence of the column "Item" ("Item" is not used in the GUI **)
Then JMP doesn't have enough knowledge to do the job properly.

Case #2:
There is a 3rd drop zone. It allows the user to use

@BHarris wrote:
Note that Category=B, Item=2 is missing.

in a structured way - as a grouping column.

"Grouping" fits very well to what you want:
Provide JMP with additional information so that entries can be placed into specific rows, determined not by the Rank(row(), Categories) of the input data, but grouped via grouping columns.

**) edit:
"Item" is used in the GUI - implicitly via Keep All
but this is too "passive" - see below.

Re: Bad Stacking?

BHarris — Fri, 09 May 2025 21:46:36 GMT

@hogi -- you've jarred something loose here, the idea that I wasn't including the Item column anywhere in the interface, and therefore there's an implication that maybe the Split algorithm isn't even really aware of its existence.

I think I was believing that *all* columns were important in the Split operation, and it was trying to maintain all relationships where possible, e.g. that "lima" was also only associated with item "3". Now I'm left trying to figure out if all of the other columns should be grouping columns if I'm doing "Keep all"...

Someday I hope to achieve the status of JMP-Master, and be fully enlightened, and perhaps then it will make sense why JMP doesn't implicitly use all other "Kept" columns in the Group role. Until then, it will likely continue to make me nervous, as its behavior for me in these conditions isn't obvious, results in faulty output data under conditions that may be hard to identify, and is not well-enough documented for me to understand. At least that nervousness will keep me more alert in the future when using it.

Thx.

Re: Bad Stacking?

hogi — Sat, 10 May 2025 19:54:01 GMT

You mentioned it, but maybe it helps to write it explicitly:

JMP applies the rule:

- Group: Keep and use for grouping

- Keep: keep (~ don't drop)

@BHarris wrote:

Now I'm left trying to figure out if all of the other columns should be grouping columns if I'm doing "Keep all"...

I guess: yes.
You could try both ways, check the differences - and then decide.
Group helps you to "structure" the data. If the data is already structured, it doesn't hurt. If it is not structured, it will guarantee a meaningful output.
With Keep you have to know very precisely what you are doing. Below are some examples.

These little discussions in the community help a lot to understand - to challenge and polish the user's knowledge.
Here is a that came up in our user coaching sessions. I've collected them to help other users understand JMP. Many of the questions are just rhetoric one - they illustrate that sometimes it's just the "JMP slang" that makes understanding JMP logic non-trivial.

@BHarris wrote:

Until then, it will likely continue to make me nervous, as its behavior for me in these conditions isn't obvious, results in faulty output data under conditions that may be hard to identify, and is not well-enough documented for me to understand. At least that nervousness will keep me more alert in the future when using it.

I agree. It's very important to "know what JMP is doing" - and to know it down to a level of detail that the user feels capable to do the right thing. Not nervous, not surprised. If something unexpected happens, a user starts to feel anxious - where are the other ?!?!

Maybe there is a great feature in JMP, but it annoys or frightens users - at least those who don't expect what JMP does ...
Having a map of these trap doors can help users escape them. Every JMP user should have such a map to "feel safe" working with JMP.

Even if the details are well documented - JMP is soooo powerful, at the same time there is too litte and too much documentation!
This makes it difficult to find the right information. The LearnBot makes it much easier now, but it still does not know all the details.

Re: Bad Stacking?

hogi — Sat, 10 May 2025 08:15:44 GMT

An extreme case for Split:

Keep: name

no grouping column

With sex = M, F, 2 rows are merged into one row and many "names" are lost:

The first *) item wins, so we start with female names, the corresponding male names are dropped.
All male names? no -- at the end, there are some male students left.

*) There is NO universal rule which holds for all tables platforms:

which value will be kept and which value will be lost.

Join also keeps the first value (and doesn't care about subsequent values).
Update "keeps" every single value - and overwrites the previous entry - the last one "sticks".
This is why Tables/Update is so slow (it can take hours to execute the command):

Re: Bad Stacking?

hogi — Sun, 11 May 2025 09:49:41 GMT

@hogi wrote:

The first *) item wins

oops ... this is just true for columns added via Keep.

For the Split Columns, Split behaves like Update: the last one "sticks".

I tried to illustrate it with this example where I marked the "name" entries and the M/F split entries which end up in the final data set. One can see:
- columns added via Keep (like name) : Rank=1 (take first one, skip subsequent ones)

- rows added via Split Columns ( height --> M/F): RankReverse=1 (last one "sticks")

Re: Bad Stacking?

hogi — Sun, 11 May 2025 09:50:29 GMT

The behavior follows a logic, but one can argue whether the logic is straightforward / expected by every user : )

~~I wonder if Keep All is the right choice as default? If JMP starts with~~

~~... then a user who activates the option Keep All can consult the documentation to be sure that it does what he wants.~~
JMP could even show a warning: "Are you sure that you want to use Keep? better use Group?"

edit: sorry, I proposed what is already there:
This Platform already starts in "safe mode", with the "Drop all" setting.
And once the user activates "keep all", JMP SHOWS a warning [the one in post N-2].

________________________________________________

Under the line: No surprises | anxiety for users who don't use the option.

Much more dangerous than Split: Tables /Update
The default interface also starts with "All"
a) for Add Columns

b) for Replace columns (!)

Some years ago, I considered it dangerous enough to write an AddIn which replaces the default settings with:

The additional befit: with this change the platform doesn't start with a setting that tries to Update thousands of columns.
This makes it orders of magnitude faster than the original one : )
https://community.jmp.com/t5/JMP-Wish-List/Speed-up-Tables-Update/idc-p/653334/highlight/true#M4531

Re: Bad Stacking?

hogi — Sat, 10 May 2025 09:14:00 GMT

I added the topic to

Re: Bad Stacking?

BHarris — Sat, 10 May 2025 23:20:56 GMT

I'd love to see an animation someday that shows what split (and stack) are doing. I imagine the split-by column values being pushed up into the column headers, and the split-columns getting pushed into the cells of those new columns. (I was surprised just now to learn that if there are multiple split-columns for a given split-by column, then new numbered columns are created.)

But the real interesting part comes when there are conflicts, e.g. multiple values trying to end up in the same cell. JMP does give a warning that this is happening (which is great!), but this whole post came about from a dataset that looked like it was being split correctly, but after I dug into it I realized it wasn't, and I had ignored that warning because I didn't understand it.

My 2 cents, the warning should be changed from: "Multiple rows, possibly with different values, of the columns to be kept are mapped to the same row in the split table. Only one value of each column is retained." ---> to: "Multiple values being mapped to the same cell; only one is kept."

Re: Bad Stacking?

hogi — Sun, 11 May 2025 15:49:31 GMT

@BHarris wrote:

I was surprised just now to learn that if there are multiple split-columns for a given split-by column, then new numbered columns are created.

if multiple split-columns are used, JMP creates the column names as a combination of the name of the split-column - and the entry of the split-by-Column:

I only one column is selected as split-column, the name of the split-column is not included in the column name:

My colleagues roll their eyes when I start using Big Class to explain a functionality of JMP.
But it's a wonderful trick to use an alternative data set to play with the settings:
with "M" and "F" it's obvious what JMP is doing, no need to wonder about the "numbers" that appeared with the original data set.

Big Class looks very simple, but it's complicated enough to explain many of the details.
Just collinearities are missing ...