topic Re: transpose list of lists (summarize()) without a for loop in Discussions

transpose list of lists (summarize()) without a for loop

vince_faller — Fri, 28 Sep 2018 18:16:52 GMT

Can anyone think of a way to transpose a list of lists like what comes from the following script so that each row is grouped together instead of by column.

I know I can do it with a for loop but I'm wondering if there's anything better or less brute.

Names default to here(1);
dt = open("$SAMPLE_DATA\Big Class.jmp");
summarize(dt, d = By(:age, :sex));

rows = nitems(d);
cols = nitems(d[1]);

show(rows, cols);

// rows = 2;
// cols = 12;

For this I want to change it from 2 lists of 12 to 12 lists of 2.

Adding a few people that I think might either like this challenge or already have something.

@pmroz, @Justin_Chilton, @gzmorgan0, @msharp, @ms

Re: transpose list of lists (summarize()) without a for loop

pmroz — Fri, 28 Sep 2018 18:35:04 GMT

It's not a gold watch but it works.

Names default to here(1);
dt = open("$SAMPLE_DATA\Big Class.jmp");
summarize(dt, d = By(:age, :sex));

nrows = nitems(d);
ncols = nitems(d[1]);

show(nrows, ncols);

// nrows = 2;
// ncols = 12;

trans_list = {};

for (i = 1, i <= ncols, i++,
	one_list = {};
	for (k = 1, k <= nrows, k++,
		one_list[k] = d[k][i];
	);
	trans_list[i] = one_list;
);
show(trans_list);

trans_list = {{"12", "F"}, {"12", "M"}, {"13", "F"}, {"13", "M"}, {"14", "F"}, {"14", "M"}, {"15", "F"}, {"15", "M"}, {"16", "F"}, {"16", "M"}, {"17", "F"}, {"17", "M"}};

Re: transpose list of lists (summarize()) without a for loop

vince_faller — Fri, 28 Sep 2018 18:40:57 GMT

I feel like you didn't read the without a for loop part. :p

Re: transpose list of lists (summarize()) without a for loop

pmroz — Fri, 28 Sep 2018 18:59:25 GMT

DOH! I suppose you could write the results to a table, transpose that, and create lists from the table. Might still have a for loop in there though. Hmmmm...

Re: transpose list of lists (summarize()) without a for loop

msharp — Fri, 28 Sep 2018 19:50:59 GMT

PMROZ's nested for loops is probably the best. If you are looking for alternatives, I can think of two additional ways.

If your data is all numeric you could use matrices which would be pretty slick. This doesn't have for loops however, the list of lists returned from summarize returns all character values, even for numeric columns. Which means you would need a nested for loop to convert everything to numbers (if all your data is columns) so that won't be helpful for your problem.

The second option is to convert the list of lists to a data table, transpose that, then turn back. This requires two for loops for the conversions (I'm not aware of any slick way to convert between nested lists and data tables). This is probably slower than PMROZ's solution since you have the overhead of creating a data table, however, it's possibly faster for larger datasets. If you don't actually convert back to a list of lists and just use the transposed data table instead, you could save some time as well.

Names default to here(1);
dt = open("$SAMPLE_DATA\Big Class.jmp");
summarize(dt, d = By(:age, :sex));
rows = nitems(d);
cols = nitems(d[1]);
show(rows, cols);


//Turn List of Lists into DT ()
dtLL = New Table("Lists of Lists");
for(i=1,i<=nitems(d),i++,
	dtLL << New Column(char(i), Character, Nominal, Values(d[i]));
);
//Transpose DT
cols = dtLL << Get Column Names;
dtT = dtLL << Transpose(columns(cols));
//Turn back into list of lists
trans_list = {};
for(i=2,i<=ncols(dtT),i++,
	trans_list[i] = column(dtT,i) <<Get Values;
);
print(trans_list);


//Matrix if all numbers
summarize(dt, d2 = By(:age, :height));
//d2 is all characters...
m = Matrix(d2);
//Example with list of lists of numbers
d2 = {};
d2[1] = As List(dt:height << Get Values);
d2[2] = As List(dt:height << Get Values);
m = Matrix(d2);
mT = Transpose(m);
d2T = As List(mT);
print(d2T);

Re: transpose list of lists (summarize()) without a for loop

vince_faller — Fri, 28 Sep 2018 21:42:48 GMT

I found this way but it's slower. :(

Names default to here(1);
dt = open("$SAMPLE_DATA\Big Class.jmp")
summarize(dt, list = By(:age, :sex));

rows = nitems(list[1]);
cols = nitems(list);
list_flat = parse("{"||substitute(char(list), "}", "", "{", "")||"}");
insert into(list_flat, repeat({"@@@"}, rows));
list_sort = list_flat[	shape(1::rows*(cols+1), cols+1)` ];
remove from(list_sort, -1);
t_list = parse("{"|| 
	substitute(char(list_sort), 
		", \!"@@@\!",", "},{"
	) 
||"}");

Re: transpose list of lists (summarize()) without a for loop

ms — Fri, 28 Sep 2018 22:03:30 GMT

Here's an idea based on Repeat(). A function that essentially is a loop but returns the list directly, without the need to loop any assignments.

In earlier versions of JMP I experienced that Repeat() was way faster than For() in building lists but I think that was "fixed" in JMP 13.

Names Default To Here(1);
dt = Open("$SAMPLE_DATA\Big Class.jmp");
Summarize(dt, d = By(:age, :sex));

transposed_d = Local({i = 0}, Repeat({i++ ; Eval List({d[1][i], d[2][i]})}, N Items(d[1])));

Re: transpose list of lists (summarize()) without a for loop

msharp — Fri, 28 Sep 2018 22:31:12 GMT

Using the new Python hooks in JMP 14 I thought I could cheat. It mostly works.

Names Default To Here(1);
dt = Open("$SAMPLE_DATA\Big Class.jmp");
Summarize(dt, d = By(:age, :sex));

Python Init();
ml = Python Execute(
	{d},
	{x, d},
	"\[
import numpy as np
x = np.transpose(d) # matrix product
print('x=', x)
]\"
);
Show( x, d );
Python Term();

Re: transpose list of lists (summarize()) without a for loop

gzmorgan0 — Sat, 29 Sep 2018 09:17:04 GMT

Names default to here(1);
dt = open("$SAMPLE_DATA\Big Class.jmp");

//Since column formulas are by row, it uses a built in For Loop this maintains the table values but requires a table....
//This is not a solution for two generic lists

dt << New Column( "Age,Sex",
		Expression,
		"None",
		Formula( Eval List( {:age[Empty()], :sex[Empty()]} ) ),
		Suppress Eval
	);

myList=  (Associative Array(Column("Age,Sex")) <<get keys);

Re: transpose list of lists (summarize()) without a for loop

gzmorgan0 — Sat, 29 Sep 2018 09:13:49 GMT

MS, repeat soution is very nice!!

Re: transpose list of lists (summarize()) without a for loop

pmroz — Mon, 01 Oct 2018 17:48:05 GMT

Repeat: MS your repeat solution is very nice. :)

Re: transpose list of lists (summarize()) without a for loop

nikles — Fri, 15 Jan 2021 02:19:33 GMT

I had the exact same question @vince_faller . Thanks for asking. I've posted a request for a feature like this to be added to the next version of JMP. Let's see...

https://community.jmp.com/t5/JMP-Wish-List/Transpose-List-of-Lists-without-a-for-loop/idi-p/348968