topic Re: Convert List to Matrix in Discussions

Convert List to Matrix

MarkDayton — Fri, 11 Jan 2019 18:30:06 GMT

What is the quickest way to convert a list: {"a", 1, 2, "b"} into a matrix: [. 1 2 .]?

I expect that I can just loop through all of the elements, but I'm hoping there is a quicker way since I will be doing this many thousands of times in my application, with each list containing hundreds of elements.

Re: Convert List to Matrix

uday_guntupalli — Fri, 11 Jan 2019 21:25:09 GMT

@MarkDayton,

I don't know if using a data table is ok ? if yes, here you go . Since the data table is private, I would expect it to be more than reasonably fast for hundreds of values

MyList = {"a", 1, 2, "b"} ; 

dt = New Table("Temp",
		New Column("C",Numeric,values(MyList)),
		"Private"
              );
Mat = dt << get as matrix; 
Close(dt,"No Save");
Show(Mat);

Re: Convert List to Matrix

MarkDayton — Fri, 11 Jan 2019 21:26:11 GMT

Thanks, I'll give that a try.

Re: Convert List to Matrix

txnelson — Fri, 11 Jan 2019 22:02:45 GMT

Looking at your original question, you wanted a matrix of [1,2]. I have added in a statement to get rid of the missing values

MyList = {"a", 1, 2, "b"} ; 

dt = New Table("Temp",
		New Column("C",Numeric,values(MyList)),
		"Private"
              );
dt << select where( ismissing(:c)) << delete rows;
Mat = dt << get as matrix; 
Close(dt,"No Save");
Show(Mat);

Re: Convert List to Matrix

MarkDayton — Fri, 11 Jan 2019 22:13:37 GMT

Actually I need the missing values. Sometimes a missing value isn't a problem, for example a test is added later on creating additional columns in the dataset. Earlier data isn't out of spec for tests that weren't performed. Unfortunately, sometimes missing data is a problem, test aborts but prior results are in spec. I haven't figured out how to deal with that yet, but I am making progress.

Re: Convert List to Matrix

ms — Sat, 12 Jan 2019 00:16:20 GMT

This is pretty fast:

MyList = {"a", 1, 2, "b"};
Mat = Number Col Box("", MyList) << get as matrix;

Re: Convert List to Matrix

MarkDayton — Mon, 14 Jan 2019 14:17:53 GMT

Practically perfect! Question, what happens to the Number Col Box? Does it stay in memory, or is it only created long enough to send the message to?

Re: Convert List to Matrix

MarkDayton — Tue, 18 Feb 2020 14:07:38 GMT

To be perfect, I did this:

data = shape(Number Col Box("", dt[1,0]) << get as matrix, 1, N Cols(dt));

Now, given similarly constructed matrices for the LSL/USL specs, I can do:

If( max(data < LSL) > 0 | max(data > USL) > 0, "FAIL", "PASS")

to check a row of data against the limits specified in the column properties, without having to loop through the columns for every row - taking a mXn process and turning it into a m+n process.

Re: Convert List to Matrix

ms — Mon, 14 Jan 2019 14:54:02 GMT

Not sure about memory handling here. But since the NumberColBox isn't drawn or stored in a variable I don't think it uses or locks up significant amounts of memory.

It also seems to scale linearly: If looped 10000x , 100000x and 1000000x it takes 0.27, 2.7 and 26.7 seconds, respectively. No difference in memory usage.

Applied to this small list, the number col box is about 30 times faster than the data table approach posted above. For very long lists – there may be an upper limit for how many items a NumberColBox can hold – the data table method may be more efficient.

Re: Convert List to Matrix

MarkDayton — Mon, 14 Jan 2019 17:09:57 GMT

BOOYAH!, Declaring victory and going home! On my data table with 30,000 rows and 150 columns, my routine went from a runtime of 8.5 minutes looping through every column on every row and checking against the limits for that column, to a runtime of 4.5 SECONDS checking a whole row in one shot! As I said, BOOYAH!