BookmarkSubscribe
Choose Language Hide Translation Bar

Suggest faster & better ways to flatten varying lengths of matrices

All,
While I know of a way to do this, I dont think it is the most efficient way. Any thoughts to make this more efficient ?

Matrix1 = [1,2,3,4,5];
Matrix2 = [2,3,5];
Matrix3 = [7,8,9,10];

ColAVals = {};
ColBVals = {};
ColCVals = {};
for(i = 1, i <= N rows(Matrix1), i++,
for(r = 1, r <= N rows(Matrix2), r++,
for(s = 1, s <= N rows(Matrix3), s++,
Insert Into(ColAVals,Matrix1[i]);
Insert Into(ColBVals,Matrix2[r]);
Insert Into(ColCVals,Matrix3[s]);
);
);
);

Show(ColAVals);
Show(ColBVals);
Show(ColCVals);

dt = New Table("Test");
dt << New Column("A",Numeric,Continuous, << set Values(ColAVals))
<< New Column("B",Numeric,Continuous, << set Values(ColBVals))
<< New Column("C",Numeric,Continuous, << set Values(ColCVals));
Best
Uday
1 ACCEPTED SOLUTION

Accepted Solutions  Jeff_Perkinson Community Manager

Re: Suggest faster & better ways to flatten varying lengths of matrices

This method does seem to be relatively quick.

nmat = 15;
matrices = {};
For( i = 1, i <= nmat, i++,
matrices[i] = J( Random Integer( 5 ), 1, Random Uniform() )
);

start = Tick Seconds();
m1 = As Table( matrices );
m2 = As Table( matrices );
dt = m1 << Join( With( m2 ), Cartesian Join, private );
Close( m1, nosave );
Close( m2, nosave );

For( i = 3, i < N Items( matrices ), i++,
mi = As Table( matrices[i] );
dt = dt << Join( With( mi ), Cartesian Join, private );
Close( mi, nosave );
);
mlast = As Table( matrices[N Items( matrices )] );
dt = dt << Join( With( mlast ), Cartesian Join );
Close( mlast, nosave );

end = Tick Seconds();
total = end - start;
Show( total );

By the way, be careful with Cartesian joins, they get big really fast. ;-)

-Jeff
4 REPLIES 4  Jeff_Perkinson Community Manager

Re: Suggest faster & better ways to flatten varying lengths of matrices

I haven't done any timing tests but using As Table() like this may be faster.

matrix1 = [1,2,3,4,5];
matrix2 = [2,3,5];
matrix3 = [7,8,9,10];

maxrows=maximum(nrow(matrix1), nrow(matrix2), nrow(matrix3));

matrix1 = matrix1 |/ j(maxrows-nrow(matrix1), 1, .);
matrix2 = matrix2 |/ j(maxrows-nrow(matrix2), 1, .);
matrix3 = matrix3 |/ j(maxrows-nrow(matrix3), 1, .);

mat=matrix1 || matrix2 || matrix3;

dt=as table(mat);
-Jeff

Re: Suggest faster & better ways to flatten varying lengths of matrices

@Jeff_Perkinson
I am worried about the looping rather than the conversion into the data table. Also, I want my resulting table to have all possible combinations  like Best
Uday  Jeff_Perkinson Community Manager

Re: Suggest faster & better ways to flatten varying lengths of matrices

So, you want the cartesian join of all of your matrices?

Again, I haven't done any timing tests but what about this?

Matrix1 = [1, 2, 3, 4, 5];
Matrix2 = [2, 3, 5];
Matrix3 = [7, 8, 9, 10];

dts = {};
dts = As Table( matrix1 );
dts = As Table( matrix2 );
dts = As Table( matrix3 );

dt = dts << Join( With( dts ), Cartesian Join );

For( i = 3, i <= N Items( dts ), i++,
dt = dt << Join( With( dts[i] ), Cartesian Join );

);
-Jeff  Jeff_Perkinson Community Manager

Re: Suggest faster & better ways to flatten varying lengths of matrices

This method does seem to be relatively quick.

nmat = 15;
matrices = {};
For( i = 1, i <= nmat, i++,
matrices[i] = J( Random Integer( 5 ), 1, Random Uniform() )
);

start = Tick Seconds();
m1 = As Table( matrices );
m2 = As Table( matrices );
dt = m1 << Join( With( m2 ), Cartesian Join, private );
Close( m1, nosave );
Close( m2, nosave );

For( i = 3, i < N Items( matrices ), i++,
mi = As Table( matrices[i] );
dt = dt << Join( With( mi ), Cartesian Join, private );
Close( mi, nosave );
);
mlast = As Table( matrices[N Items( matrices )] );
dt = dt << Join( With( mlast ), Cartesian Join );
Close( mlast, nosave );

end = Tick Seconds();
total = end - start;
Show( total );

By the way, be careful with Cartesian joins, they get big really fast. ;-)

-Jeff