cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Choose Language Hide Translation Bar
lehaofeng
Level IV

Is there a quick way to transform column data type?

I have a question for help, thanks.
There is a table with more than 1200 columns, since many data values starting with “i”, there are about 900 columns, so I need to convert the column format from character to numeric, but with a for loop, it's too slow, is there a fast way?

 

7 REPLIES 7
txnelson
Super User

Re: Is there a quick way to transform column data type?

For clarification, when you state 'many data values starting with “i”', you are saying that within a column you have data values such as

     i78

?

If this is what you are referring to, dose the value need to be changed to 

     178

or just remove the "i" and leave

     78

or just set the value to a missing value

Jim
lehaofeng
Level IV

Re: Is there a quick way to transform column data type?

There are many values starting with i, we define these datas  as invalid values anyway, must be turned into missing values

shampton82
Level VII

Re: Is there a quick way to transform column data type?

You could try selecting the columns, right clicking and using 

shampton82_0-1716298508118.png

 

shampton82_2-1716298574876.png

 

 

lehaofeng
Level IV

Re: Is there a quick way to transform column data type?

For Each( {col, index}, {:a, :b, :c},
		col << Data Type( numeric ) << 
		Set Modeling Type( "continuous" )
	);

Thanks!  But it is still slowly when there are thousand columns with many values.

jthi
Super User

Re: Is there a quick way to transform column data type?

You can try to speed it up by making your data table invisible (open it as invisible/query it as invisible or use << show window(0)) and using << Begin Data Update before and << End Data Update after your loop.

-Jarmo
lehaofeng
Level IV

Re: Is there a quick way to transform column data type?

col_names=dt<<get column names(string);
st=hptime();
dt<<Begin Data Update;

for each({col,index},col_names,
	column(dt,col)<<data type(numeric)<<set modeling type(continuous)
);

dt << End Data Update;
en=hptime();
show(en-st)

Thanks!  It's still slow, 20minutes...

Is there a way to change the format before data importing?

 

jthi
Super User

Re: Is there a quick way to transform column data type?

Maybe (I don't think you have defined where the data is coming from at all)? In best case you can fix your data at the source

-Jarmo