I have a question for help, thanks.
There is a table with more than 1200 columns, since many data values starting with “i”, there are about 900 columns, so I need to convert the column format from character to numeric, but with a for loop, it's too slow, is there a fast way?
For clarification, when you state 'many data values starting with “i”', you are saying that within a column you have data values such as
i78
?
If this is what you are referring to, dose the value need to be changed to
178
or just remove the "i" and leave
78
or just set the value to a missing value
There are many values starting with i, we define these datas as invalid values anyway, must be turned into missing values
You could try selecting the columns, right clicking and using
For Each( {col, index}, {:a, :b, :c},
col << Data Type( numeric ) <<
Set Modeling Type( "continuous" )
);Thanks! But it is still slowly when there are thousand columns with many values.
You can try to speed it up by making your data table invisible (open it as invisible/query it as invisible or use << show window(0)) and using << Begin Data Update before and << End Data Update after your loop.
col_names=dt<<get column names(string);
st=hptime();
dt<<Begin Data Update;
for each({col,index},col_names,
column(dt,col)<<data type(numeric)<<set modeling type(continuous)
);
dt << End Data Update;
en=hptime();
show(en-st)Thanks! It's still slow, 20minutes...
Is there a way to change the format before data importing?
Maybe (I don't think you have defined where the data is coming from at all)? In best case you can fix your data at the source