cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Try the Materials Informatics Toolkit, which is designed to easily handle SMILES data. This and other helpful add-ins are available in the JMP® Marketplace
Choose Language Hide Translation Bar

Fast text file format for importing to JMP

I produce large delimited data files (typically csv, or tsv) which will ultimately be imported into JMP by users. Typically 1-3GB in size. I can specify the format of the files and would like to get some opinion on which is the best non JMP format to use and are there any hacks for speeding up the importation of large text based data files. 

 

Cheers, Troy

 

 

 

2 REPLIES 2
hogi
Level XII

Re: Fast text file format for importing to JMP

Depending on the settings in the preferences, it's possible to import txt import - with point and comma .
"not possible" means: data can be imported but numbers will be interpreted as characters.

 

JMP can handle numbers much more efficiently than text. The bottleneck to import them as strings - an then convert them to numbers could be too narrow to get your 1-3GB into the computer memory.

 

With JMP 17(?) compact columns were introduced. This allows to store character columns (with repeating entries) much more efficient in the computer memory. But there could be a bottleneck to get the data from a csv file into the compact column.

 

If you want to load numeric data in a "grid", please have a look at hdf5.
JMP can import such files, but the functionality is restricted. Better load the data via Python.

hogi
Level XII

Re: Fast text file format for importing to JMP

If there is time/date data in the input file, there are some restrictions: 
CSV import force MDY or DMY date format