How to minimize the size of an integer table in numeric format?

[#1]

The tables need to save are all integer numbers.

Thanks!

dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
dt << delete columns( 1 :: 3 );
New Column( "A" );
Column( "A" ) << Formula( 1000 + Row() );
dt << run formulas;
Column( "A" ) << deleteFormula;

[#2]

Minimize in memory, or on disk?

In memory: use a short numeric format, numeric(1), numeric(2), or numeric(4).  These specify a signed integer of the same number of bytes with some reserved values for missing. They are all smaller than a double (8 bytes) which can represent integers up to ~50 bits...a much larger range than the 8/16/32 bit range. Also saves space on disk. (I don't think it is required, but a recent comment in the community said the pref for short numerics has to be on to use them. After turn on pref, look at the available numeric types when creating columns and look at the JSL that gets saved. The ranges are roughly +/-126, +/-32000, +/-2e9.)

On disk: use compression. Two approaches, either use the red triangle to do a zip-like compression of all columns, or use the compress-columns utility that looks to see if the column has a limited number of values that could be represented as a lookup. The zip approach is slow when saving, faster when decompressing, but not as fast as loading the uncompressed data.

You can use a numeric(1 or 2 or 4) and zip the file to good effect if there is still some redundancy in the data that zip can squeeze out. But it is unzipped in memory so doesn't help there.

[#3]

Thank Craige!

I need to reduce the size of the file stored on the hard disk.But I don't know how to do that with JSL.

[#4 · Accepted solution]

I made a file with the GUI, set the column to numeric(1) and used the red triangle to request compression on disk. Then I grabbed the script.

New Table( "Untitled",
    Add Rows( 0 ),
    Compress File When Saved( 1 ),
    New Column( "Column 1", Numeric( 1 ), "Continuous", Set Selected )
)

[#5]

Thanks Experts!

[#6 · Accepted solution]

Hello,

You can set this preference and after by default all tables will be compressed.

or by scripting as mentioned by @Craige_Hales 

Compress File When Saved( 1 );

You also could compress Column by using 

Compress Selected Columns( );

[#7]

• It's amazing how much the file size is reduced.

•