topic Re: How can JSL modify data in a table directly to a specified binary file? in Discussions

How can JSL modify data in a table directly to a specified binary file?

lwx228 — Fri, 09 Jun 2023 23:41:57 GMT

"Text.blk" is a binary file saved in C:\, how JSL can write, rewrite, and replace the data in the data table.
The following code is the VBA code of Excel, which can write the data of excel column A to C :\ text.blk.

Sub text()
    Dim arr, I, filename As String, t As Long
   filename = "c:\text.BLK"
    Open filename For Binary As #1: arr = Range("a2:a" & Cells(Rows.Count, "a").End(3).Row)
    For r = 1 To UBound(arr)
        If Left(arr(r, 1), 2) = "12" Then arr(r, 1) = 1000000 + arr(r, 1)
    Next
    For I = 1 To UBound(arr, 1)
        If IsNumeric(arr(I, 1)) Then Put #1, , CLng(arr(I, 1))
    Next
    Close #1
End Sub

Re: How can JSL modify data in a table directly to a specified binary file?

lwx228 — Thu, 29 Oct 2020 12:20:13 GMT

text.BLK

code
123029
128108
113555
113038

Re: How can JSL modify data in a table directly to a specified binary file?

Craige_Hales — Thu, 29 Oct 2020 14:34:31 GMT

Use loadTextFile with the blob option (it ignores the file type). Then, depending what you want to do, one of these:

convert the blob to hex, manipulate the hex (twice as many characters as binary bytes), convert the hex back to blob
use blobPeek() to segment the blob, then reassemble the blob with your data injected
use blobToMatrix to extract the data and matrixToBlob to reassemble the blob. This is perhaps the most powerful, fastest, and either easiest or hardest to use, depending on the file. The blobToMatrix function will decompose the entire blob (which might be from 2, above) into 1, 2, 4, or 8 byte chunks in a matrix; the chunks can be big- or little- endian integers or floats. (If the data was a stereo wav file, you could make a 2-column matrix of 16-bit integer samples, for example.)

Finally, save the blob with saveTextFile. It does not need a blob option because you either give it a blob or a text string to save.

Re: How can JSL modify data in a table directly to a specified binary file?

lwx228 — Fri, 30 Oct 2020 00:07:58 GMT

Thank Craige!

Can these operations be done directly using JSL?
I was wondering if you could write the first method as JSL.

Thanks Experts!

Re: How can JSL modify data in a table directly to a specified binary file?

lwx228 — Fri, 30 Oct 2020 00:14:44 GMT

I tried to write it, but the key points remained unclear.

Thanks!

Re: How can JSL modify data in a table directly to a specified binary file?

Craige_Hales — Fri, 30 Oct 2020 03:19:32 GMT

b = loadTextFile("z:/text.blk",blob()); // load your file

hex = hex(b); // operate on hex data
show(hex); // the binary data is now character data
last4hex = right(hex,8); // two hex characters per byte

last4blob = blobpeek(b,length(b)-4,4); // operate on blob data
show(hex(last4blob)); // same data as above

mat = blobtomatrix(b,"int",4,"big"); // operate on matrix data
show(mat[nrows(mat)],hextonumber(last4hex),blobtomatrix(last4blob,"int",4,"big")); // all the same

// change the hex data (1-based offset to substr function, doubled. this is why programmers count from 0.)
show(substr(hex,5,4)); // the 3rd and 4th bytes, "1100". lets change them to "FE23" 
hex = substr(hex,1,4) || "FE23" || substr(hex,9);
show(hex);
f1=savetextfile("$temp/text1.blk", hextoblob(hex)); // convert hex to blob before saving

// do the same with a blob (0-based offset to blobpeek function)
f2=savetextfile("$temp/text2.blk",blobpeek(b,0,2)||hextoblob("FE23")||blobpeek(b,4));

// do the same with a matrix. the matrix above is 4-bytes/element, lets use 1 instead
mat = blobtomatrix(b,"int",1,"big");
mat[3]=hextonumber("FE"); // 1-based matrix index
mat[4]=hextonumber("23");
f3=savetextfile("$temp/text3.blk",matrixtoblob(mat,"int",1,"big"));

// check our work:
b1=loadtextfile(f1,blob());
b2=loadtextfile(f2,blob());
b3=loadtextfile(f3,blob());

if(b1==b2==b3, write("\!nmatch, as expected"), write("\!nERROR"));

// this shows the changed data 
show(shortesteditscript(hex(b),hex(b1)));

Re: How can JSL modify data in a table directly to a specified binary file?

lwx228 — Fri, 30 Oct 2020 15:05:21 GMT

Thank Craige!
I have learned your JSL code, I have no cleavage.

I was so careless that I uploaded the "text.blk" file by mistake.I'm sorry.

The original upload was not corresponding to the four data in the table.

My requirement is to output the 4 data points of the table as a new file as *.blk file.The original BLK file was deleted before the output.
I re-uploaded the specific VBA Excel file, opened it and clicked the "BLK" button, which would generate a "1t. BLK" file in the path of "C:\".

The request how to write JSL code can achieve this function of VBA: the four data in the table is directly generated and VBA generated BLK file exactly the same.

Thanks!

Re: How can JSL modify data in a table directly to a specified binary file?

Craige_Hales — Fri, 30 Oct 2020 16:56:56 GMT

I don't know anything about BLK files and very little about VBA, but it *might* be the case that a BLK file is really simple:

b = loadTextFile("z:/text.blk",blob()); // load your file

mat = blobtomatrix(b,"int",4,"little"); // operate on matrix data

/*:

[1123029, 1113555, 1110044, 1113038, 1113581, 1128108, 1110073, 1113509, 1113521,
1113037, 1113576, 1113586, 1113580, 1113035, 1113027, 1113592, 1113599, 1110059,
1113556, 1113565, 1113595, 1113543, 1113600, 1113520, 1113575, 1110072, 1113571,
1110067, 1113029, 1113582, 1113601, 1113011, 1110058, 1113034, 1113561, 1113593,
1113550, 1113008, 1110066, 1113548, 1113583, 1113039, 1113562, 1113537, 1113577,
1113527, 1113587, 1113541, 1113596, 1110053, 1113578, 1113526, 1113566, 1113591,
1113585, 1113589, 1110074, 1113572, 1110060, 1113590, 1113568, 1113021, 113555,
113038]

If you were expecting 64 integers in the earlier file, that's all it contains. I have no idea what order (row major? column major? other?) the values are in.

Re: How can JSL modify data in a table directly to a specified binary file?

lwx228 — Sat, 31 Oct 2020 00:07:53 GMT

Thank Craige!
I see. It's the wrong direction.

Instead of writing "1T.blk" data to the table, save the data in the table as "1T.blk".
I give "1T. BLK ", just to indicate the file form in which the data in the table will be saved.
I tried writing JSL:

But I still don't know how to save the data in the table in the form of the graph.The saved file suffix name is not the key.

dt = Current Data Table();
dt << New Column( "new", formula( If( Left( Char( :code ), 2 ) == "12", 1000000 + :code, :code ) ) );
dt << run formulas;
Column( "new" ) << deleteFormula;
Wait( 0 );
mx = dt << GetAsMatrix( {2} );

mx << Save( "C:\1T.BLK" );//The saved file suffix name is not the key.

Re: How can JSL modify data in a table directly to a specified binary file?

lwx228 — Sat, 31 Oct 2020 00:06:52 GMT

Please download the "1T.blk" file again,

see its structure, and how to save the 4 rows in the table in the form of "1T.blk".

Thank Craige!

Re: How can JSL modify data in a table directly to a specified binary file?

Craige_Hales — Sat, 31 Oct 2020 14:53:29 GMT

You'll still need to add 1,000,000 if that's what you need, but something like this

dt = Open( "$desktop/1T.jmp" );
filename = Save Text File(
	"$temp/deleteme.blk",
	Matrix To Blob( dt:code << getvalues, "int", 4, "little" )
);

Anyone following along:

the LoadTextFile and SaveTextFile functions normally work with character string text data and will possibly change the CRLF endings, add UNICODE Byte Order Mark, and can't deal with the ASCII NUL character. But, when used with JMP's BLOB type, the file is read/written with just the binary data.
BlobToMatrix and MatrixToBlob are inverse functions. They treat all the array elements or all the blob elements as 1, 2, 4, or 8 byte chunks. They also can make the array elements (which are always double precision 8 byte numbers) be signed or unsigned integers or floating point numbers in the blob. And they can handle big- vs little-endian representations in the blob. And BlobToMatrix has an extra argument to make multiple columns in the matrix.
You might want to use BlobPeek to deal with sections of the binary data that are not uniform chunk sizes, and the || concatenate operator to join blobs back together.

Re: How can JSL modify data in a table directly to a specified binary file?

lwx228 — Sat, 31 Oct 2020 23:49:43 GMT

Thank Craige!

That will do.It can be done directly with JSL in the future.

I tried: the specific data still needs +1000000.

The BLK file is read by other software.

Re: How can JSL modify data in a table directly to a specified binary file?

lwx228 — Sun, 01 Nov 2020 02:31:50 GMT

JMP is powerful