topic Re: How to exclude header and footer when using open () to import a text file, in JSL? in Discussions

How to exclude header and footer when using open () to import a text file, in JSL?

Neo — Sat, 10 Jun 2023 23:41:36 GMT

I have data which has several header rows at the beginning of data and footer at the end of the data.

Both header and footer text start with an identifier !.

(number of rows of header, footer and data in-between vary)

Example with arbitrary data (available as .txt file)

! header first line

! header second line

Parameter LSL USL Units

voltage 0.1 0.5 V

Capacitance 0.3 0.7 pF

Resistance 0.5 9 Ohms

! Footer second line

! Footer third line

Can I use open () so that I can exclude header and footer data while importing data in text format?

I am able to use open() to import data keeping the header and footer text (say into a table dt).

I also want to know how to extract (to a new table dt_new) data excluding the header and footer which start with an identifier, say !.

Finally, I want to save the header and footer data in dt in a separate table, say dt_hf. How to do this?

Thanks.

Re: How to exclude header and footer when using open () to import a text file, in JSL?

Craige_Hales — Thu, 09 Dec 2021 23:06:36 GMT

One way:

filename = Save Text File(
	"$temp/deleteme.txt",
	"\[! header first line
! header second line
Parameter         LSL   USL     Units
voltage              0.1    0.5        V
Capacitance     0.3    0.7       pF
Resistance        0.5     9       Ohms
! Footer second line
! Footer third line
! Footer third line]\"
);

lines = Words( Load Text File( filename ), "\!n\!r" );

metadata = {};
data = {};
For( i = 1, i <= N Items( lines ), i += 1,
	line = lines[i];
	If( Starts With( line, "!" ),
		Insert Into( metadata, line ),
		Insert Into( data, line )
	);
);

datafilename = Save Text File( "$temp\data.txt", Concat Items( data, "\!n" ) );
metafilename = Save Text File( "$temp\metadata.txt", Concat Items( metadata, "\!n" ) );
dtData = Open( datafilename, Import Settings( End Of Field( Tab, Spaces, Space ) ) );
dtMetadata = Open( metafilename, Import Settings( Labels( 0 ) ) );

meta data separated from data

Re: How to exclude header and footer when using open () to import a text file, in JSL?

Neo — Mon, 13 Dec 2021 14:33:25 GMT

Thanks @Craige_Hales.

What changes are needed if my data is in a data table and not in a text file?

(I load the text file and call it say dt).

Now I need to separate the header and footer from dt.

Also, how to handle empty lines between header and footer entries, e.g. empty space between footer second line and footer third line and not include them in data?

Re: How to exclude header and footer when using open () to import a text file, in JSL?

txnelson — Mon, 13 Dec 2021 15:28:42 GMT

Is this what your data table looks like? If not, can you supply a sample data table?

Re: How to exclude header and footer when using open () to import a text file, in JSL?

Neo — Mon, 13 Dec 2021 16:46:40 GMT

@txnelson. Below is how it looks like. I would like to do the following

1. Separate the header and footer text into one list.

2. Separate the data between the header and foot except the empty line into a data table.

3. Count the number of empty entries in the data table (they change depending on which text file is loaded)

Thanks.

Re: How to exclude header and footer when using open () to import a text file, in JSL?

txnelson — Tue, 14 Dec 2021 05:06:45 GMT

Here is a simple example of what I think you need.

Names Default To Here( 1 );
dt = Current Data Table();

headFootList = {};
col1Name = Column( 1 ) << get name;
Eval(
	Parse(
		Eval Insert(
			"For Each Row( If( Contains( :^col1Name^, \!"!\!" ) == 1, Insert Into( headFootList, :^col1Name^ ) ) )"
		)
	)
);

Eval(
	Parse(
		Eval Insert(
			"emptyCount = N Rows( dt << get rows where( :^col1Name^ == \!"\!" ) )"
		)
	)
);
Eval(
	Parse(
		Eval Insert(
			"dt << select where( contains(:^col1Name^,\!"!\!")==1 | :^col1Name^ == \!"\!")"
		)
	)
);
Try( dt << delete rows );

dtHeadFoot = New Table( "Headers and Footers",
	New Column( "Headers and Footers", character, set values( headFootList ) )
);

New Window( "Count of Empty Rows",
	<<modal,
	Spacer Box( size( 0, 15 ) ),
	H List Box( Text Box( "The numer of empty rows is: " || Char( emptyCount ) ) )
);