cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
The Discovery Summit 2025 Call for Content is open! Submit an abstract today to present at our premier analytics conference.
Choose Language Hide Translation Bar
View Original Published Thread

How to exclude headers and footers while importing multiple txt files via Import Multiple Files Wizard?

Neo
Neo
Level VI

I am trying to import multiple *.txt data files from folders and subfolders. All data files have headers and footers (both starting with !). 

Headers are from line 1 to line 21. Line 22 has the test parameter names while test data starts from line 23. When data ends, the footer information starts (with !). All data is tab separated.

 

1. I am unable to get data imported with parameter names as column names using settings as per numbers, even when I try for a single data file. The header line is always showing up as column name and parameters in the first data row.

2. I do not expect to be able to remove the footer information with my settings, but I need to. How to achieve this?

 

Al the end, I need to vertically concatenate the data with an additional file-name column and correct column (parameter) names. Where am I going wrong?

If this is not possible via the multiple file import wizard, what are my options via JSL?

When it's too good to be true, it's neither
4 REPLIES 4
jthi
Super User


Re: How to exclude headers and footers while importing multiple txt files via Import Multiple Files Wizard?

Without seeing your files: consider doing this in few separate multiple file imports and then combine the resulting tables. Or use JSL and you are not limited by Multiple File Imports options

-Jarmo
Neo
Neo
Level VI


Re: How to exclude headers and footers while importing multiple txt files via Import Multiple Files Wizard?

@jthi Thanks. Here

https://community.jmp.com/t5/Discussions/How-to-exclude-header-and-footer-when-using-open-to-import-...

I had given an example earlier of how my data file looks like only now I have hundreds of them in folders and subfolder within.

Also, they are *.txt as mentioned in my OP (and not a JMP data table), hence I went the Import Multiple Files way.

Ideally, I would like to use the Import Multiple Files wizard, but perhaps invoke it via a JSL to take into account my requirements. Could I get some direction please?

When it's too good to be true, it's neither
jthi
Super User


Re: How to exclude headers and footers while importing multiple txt files via Import Multiple Files Wizard?

Are header and footer data useless information?

-Jarmo
jthi
Super User


Re: How to exclude headers and footers while importing multiple txt files via Import Multiple Files Wizard?

As you have not provided example file, I assume that the "file" I copy pasted is what you have

jthi_0-1740074772508.png

You should be able to ignore headers by just utilizing settings, for footers, I think you will have to remove them afterwards

Names Default To Here(1);

dt = Multiple File Import(
	<<Set Folder("$DOWNLOADS\843237\"),
	<<Set Add File Name Column(1),
	<<Set Import Mode("CSVData"),
	<<Set Charset("Best Guess"),
	<<Set Stack Mode("Stack Similar"),
	<<Set CSV Has Headers(1),
	<<Set CSV Allow Numeric(1),
	<<Set CSV First Header Line(3),
	<<Set CSV Number Of Header Lines(1),
	<<Set CSV First Data Line(4),
	<<Set CSV EOF Comma(0),
	<<Set CSV EOF Tab(0),
	<<Set CSV EOF Space(0),
	<<Set CSV EOF Spaces(1),
	<<Set CSV EOF Other(""),
	<<Set CSV EOL CRLF(1),
	<<Set CSV EOL CR(1),
	<<Set CSV EOL LF(1),
	<<Set CSV EOL Semicolon(0),
	<<Set CSV EOL Other(""),
	<<Set CSV Quote("\!""),
	<<Set CSV Escape(""),

) << Import Data;

dt = dt[1];

rows_to_delete = dt << Get Rows Where(Starts With(:Parameter, "! "));
dt << Delete Rows(rows_to_delete);

jthi_1-1740074845363.png

 

-Jarmo