topic Re: pulling information with date condition: put filename into list if it is greater than a specified date and time in Discussions

pulling information with date condition: put filename into list if it is greater than a specified date and time

dadawasozo — Fri, 09 Jun 2023 16:05:32 GMT

Hi,

I m trying to get the list of files name in a directory with given condition if it is 1) zip archive and also the 2) creation date greater than yyyy/mm/dd. Below is my script. My question

1) it ran without error but not return empty list. I 'm not sure what is the issue (perhaps the date).
2) how can I set the creation date () output format to yyyy/mm/dd hh:mm:ss?

3) it take long time to loop through the path2 because I have over 100K zip folders. Is there other faster way to do it?

Thanks

path2 = "C:\Users\test\;
filelist = files in directory (path2);
lst2 = {};
For(i=1, i<= N items(filelist), i++,
If( Contains(filelist[i], ".zip") & Creation Date( path2||filelist[i]) > 2022-10-12T08:00:00, Insert Into( lst2, filelist[i] )
),
);

Re: pulling information with date condition: put filename into list if it is greater than a specified date and time

txnelson — Sat, 24 Dec 2022 04:31:50 GMT

In the example you provided, there is a missing double quote on the path2 = line, and also, the time reference does not seem to be handling by JMP, so by changing it to an Informat() function, one can be insured that the JMP Date/Time reference is being interpreted correctly.

names default to here(1);
path2 = "C:\Users\test\";
filelist = Files In Directory( path2 );
lst2 = {};
For( i = 1, i <= N Items( filelist ), i++,
	If( Contains( filelist[i], ".zip" ) & Creation Date( path2 || filelist[i] ) > informat("2022-10-12T08:00:00", "yyyy-mm-ddThh:mm:ss"),
		Insert Into( lst2, filelist[i] )
	)
);

Re: pulling information with date condition: put filename into list if it is greater than a specified date and time

dadawasozo — Sat, 24 Dec 2022 21:53:11 GMT

Hi Jim, Thanks. that fixed the problem. However, it take too much time when a folder has too many files to go through. Below are the new steps I have so next time when I just want to pull the latest files, it don't take the long time like using the date/time as filter (above). I 'm not sure if it is an effective way to do it. Is there any better way to just extract latest files with shorter time?

1) the first time get the whole filelist in the defined folder (Files In Directory(dir)). Saved it into a column of a table, "T1" (only run one time)

2) after sometime, latest files added to the same folder. script will go to the same folder and take the whole filelist as "newlist" (newlist = Files In Directory(dir)).

3) get column in table 1) as list, then loop through to remove the elements out from newlist (I use Remove From ()) .

4) do whatever needed on newlist.

5) append newlist into table in 1).

6) repeat step 2)-5) next time when I need to get latest files again.

Any suggestion?

Re: pulling information with date condition: put filename into list if it is greater than a specified date and time

txnelson — Sat, 24 Dec 2022 22:10:11 GMT

You could try this....it should speed it up

Names Default To Here( 1 );
path2 = "C:\Users\test\";
filelist = Files In Directory( path2 );
For( i = N Items( filelist ), i >= 1, i--,
	If( Right( filelist[i], 4 ) != ".zip",
		Remove From( filelist, i, 1 )
	)
);
lst2 = {};
For( i = 1, i <= N Items( filelist ), i++,
	If( Creation Date( path2 || filelist[i] ) > informat("2022-10-12T08:00:00", "yyyy-mm-ddThh:mm:ss"),
		Insert Into( lst2, filelist[i] )
	)
);