topic Re: Monitoring log file updates from console in Discussions

Monitoring log file updates from console

SteveTerry — Sat, 18 Nov 2023 02:11:52 GMT

With JMP 17.2.0, I'm trying to monitor the output of a JSL log file from a console — (Mac OS Terminal, in this case). I'm using the tail command with the -f option to live-view the updates whenever the file is written.

$ tail -f log.txt

However, the console output never updates when new text is written to the log. The sample JSL below outputs a line of text every second:

Open Log();
Clear Log();
For( ii=0, ii<10, ii++,
	Write( "hello, world " || Char( ii ) || "\!n" );
	Save Log( "log.txt" );
	Wait( 1 );
);

When I start the script, then run the tail command a few seconds later, I get an expected result:

$ tail -f log.txt
/*:hello, world 0
hello, world 1
hello, world 2
hello, world 3

But subsequent JSL log messages are not picked up by the tail command. If I Ctrl-C out, then re-run tail, I properly get everything output to the log file at that point. But the output never again updates.

Any thoughts as to why this doesn't work? Or is there maybe another way to manage a log file where this would work as expected?

Re: Monitoring log file updates from console

jthi — Sat, 18 Nov 2023 06:12:25 GMT

Using "powershell version of tail" seems to work fine (this won't run unless the file exists first)

Get-Content -Path "<logfilepath>" -Wait

https://stackoverflow.com/questions/4426442/unix-tail-equivalent-command-in-windows-powershell

I did write the log with

Names Default To Here(1);

For(i = 1, i <= 30, i++,
	Show(i);
	wait(1);
	Save Log("$TEMP/jmplog/log.txt");
);

You could manually check that the log file is being updated as you think it is and if that is the case, then start figuring out what is wrong with tail. Could save log be for example recreating the file every time and tail drops the live tracking because of that?

Usually when I write logs from JMP I use Save Text File() . First create the file with headers (if I have those) and then use mode("append") to add new lines. Only if I wanted to get the whole log content I did use Save Log() (for example saving log when code had errors). Get Log() can also be useful sometimes.

Edit: Maybe changing -f to -F could help? https://man7.org/linux/man-pages/man1/tail.1.html

       -f, --follow[={name|descriptor}]
              output appended data as the file grows;
       -F     same as --follow=name --retry
       --retry
              keep trying to open a file if it is inaccessible

Re: Monitoring log file updates from console

Craige_Hales — Sat, 18 Nov 2023 12:28:51 GMT

@SteveTerry If it is because JMP is replacing the file and tail can't latch on to the new file (seems likely) then (pseudo code, approximately) something like this:

xxx = get log text();
clear log(); // so each chunk is appended only once
savetextfile("log.txt", xxx, mode("append") );

alternatively, you might use watch, something like

watch tail log.txt

to rerun the tail command every few seconds. Easier, but not as efficient.

Re: Monitoring log file updates from console

SteveTerry — Sat, 18 Nov 2023 18:44:52 GMT

Thanks @jthi, the -F option is what got it to work. Not sure why I didn't pick up on that in the man page — probably because I've been using -f for decades and wasn't keen on thinking outside the box. Fyi, the --retry option doesn't exist on Mac; it kinda looks like you're using JMP in a Linux environment?? I'd like to know more about that.

I use Save Text File() iteratively quite a bit for building JSL scripts dynamically, but I don't think that's appropriate here since it doesn't capture stuff sent to the log, e.g., Write() and Show(), so it would be double work.

@Craige_Hales, thanks for your suggestions. Fyi, watch doesn't appear on Mac (for base installs anyway — perhaps I could manually install it), though I see in on Ubuntu. Oh my, are you somehow combining JMP with Linux also?? ;–)

As you both indicate, it does appear that Save Log() is re-writing the file in a manner that gives it a new inode, which tail -f loses. This seems very inefficient. I can imagine how this might be useful in some circumstances where lots of log buffer manipulation is going on such as periodic clearing. But I think, for most applications, It would be far preferable if there existed a buffer flush mechanism of the log to a pre-assigned file. My code actually issues Save Log() commands very frequently in order to capture recent activity in case of a runtime fault (so I can identify what it was last doing before the fault). It sounds now like that's creating a lot of disk thrashing. Is this possibly a new feature I may want to suggest?

Re: Monitoring log file updates from console

Craige_Hales — Sat, 18 Nov 2023 19:04:52 GMT

I run JMP on windows running in a VirtualBox under Linux. for example.

I think the wish you might want to add is for JMP to also be able to write a disk file log, in parallel with the log window, possibly flushing the output every line.

Re: Monitoring log file updates from console

jthi — Mon, 20 Nov 2023 05:32:21 GMT

I'm running JMP on windows. I just googled tail and checked that man page (I have used it with cygwin to monitor logs though).

Re: Monitoring log file updates from console

SteveTerry — Mon, 20 Nov 2023 19:21:15 GMT

Added this feature to Wish List in case anyone would like to comment on or up-vote it.

https://community.jmp.com/t5/JMP-Wish-List/Provide-option-to-write-log-file-to-disk-in-parallel-with-the/idi-p/701103