Blog post: FileSnapper    Windows only.   FileSnapper makes periodic snapshots of JSL editor windows. If JMP does not have a chance to save your JSL (crash, power failure, cat on keyboard), the snapshot might help you recover. The escape key can sometimes stop FileSnapper. Because of the way FileSnapper works, there will not be a warning and the snapshots will not be made. You should still make your own backups, etc. The snapshots are deleted after 14 days by default. FileSnapper dialog ... View more

@mlrpadilla asked about recovering files when JMP does not get a chance to save them. How do I recover most recent work on JMP files in cases of unavoidable computer/JMP shutdown?  I had a similar experience a year ago, with .JSL scripts, when a power failure cost me several hours of work. I wrote an add-in to periodically look for open JSL files and make snapshots of them. It does not snapshot JMP data tables. And it only works on Windows. But it has saved my JSL for me a couple of times since. It has also failed me at least once (below.) FileSnapper Add-in After installing, the FileSnapper add-in script runs every five minutes (default) and looks at the open windows to find script windows that have been modified. When it finds one, it grabs the text and saves it away.  If JMP does not get a chance to close normally, a sentinel file is left behind. The next time JMP launches and finds the sentinel, JMP will open the FileSnapper dialog to let you see if there is a snapshot you want to restore. You can also open the dialog from the add-ins menu and set a few properties (like the 5-minute interval and the 14-day retention period.) How it works To keep FileSnapper as invisible as possible, it does not use the JSL scheduler. (The scheduler opens a window, and there is only one scheduler that you might want to use for something else.) Instead, FileSnapper uses RunProgram to get a periodic wake-up call. There were a number of iterations; the current version is using CMD.EXE and the DOS waitfor command in a loop to send some text back to FileSnapper every five minutes. When JMP shuts down, it signals the waitfor event to make CMD.EXE exit quickly. CMD.EXE will also terminate if the sentinel disappears. A tiny JMP data table is used for the sentinel file. The table is saved, but also kept open as a private table. If a second copy of JMP runs, it detects the table is open in another copy of JMP and (1) does not open the recovery dialog and (2) makes a different sentinel file. If any non-open sentinel is found, all the non-open sentinels are removed and the recovery dialog is opened. You should only get prompted once for recovering files. The add-in uses the shutdown hook to remove the sentinel and do several things to make sure RunProgram terminates. There may be a few diagnostic show() calls left behind, but you can't normally see their output because JMP is shutting down. You can use the view->add-ins to enable/disable FileSnapper and watch them in the log window. The backup() function is called from RunProgram every five minutes. Backup() looks for editor windows that are dirty (modified), compares their content to the last time, and if changed saves a copy. A special folder is created under Documents for the snapshots. The files are marked read only to help you not try to use them because... Every time FileSnapper starts up, it studies the old snapshot files. FileSnapper quietly deletes old snapshots. You can control how noisy the log is, but you don't want to make FileSnapper log every snapshot it makes because you may lose a keystroke or two every five minutes, and more log activity just makes it worse. The dialog window (after a failure, or from the menu) shows a list of most-recent to oldest snapshots. You can (1) double click to open, or (2) single click to select. If you select one file you can right-click the selected file (right-click does NOT select, bummer!) and compare to an existing file. if you select two files, you can right-click and compare the two. Check the title of the compare window to quickly see if the files are the same. Compare to an existing file does not prompt; it looks in the registry and some identified directories for a file of the right name. It may or may not be the file you want to compare to. But I still lost some work. When you are developing a script, you'll often press escape to stop the script. FileSnapper is a script. There appears to be no way to prevent it from occasionally processing the escape key and stopping at just the wrong time, never to run again five minutes later. There is a half-baked idea in FileSnapper to make a report that it stopped, but only if a data table is opened after five minutes later. In my case, I don't use data tables that often when I'm writing scripts, so I might not get the notice. At one point I set the retention to 100 days and the snapshot frequency to 30 seconds. This is too much. There were over 10,000 files and JMP startup could take 30 seconds. Choose your pain level in the dialog's options. Press the save button, then close the dialog. I'd wish for this to be built in to JMP, and I'd wish for better callback timers. There are a lot of improvements that could be made (like the save button in the dialog does not give any feed back). It might or might not actually help someone. If you get tired of the occasional missed keystroke, you can uninstall it. The same view->add-ins menu will also let you look at the source. Thanks to @Justin_Chilton for pointing out how the JSL windows in a project work.   Add-in.   ... View more

I've been re-purposing old cell phones as clocks for several years. Android 2.2.2 - Froyo. This one still going, its sibling has failed. And more recent phones have failed. Two have just failed. That's the last one running...good hardware. Really old OS, but still gets the time off the network and still updates when we switch to daylight time and back. So I need to replace them, and I started thinking about Arduinos and Raspberries and stumbled into ESP32. Quarter inch squares, this is about 25cm x 50cm This is from espressif and I got this one on amazon, 3 for $17. So far I've played with blinking an LED and using the built-in WIFI hardware. Yes, WIFI, and blue tooth, and lots of I/O pins, 4M rom and 512K ram (I think. have not explored far enough to know for sure.) In the process, I ran the NTP (Network Time Protocol) example and started wondering how NTP works. At the bottom, NTP is really simple. A computer that wants to know the time sends a short message to another computer that hosts an NTP server, and that computer sends a short message back that includes the GMT time. Done. (It is much more complicated; apparently the NTP servers may answer the question 1000 times a second! And the mechanisms for synchronizing the servers use statistics. And David L Mills invented the NTP stuff around 1980.) The internet mostly uses TCP  (Transmission Control Protocol) , a robust protocol built on top of IP. TCP retries, etc, to make sure data is delivered, in order, and intact. It is a heavy-weight protocol. NTP servers use a light weight protocol, UDP (also on top of IP) which guarantees nothing. Your messages might get lost, never arrive, arrive out of order. But the load on the server is minimal; the server doesn't need to do any extra hand-shaking to make sure you get an answer. The U in UDP  (User Datagram Protocol) means the user is responsible for handling errors. This JSL creates a socket to use port 123 (the NTP port) and the UDP protocol to send a message to a pool of NTP servers and get the response. It does some really simple statistics math that assumes the travel time to and from the server are equal so the server's time can be synced with the local computer better. The results are pretty good! Here's a slightly better than average example that found a server in the pool half-way around the world:   pool.ntp.org(rn5.quickhost.hk) TotalTime = 0.143846988677979 FlightTime = 0.143799781799316 Difference = 0:00:00.01689   Assuming .hk is really in Hong Kong, that's 8,000 miles from North Carolina. 16,000 round trip. The Flight Time is the Total Time less the processing time in Hong Kong, which was negligible. If I did my math right, that's more than 100,000 miles/second, a significant fraction of the speed of light. The last number, difference, is the difference between my CPU clock and the NTP server's clock. That's less than 1/50 second. And I get results from most servers within 1/10 second, unless the flight time is large, in which case it is asymmetrical and the correction technique breaks down.   pool.ntp.org(LAX.CALTICK.NET) TotalTime = 0.101963043212891 FlightTime = 0.101903915405273 Difference = 0:00:00.08646   pool.ntp.org(ns2.nuso.cloud) TotalTime = 0.103617668151855 FlightTime = 0.103596687316895 Difference = 0:00:00.04034   pool.ntp.org(50-205-57-38-static.hfc.comcastbusiness.net) TotalTime = 0.0763883590698242 FlightTime = 0.0763206481933594 Difference = 0:00:00.00965   pool.ntp.org(13.86.101.172) TotalTime = 0.077089786529541 FlightTime = 0.07708740234375 Difference = 0:00:00.00663   // this snippet demonstrates a UDP (User Datagram Protocol) exchange with an NTP // (Network Time Protocol) server using a JSL socket. NTP sends enough information // from an NTP server to correct for some network delays and make a really good guess // at the local time, possibly within 1/100 second, usually within 1/10 second (depending // on how similar send and receive times are.) Using the OS time setting to sync your // clock first (which uses NTP as well) will make the deltas near zero. Otherwise the // deltas should be similar across several runs. Your computer clock will drift between // the syncs (once a day?) that your computer automatically performs with an NTP server. // // idea from https://lettier.github.io/posts/2016-04-26-lets-make-a-ntp-client-in-c.html timezone = -4; // (tz for my computer) Eastern Daylight Time is GMT-4. // build the transmit packet to send to the NTP server xPacket = Hex To Blob( "1b" || Repeat( "00", 47 ) ); // make the socket object sock = Socket( DGRAM ); // not a normal STREAM socket, this is an unreliable UDP socket. // choose a server (or server pool) addr = "pool.ntp.org"; // a pool of servers. Don't ask too often, they have a heavy load. // if nothing goes wrong, the bind and ioctl are optional, but // if we never get a response (for various reasons) we'd rather not hang forever... rc = sock << bind( addr, "123" ); // bind, maybe not required, but needed for NBIO... If( rc[2] != "ok", sock << close; Throw( "bad bind" ); ); rc = sock << Ioctl( FIONBIO, 1 ); // NBIO: non blocking i/o. the unreliable UDP may never answer. If( rc[2] != "ok", sock << close; Throw( "bad ioctl" ); ); // send our request to the server. <<sendto is for unreliable UDP packets. // it is possible the server may not get the packet. sendHPtime = HP Time(); // remember, in high precision, when the transaction begins rc = sock << sendto( addr, "123", xPacket ); If( rc[2] != "ok", sock << close; Throw( "bad send" ); ); // wait for a response from the server. If the server got the request, // it might not send a response if we've asked too many times recently. // And if it does send a response, the response could get lost. // <<recvfrom is for unreliable UDP packets. timeOut = Tick Seconds() + 5; // timeout in 5 seconds While( Tick Seconds() < timeOut, Wait( 0 );// allow processing in background rc = sock << recvfrom( 48 );// the return packet is the same 48 byte structure recvHPtime = HP Time(); // remember, in high precision, when the transaction might have ended If( !Starts With( rc[2], "WOULDBLOCK" ), Break() // the transaction ends when we actually get data back ); ); // see if we got good data. Possibly not all the data arrived, but I have not seen that yet. If( rc[2] != "ok" | Length( rc[3] ) != 48, sock << close; Throw( Char( rc ) || " time out" ); ); sock << close; // UDP, like TCP, is built on top of IP, but UDP is connectionless. Since there is // no fixed connection, the return packet tells who it came from, and that might be // any name in the pool of servers. (see addr at top) serverName = rc[4]; // [5] is the port, probably always "123" // the received data should be a 48 byte packet rPacket = rc[3]; // https://github.com/lettier/ntpclient/blob/master/source/c/main.c // added some annotations... ////uint8_t li_vn_mode; // Eight bits. li, vn, and mode. //// // li. Two bits. Leap indicator. //// // vn. Three bits. Version number of the protocol. //// // mode. Three bits. Client will pick mode 3 for client. //// //// uint8_t stratum; // Eight bits. Stratum level of the local clock. //// uint8_t poll; // Eight bits. Maximum interval between successive messages. //// uint8_t precision; // Eight bits. Precision of the local clock. //// //// uint32_t rootDelay; // 32 bits. Total round trip delay time. //// uint32_t rootDispersion; // 32 bits. Max error aloud from primary clock source. //// uint32_t refId; // 32 bits. Reference clock identifier. //// //// uint32_t refTm_s; // 32 bits. Reference time-stamp seconds. (when the server clock was last set, ignore this) //// uint32_t refTm_f; // 32 bits. Reference time-stamp fraction of a second. //// //// uint32_t origTm_s; // 32 bits. Originate time-stamp seconds. (filled with zeros above, use sendHPtime) //// uint32_t origTm_f; // 32 bits. Originate time-stamp fraction of a second. //// //// uint32_t rxTm_s; // 32 bits. Received time-stamp seconds. (when server got xPacket) //// uint32_t rxTm_f; // 32 bits. Received time-stamp fraction of a second. //// //// uint32_t txTm_s; // 32 bits Transmit time-stamp seconds. (when server returned rPacket) //// uint32_t txTm_f; // 32 bits. Transmit time-stamp fraction of a second. //// //// and finally, recvHPtime is when we got the rPacket back https://tools.ietf.org/html/rfc5905 //// //// } ntp_packet; // Total: 384 bits or 48 bytes. // unpack... https://stackoverflow.com/questions/29112071/how-to-convert-ntp-time-to-unix-epoch-time-in-c-language-linux // convert the 48-byte blob into twelve 4-byte integer values, then // grab the whole and fraction values of the receive and transmit times. rIntMatrix = Blob To Matrix( rPacket, "uint", 4, "big" ); rxTm_s = rIntMatrix[9]; // the server's receive time (seconds) rxTm_f = rIntMatrix[10]; // the server's receive time (fraction) txTm_s = rIntMatrix[11]; // ditto, server transmit time txTm_f = rIntMatrix[12]; // join the whole seconds to the fractional seconds and convert to // JMP's date time epoch (1904 for JMP. 1900 for NTP. 1970 for linux. 1601 for windows with 10^-7 scale.) // "rx" and "tx" Tm are on the server computer rxTm = As Date( rxTm_s + rxTm_f / (2 ^ 32) + 1jan1900 ); txTm = As Date( txTm_s + txTm_f / (2 ^ 32) + 1jan1900 ); // spin for a second to sync hptime to actual time // hptime is in microseconds since JMP started. Find the // hptime that matches a change in the current second. oldTd = Today(); // truncated to seconds While( (nowHp = HP Time() ; newTd = Today()) == oldTd, 0 ); // newTd and nowHp are the same point in time with different units. // addToHpSeconds is in seconds (not microseconds) and can be added // to (hp/1e6) to get local time with fractional seconds addToHpSeconds = newTd - nowHp / 1e6; // get our original send and receive times with fraction seconds. // "orig" and "dest" Tm are on the local computer origTm = sendHPtime / 1e6 + addToHpSeconds; destTm = recvHPtime / 1e6 + addToHpSeconds; // "flightTime" is the time the packet (x and r) was "in the air", which does not include the // time the packet was being processed by the server. flightTime = (destTm - origTm) - (txTm - rxTm);// dest-orig is total time we waited, tx-rx is server processing // assume the outbound and inbound times are the same. Half of the flight time // is how long it has been since the server sent the current time to us. returnTime = flightTime / 2; // this is our estimate of the return trip duration localTimeWhenServerSent = destTm - returnTime; // when we received, minus estimate // finally, see how much the local clock and server clock differ. txTm is the // server's transmit time in GMT. Add our (-4) timezone correction to get EDT local time. // subtract our EDT local time estimate of when the server sent the data. lagtime = (txTm + In Hours( timezone )) - (localTimeWhenServerSent); Write( "\!n", addr, "(", serverName, ")", "\!n TotalTime =\!t", destTm - origTm, "\!n FlightTime =\!t", flightTime, "\!n Difference =\!t", Format( lagtime, "hr:m:s", 15, 5 ) );       ... View more