I've been re-purposing old cell phones as clocks for several years.
Two have just failed. That's the last one running...good hardware. Really old OS, but still gets the time off the network and still updates when we switch to daylight time and back.
So I need to replace them, and I started thinking about Arduinos and Raspberries and stumbled into ESP32.
This is from espressif and I got this one on amazon, 3 for $17. So far I've played with blinking an LED and using the built-in WIFI hardware. Yes, WIFI, and blue tooth, and lots of I/O pins, 4M rom and 512K ram (I think. have not explored far enough to know for sure.) In the process, I ran the NTP (Network Time Protocol) example and started wondering how NTP works.
At the bottom, NTP is really simple. A computer that wants to know the time sends a short message to another computer that hosts an NTP server, and that computer sends a short message back that includes the GMT time. Done.
(It is much more complicated; apparently the NTP servers may answer the question 1000 times a second! And the mechanisms for synchronizing the servers use statistics. And David L Mills invented the NTP stuff around 1980.)
The internet mostly uses TCP (Transmission Control Protocol), a robust protocol built on top of IP. TCP retries, etc, to make sure data is delivered, in order, and intact. It is a heavy-weight protocol. NTP servers use a light weight protocol, UDP (also on top of IP) which guarantees nothing. Your messages might get lost, never arrive, arrive out of order. But the load on the server is minimal; the server doesn't need to do any extra hand-shaking to make sure you get an answer. The U in UDP (User Datagram Protocol) means the user is responsible for handling errors.
This JSL creates a socket to use port 123 (the NTP port) and the UDP protocol to send a message to a pool of NTP servers and get the response. It does some really simple statistics math that assumes the travel time to and from the server are equal so the server's time can be synced with the local computer better. The results are pretty good! Here's a slightly better than average example that found a server in the pool half-way around the world:
pool.ntp.org(rn5.quickhost.hk)
TotalTime = 0.143846988677979
FlightTime = 0.143799781799316
Difference = 0:00:00.01689
Assuming .hk is really in Hong Kong, that's 8,000 miles from North Carolina. 16,000 round trip. The Flight Time is the Total Time less the processing time in Hong Kong, which was negligible. If I did my math right, that's more than 100,000 miles/second, a significant fraction of the speed of light.
The last number, difference, is the difference between my CPU clock and the NTP server's clock. That's less than 1/50 second. And I get results from most servers within 1/10 second, unless the flight time is large, in which case it is asymmetrical and the correction technique breaks down.
pool.ntp.org(LAX.CALTICK.NET)
TotalTime = 0.101963043212891
FlightTime = 0.101903915405273
Difference = 0:00:00.08646
pool.ntp.org(ns2.nuso.cloud)
TotalTime = 0.103617668151855
FlightTime = 0.103596687316895
Difference = 0:00:00.04034
pool.ntp.org(50-205-57-38-static.hfc.comcastbusiness.net)
TotalTime = 0.0763883590698242
FlightTime = 0.0763206481933594
Difference = 0:00:00.00965
pool.ntp.org(13.86.101.172)
TotalTime = 0.077089786529541
FlightTime = 0.07708740234375
Difference = 0:00:00.00663
// this snippet demonstrates a UDP (User Datagram Protocol) exchange with an NTP
// (Network Time Protocol) server using a JSL socket. NTP sends enough information
// from an NTP server to correct for some network delays and make a really good guess
// at the local time, possibly within 1/100 second, usually within 1/10 second (depending
// on how similar send and receive times are.) Using the OS time setting to sync your
// clock first (which uses NTP as well) will make the deltas near zero. Otherwise the
// deltas should be similar across several runs. Your computer clock will drift between
// the syncs (once a day?) that your computer automatically performs with an NTP server.
//
// idea from https://lettier.github.io/posts/2016-04-26-lets-make-a-ntp-client-in-c.html
timezone = -4; // (tz for my computer) Eastern Daylight Time is GMT-4.
// build the transmit packet to send to the NTP server
xPacket = Hex To Blob( "1b" || Repeat( "00", 47 ) );
// make the socket object
sock = Socket( DGRAM ); // not a normal STREAM socket, this is an unreliable UDP socket.
// choose a server (or server pool)
addr = "pool.ntp.org"; // a pool of servers. Don't ask too often, they have a heavy load.
// if nothing goes wrong, the bind and ioctl are optional, but
// if we never get a response (for various reasons) we'd rather not hang forever...
rc = sock << bind( addr, "123" ); // bind, maybe not required, but needed for NBIO...
If( rc[2] != "ok",
sock << close;
Throw( "bad bind" );
);
rc = sock << Ioctl( FIONBIO, 1 ); // NBIO: non blocking i/o. the unreliable UDP may never answer.
If( rc[2] != "ok",
sock << close;
Throw( "bad ioctl" );
);
// send our request to the server. <<sendto is for unreliable UDP packets.
// it is possible the server may not get the packet.
sendHPtime = HP Time(); // remember, in high precision, when the transaction begins
rc = sock << sendto( addr, "123", xPacket );
If( rc[2] != "ok",
sock << close;
Throw( "bad send" );
);
// wait for a response from the server. If the server got the request,
// it might not send a response if we've asked too many times recently.
// And if it does send a response, the response could get lost.
// <<recvfrom is for unreliable UDP packets.
timeOut = Tick Seconds() + 5; // timeout in 5 seconds
While( Tick Seconds() < timeOut,
Wait( 0 );// allow processing in background
rc = sock << recvfrom( 48 );// the return packet is the same 48 byte structure
recvHPtime = HP Time(); // remember, in high precision, when the transaction might have ended
If( !Starts With( rc[2], "WOULDBLOCK" ),
Break() // the transaction ends when we actually get data back
);
);
// see if we got good data. Possibly not all the data arrived, but I have not seen that yet.
If( rc[2] != "ok" | Length( rc[3] ) != 48,
sock << close;
Throw( Char( rc ) || " time out" );
);
sock << close;
// UDP, like TCP, is built on top of IP, but UDP is connectionless. Since there is
// no fixed connection, the return packet tells who it came from, and that might be
// any name in the pool of servers. (see addr at top)
serverName = rc[4]; // [5] is the port, probably always "123"
// the received data should be a 48 byte packet
rPacket = rc[3];
// https://github.com/lettier/ntpclient/blob/master/source/c/main.c
// added some annotations...
////uint8_t li_vn_mode; // Eight bits. li, vn, and mode.
//// // li. Two bits. Leap indicator.
//// // vn. Three bits. Version number of the protocol.
//// // mode. Three bits. Client will pick mode 3 for client.
////
//// uint8_t stratum; // Eight bits. Stratum level of the local clock.
//// uint8_t poll; // Eight bits. Maximum interval between successive messages.
//// uint8_t precision; // Eight bits. Precision of the local clock.
////
//// uint32_t rootDelay; // 32 bits. Total round trip delay time.
//// uint32_t rootDispersion; // 32 bits. Max error aloud from primary clock source.
//// uint32_t refId; // 32 bits. Reference clock identifier.
////
//// uint32_t refTm_s; // 32 bits. Reference time-stamp seconds. (when the server clock was last set, ignore this)
//// uint32_t refTm_f; // 32 bits. Reference time-stamp fraction of a second.
////
//// uint32_t origTm_s; // 32 bits. Originate time-stamp seconds. (filled with zeros above, use sendHPtime)
//// uint32_t origTm_f; // 32 bits. Originate time-stamp fraction of a second.
////
//// uint32_t rxTm_s; // 32 bits. Received time-stamp seconds. (when server got xPacket)
//// uint32_t rxTm_f; // 32 bits. Received time-stamp fraction of a second.
////
//// uint32_t txTm_s; // 32 bits Transmit time-stamp seconds. (when server returned rPacket)
//// uint32_t txTm_f; // 32 bits. Transmit time-stamp fraction of a second.
////
//// and finally, recvHPtime is when we got the rPacket back https://tools.ietf.org/html/rfc5905
////
//// } ntp_packet; // Total: 384 bits or 48 bytes.
// unpack... https://stackoverflow.com/questions/29112071/how-to-convert-ntp-time-to-unix-epoch-time-in-c-language-linux
// convert the 48-byte blob into twelve 4-byte integer values, then
// grab the whole and fraction values of the receive and transmit times.
rIntMatrix = Blob To Matrix( rPacket, "uint", 4, "big" );
rxTm_s = rIntMatrix[9]; // the server's receive time (seconds)
rxTm_f = rIntMatrix[10]; // the server's receive time (fraction)
txTm_s = rIntMatrix[11]; // ditto, server transmit time
txTm_f = rIntMatrix[12];
// join the whole seconds to the fractional seconds and convert to
// JMP's date time epoch (1904 for JMP. 1900 for NTP. 1970 for linux. 1601 for windows with 10^-7 scale.)
// "rx" and "tx" Tm are on the server computer
rxTm = As Date( rxTm_s + rxTm_f / (2 ^ 32) + 1jan1900 );
txTm = As Date( txTm_s + txTm_f / (2 ^ 32) + 1jan1900 );
// spin for a second to sync hptime to actual time
// hptime is in microseconds since JMP started. Find the
// hptime that matches a change in the current second.
oldTd = Today(); // truncated to seconds
While( (nowHp = HP Time() ; newTd = Today()) == oldTd, 0 );
// newTd and nowHp are the same point in time with different units.
// addToHpSeconds is in seconds (not microseconds) and can be added
// to (hp/1e6) to get local time with fractional seconds
addToHpSeconds = newTd - nowHp / 1e6;
// get our original send and receive times with fraction seconds.
// "orig" and "dest" Tm are on the local computer
origTm = sendHPtime / 1e6 + addToHpSeconds;
destTm = recvHPtime / 1e6 + addToHpSeconds;
// "flightTime" is the time the packet (x and r) was "in the air", which does not include the
// time the packet was being processed by the server.
flightTime = (destTm - origTm) - (txTm - rxTm);// dest-orig is total time we waited, tx-rx is server processing
// assume the outbound and inbound times are the same. Half of the flight time
// is how long it has been since the server sent the current time to us.
returnTime = flightTime / 2; // this is our estimate of the return trip duration
localTimeWhenServerSent = destTm - returnTime; // when we received, minus estimate
// finally, see how much the local clock and server clock differ. txTm is the
// server's transmit time in GMT. Add our (-4) timezone correction to get EDT local time.
// subtract our EDT local time estimate of when the server sent the data.
lagtime = (txTm + In Hours( timezone )) - (localTimeWhenServerSent);
Write( "\!n", addr, "(", serverName, ")",
"\!n TotalTime =\!t", destTm - origTm,
"\!n FlightTime =\!t", flightTime,
"\!n Difference =\!t", Format( lagtime, "hr:m:s", 15, 5 )
);