What time is it?

[#1]

I've been re-purposing old cell phones as clocks for several years.

Android 2.2.2 - Froyo. This one still going, its sibling has failed. And more recent phones have failed.

Two have just failed. That's the last one running...good hardware. Really old OS, but still gets the time off the network and still updates when we switch to daylight time and back.

So I need to replace them, and I started thinking about Arduinos and Raspberries and stumbled into ESP32.

Quarter inch squares, this is about 25cm x 50cm

This is from espressif and I got this one on amazon, 3 for $17. So far I've played with blinking an LED and using the built-in WIFI hardware. Yes, WIFI, and blue tooth, and lots of I/O pins, 4M rom and 512K ram (I think. have not explored far enough to know for sure.) In the process, I ran the NTP (Network Time Protocol) example and started wondering how NTP works.

At the bottom, NTP is really simple. A computer that wants to know the time sends a short message to another computer that hosts an NTP server, and that computer sends a short message back that includes the GMT time. Done.

(It is much more complicated; apparently the NTP servers may answer the question 1000 times a second! And the mechanisms for synchronizing the servers use statistics. And David L Mills invented the NTP stuff around 1980.)

The internet mostly uses TCP (Transmission Control Protocol), a robust protocol built on top of IP. TCP retries, etc, to make sure data is delivered, in order, and intact. It is a heavy-weight protocol. NTP servers use a light weight protocol, UDP (also on top of IP) which guarantees nothing. Your messages might get lost, never arrive, arrive out of order. But the load on the server is minimal; the server doesn't need to do any extra hand-shaking to make sure you get an answer. The U in UDP (User Datagram Protocol) means the user is responsible for handling errors.

This JSL creates a socket to use port 123 (the NTP port) and the UDP protocol to send a message to a pool of NTP servers and get the response. It does some really simple statistics math that assumes the travel time to and from the server are equal so the server's time can be synced with the local computer better. The results are pretty good! Here's a slightly better than average example that found a server in the pool half-way around the world:

pool.ntp.org(rn5.quickhost.hk)
TotalTime = 0.143846988677979
FlightTime = 0.143799781799316
Difference = 0:00:00.01689

Assuming .hk is really in Hong Kong, that's 8,000 miles from North Carolina. 16,000 round trip. The Flight Time is the Total Time less the processing time in Hong Kong, which was negligible. If I did my math right, that's more than 100,000 miles/second, a significant fraction of the speed of light.

The last number, difference, is the difference between my CPU clock and the NTP server's clock. That's less than 1/50 second. And I get results from most servers within 1/10 second, unless the flight time is large, in which case it is asymmetrical and the correction technique breaks down.

pool.ntp.org(LAX.CALTICK.NET)
TotalTime = 0.101963043212891
FlightTime = 0.101903915405273
Difference = 0:00:00.08646

pool.ntp.org(ns2.nuso.cloud)
TotalTime = 0.103617668151855
FlightTime = 0.103596687316895
Difference = 0:00:00.04034

pool.ntp.org(50-205-57-38-static.hfc.comcastbusiness.net)
TotalTime = 0.0763883590698242
FlightTime = 0.0763206481933594
Difference = 0:00:00.00965

pool.ntp.org(13.86.101.172)
TotalTime = 0.077089786529541
FlightTime = 0.07708740234375
Difference = 0:00:00.00663

// this snippet demonstrates a UDP (User Datagram Protocol) exchange with an NTP 
// (Network Time Protocol) server using a JSL socket. NTP sends enough information 
// from an NTP server to correct for some network delays and make a really good guess
// at the local time, possibly within 1/100 second, usually within 1/10 second (depending
// on how similar send and receive times are.) Using the OS time setting to sync your
// clock first (which uses NTP as well) will make the deltas near zero. Otherwise the 
// deltas should be similar across several runs. Your computer clock will drift between
// the syncs (once a day?) that your computer automatically performs with an NTP server.
//
// idea from https://lettier.github.io/posts/2016-04-26-lets-make-a-ntp-client-in-c.html

timezone = -4; // (tz for my computer) Eastern Daylight Time is GMT-4.
// build the transmit packet to send to the NTP server
xPacket = Hex To Blob( "1b" || Repeat( "00", 47 ) );
// make the socket object
sock = Socket( DGRAM ); // not a normal STREAM socket, this is an unreliable UDP socket.
// choose a server (or server pool)
addr = "pool.ntp.org"; // a pool of servers. Don't ask too often, they have a heavy load.

// if nothing goes wrong, the bind and ioctl are optional, but
// if we never get a response (for various reasons) we'd rather not hang forever...

rc = sock << bind( addr, "123" ); // bind, maybe not required, but needed for NBIO...
If( rc[2] != "ok",
	sock << close;
	Throw( "bad bind" );
);

rc = sock << Ioctl( FIONBIO, 1 ); // NBIO: non blocking i/o. the unreliable UDP may never answer.
If( rc[2] != "ok",
	sock << close;
	Throw( "bad ioctl" );
);

// send our request to the server. <<sendto is for unreliable UDP packets.
// it is possible the server may not get the packet.

sendHPtime = HP Time(); // remember, in high precision, when the transaction begins
rc = sock << sendto( addr, "123", xPacket );
If( rc[2] != "ok",
	sock << close;
	Throw( "bad send" );
);

// wait for a response from the server. If the server got the request,
// it might not send a response if we've asked too many times recently.
// And if it does send a response, the response could get lost.
// <<recvfrom is for unreliable UDP packets.

timeOut = Tick Seconds() + 5; // timeout in 5 seconds
While( Tick Seconds() < timeOut,
	Wait( 0 );// allow processing in background
	rc = sock << recvfrom( 48 );// the return packet is the same 48 byte structure
	recvHPtime = HP Time(); // remember, in high precision, when the transaction might have ended
	If( !Starts With( rc[2], "WOULDBLOCK" ),
		Break() // the transaction ends when we actually get data back
	);
);
// see if we got good data. Possibly not all the data arrived, but I have not seen that yet.
If( rc[2] != "ok" | Length( rc[3] ) != 48,
	sock << close;
	Throw( Char( rc ) || " time out" );
);

sock << close;

// UDP, like TCP, is built on top of IP, but UDP is connectionless. Since there is
// no fixed connection, the return packet tells who it came from, and that might be
// any name in the pool of servers. (see addr at top)
serverName = rc[4]; // [5] is the port, probably always "123"
// the received data should be a 48 byte packet
rPacket = rc[3];

// https://github.com/lettier/ntpclient/blob/master/source/c/main.c
// added some annotations...
////uint8_t li_vn_mode;      // Eight bits. li, vn, and mode.
////                             // li.   Two bits.   Leap indicator.
////                             // vn.   Three bits. Version number of the protocol.
////                             // mode. Three bits. Client will pick mode 3 for client.
////
////    uint8_t stratum;         // Eight bits. Stratum level of the local clock.
////    uint8_t poll;            // Eight bits. Maximum interval between successive messages.
////    uint8_t precision;       // Eight bits. Precision of the local clock.
////
////    uint32_t rootDelay;      // 32 bits. Total round trip delay time.
////    uint32_t rootDispersion; // 32 bits. Max error aloud from primary clock source.
////    uint32_t refId;          // 32 bits. Reference clock identifier.
////
////    uint32_t refTm_s;        // 32 bits. Reference time-stamp seconds. (when the server clock was last set, ignore this)
////    uint32_t refTm_f;        // 32 bits. Reference time-stamp fraction of a second.
////
////    uint32_t origTm_s;       // 32 bits. Originate time-stamp seconds. (filled with zeros above, use sendHPtime)
////    uint32_t origTm_f;       // 32 bits. Originate time-stamp fraction of a second.
////
////    uint32_t rxTm_s;         // 32 bits. Received time-stamp seconds. (when server got xPacket)
////    uint32_t rxTm_f;         // 32 bits. Received time-stamp fraction of a second.
////
////    uint32_t txTm_s;         // 32 bits Transmit time-stamp seconds. (when server returned rPacket)
////    uint32_t txTm_f;         // 32 bits. Transmit time-stamp fraction of a second.
////
//// and finally, recvHPtime is when we got the rPacket back https://tools.ietf.org/html/rfc5905
////
////  } ntp_packet;              // Total: 384 bits or 48 bytes.

// unpack... https://stackoverflow.com/questions/29112071/how-to-convert-ntp-time-to-unix-epoch-time-in-c-language-linux
// convert the 48-byte blob into twelve 4-byte integer values, then
// grab the whole and fraction values of the receive and transmit times.
rIntMatrix = Blob To Matrix( rPacket, "uint", 4, "big" );
rxTm_s = rIntMatrix[9]; // the server's receive time (seconds)
rxTm_f = rIntMatrix[10]; // the server's receive time (fraction)
txTm_s = rIntMatrix[11]; // ditto, server transmit time
txTm_f = rIntMatrix[12];
// join the whole seconds to the fractional seconds and convert to 
// JMP's date time epoch (1904 for JMP. 1900 for NTP. 1970 for linux. 1601 for windows with 10^-7 scale.)
// "rx" and "tx" Tm are on the server computer
rxTm = As Date( rxTm_s + rxTm_f / (2 ^ 32) + 1jan1900 );
txTm = As Date( txTm_s + txTm_f / (2 ^ 32) + 1jan1900 );
// spin for a second to sync hptime to actual time
// hptime is in microseconds since JMP started. Find the
// hptime that matches a change in the current second.
oldTd = Today(); // truncated to seconds
While( (nowHp = HP Time() ; newTd = Today()) == oldTd, 0 );
// newTd and nowHp are the same point in time with different units.
// addToHpSeconds is in seconds (not microseconds) and can be added
// to (hp/1e6) to get local time with fractional seconds
addToHpSeconds = newTd - nowHp / 1e6;
// get our original send and receive times with fraction seconds.
// "orig" and "dest" Tm are on the local computer
origTm = sendHPtime / 1e6 + addToHpSeconds;
destTm = recvHPtime / 1e6 + addToHpSeconds;
// "flightTime" is the time the packet (x and r) was "in the air", which does not include the
// time the packet was being processed by the server.
flightTime = (destTm - origTm) - (txTm - rxTm);// dest-orig is total time we waited, tx-rx is server processing
// assume the outbound and inbound times are the same. Half of the flight time
// is how long it has been since the server sent the current time to us.
returnTime = flightTime / 2; // this is our estimate of the return trip duration
localTimeWhenServerSent = destTm - returnTime; // when we received, minus estimate
// finally, see how much the local clock and server clock differ. txTm is the
// server's transmit time in GMT. Add our (-4) timezone correction to get EDT local time.
// subtract our EDT local time estimate of when the server sent the data.
lagtime = (txTm + In Hours( timezone )) - (localTimeWhenServerSent);

Write(	"\!n",	addr,	"(",	serverName,	")",
	"\!n TotalTime =\!t",	destTm - origTm,
	"\!n FlightTime =\!t",	flightTime,
	"\!n Difference =\!t",	Format( lagtime, "hr:m:s", 15, 5 )
);

[Craige_Hales]

update... the blue light at the far right is the esp32. It's 4:05 PM.

It's really hard to get the light right to take a picture of an LED display.

The 2 1/4 inch tall digits require 8 volts, not the 2 volts the old data sheet suggested. That means I need the extra power supply in the top right, made from junk-box parts, and the 4 driver chips came from the same box. There is a $12 GPS at the bottom right too. The time can be set from the GPS if it can see some satellites, or from the wifi if the password has not been changed.

Not sure where JMP fits into this yet, but I've got a pressure/temperature/humidity sensor I want to add. Maybe it will do some data logging and run a webserver. Or maybe I'll add a small display for the weather. Once the details are worked out, I'll be making a PCB. Those wires won't work on the wall.

[Craige_Hales]

Circuit boards are in! Designed with KiCad, manufactured by OSHPark, 20 days including 11 days of manufacturing and 8 days of free shipping (saving where I can!) Probably shouldn't show these until I know they work...prototypes...

This is my first board that will have components front and back.

The rough points around the edges are where my boards were broken out of a larger panel; I'll clean them up soon, they are pointy! The boards are about 3x9 inches. I'm using through-hole technology; all the surface mount bits will be on pre-built cards (GPS, BME, and the ESP32). I went with the AfterDark boards even though the time was a bit longer. I'm liking the look! A preliminary test fit says the parts I designed footprints for are going to work. Yay!

[Craige_Hales]

Prototype built and working.

It's hard for me to call this the back; it was the front in KiCad.

The flash makes the white balance show the red digits as red.

With no flash, the white balance messes up the colors, but this is closer to what it really looks like otherwise.

The linear regulators run really hot; the 5V regulator delivers about 1/4 amp but that means the 16V input drops 11 volts * 1/4 amp = almost 3 watts into that tiny heat sink. Time to look at a drop-in replacement switch-mode regulator. Apparently the same package size, but 90% efficient.

[Craige_Hales]

Before I order revision 2 of the PCB I'm building the other two boards from the first order. Like the first one, but in parallel, construct a bit and test. And I learned something I didn't expect.

Build phases look something like this:

1. Put all the low-profile, small, inexpensive components on the board. They are hard to do later.
2. Add the LED drivers, three 16-pin ICs.
3. Add the power supply components and see if the power indicator LEDs light up.
4. Add the CPU, make sure it will accept a downloaded program.
5. Add the digits on the other side. The digits hide things, so they must be last.

Some of these prototypes are not working correctly.The digits (four of them) are modestly expensive and somewhat delicate. I somewhat randomly placed one (hours-one position) and did a quick test. Only some of the segments worked. Back to the workbench, visual check, meter check, finally pull out the scope. At this point there are a half-dozen things that could be wrong; anything from the CPU to the driver ICs to the digits. It isn't the basic design; I've got two working prototypes (one on a breadboard, one on the first PCB.) 

The clock is a multiplexed design; each driver chip has four outputs. One of the drivers selects a digit and the other two drivers select seven segments and a decimal point. A 4X8 matrix of 32 elements, addressed by 12 bits. Up to eight elements can be on at a time (one digit). Luckily, I'm looking at a digit that should have most of its segments on and I'm seeing all missing segments that belong to one driver chip. Using the scope on that chip, I see the inputs are requesting the segments be on, but the outputs are...dead.

Replacing an IC is a pain. I grabbed a fresh one from the tube, got it installed, and...Bummer! It is better, but one segment is still not showing! Did I misdiagnose? Something else crumbling?

Wait. I'm building two cards in parallel. Let's look at the other one. Nothing. No segments. After a bit of study...Hmm. This time the problem is not the driver for the segments, it is the driver for other axis in the matrix, the one that selects a digit. Do I have a whole tube of bad ICs?

This was an unused tube of ICs, purchased for another project about 10 years earlier. I need to build a tester. Here's the tester showing four green lights for a chip that I got recently from a different distributor.

All 10 in the newer tube checked out good.

I currently believe at least 5/10 of the chips in the other tube were bad on one or more outputs:

Crush the pins in (to prevent re-use) and roll them on their backs so they look like dead bugs. Even 3/4 OK is not good enough when all four outputs are needed!

What I learned might be about testing the chips before install, but I don't think that is really needed. But what I will do is use the scope to do a test, probably reversing items 2 and 3 in the assembly, so I can test a little better before attaching a digit. Then, for placing the first digit, pick one that blocks the power supply, not a driver chip. Right now I've got to remove a digit so I can remove a driver IC.