[Jeff Geerling] has been following the various open source time projects for some time now, and is finally able to demonstrate a working and affordable solution for nanoseconds-accurate timekeeping in your local lab. The possibility of a low-cost time server came about with the introduction of the Raspberry Pi CM4 compute module back in Oct 2020, whose Broadcom network chip (BCM54210PE) supports PTP (Precision Time Protocol, IEEE-1588) 1PPS output and hardware-based time stamping. Despite the CM4 data sheet specifying PTP support, it wasn’t available in the kernel. An issue was raised in Feb last year, and Raspberry Pi kernel support was finally released this month.
[Jeff] demonstrates how easy it is to get two CM4 modules to synchronize to within a few tens of nanoseconds in the video below the break. That alone can be very useful on many projects. But if you want really stable and absolute time, you need a stratum 1 external source. These time servers, called grandmasters in PTP nomenclature, have traditionally been specialized pieces of kit costing tens of thousands of dollars, using precision oscillators for stability and RF signals from stratum 0 devices like navigation satellites or terrestrial broadcast stations to get absolute time. But as Lasse Johnsen, who worked on the kernel updates remarks in the video:
In 2022 these purpose-built grandmaster clocks from the traditional vendors are about as relevant as the appliance web servers like the Raq and Qube were back in 1998.
It is now possible to build your own low-cost stratum 1 time server in your lab from open source projects. Two examples shown in the video. The Open Time Server project’s Timecard uses a GNSS satellite receiver and a Microchip MAC-SA5X Rubidium oscillator. If that’s overkill for your projects or budget, the Time4Pi CM4 hat is about to be release for under $200. If accurate time keeping is your thing, the technology is now within reach of the average home lab. You can also add PTP to a non-CM4 Raspberry Pi — check out the Real-Time HAT that we covered last year.
I wonder if those small-scale rubidium clocks have gotten any more reliable. Bought bunches when they first came out – had the orders delayed and had to waiver out specs as the company walked back performance measures over temperature. And I’ve now got like 2 or 3 in the lab that are just bricks at this point. Avoided that company (and it’s descendant divisions) like the plague ever since.
@Pat said: “I wonder if those small-scale rubidium clocks have gotten any more reliable. Bought bunches when they first came out – had the orders delayed and had to waiver out specs as the company walked back performance measures over temperature. And I’ve now got like 2 or 3 in the lab that are just bricks at this point. Avoided that company (and it’s descendant divisions) like the plague ever since.”
Hi Pat. Are you referring to the Microsemi (since 2018 part of Microchip Technology [1]) MAC-SA5X (SA53 & SA55) Rubidium Clock Modules?[2] If so, I think those modules got off to a rough start but as Microchip puts it now: “Miniature Atomic Clock (MAC-SA5X) Rubidium Oscillator, re-designed for improved stability and new features”. So supposedly they have worked the bugs out now, but since the modules are still unobtainium (e.g. Mouser still lists them as “Notify Me”[3]), who knows what the truth is? Maybe ask on the Time-Nuts reflector?[4]
If the bad modules you have are from the pre-fixed era, I suggest you contact Microsemi [5] and ask for FREE replacements for re-evaluation. Explain your objectives and measurement capabilities. If I was Microsemi/Microchip and you sound like a capable evaluator, I would happily send you free replacements in exchange for the bad devices in return and your new test results (probably under NDA, initially at-least).
* References:
1. Microsemi
https://en.wikipedia.org/wiki/Microsemi
2. Miniature Atomic Clock (MAC – SA5X)
https://www.microsemi.com/product-directory/embedded-clocks-frequency-references/5570-miniature-atomic-clock-mac-sa5x
3. Microchip Technology MAC-SA5X Rubidium Atomic Clock @ Mouser
https://www.mouser.com/new/microchip/microchip-macsa5x-atomic-clock
4. Time-Nuts — Precise Time and Frequency for Amateurs
http://www.leapsecond.com/time-nuts.htm
5. Microsemi, Opens on Microchip
https://www.microsemi.com/
“Are you referring to the Microsemi”
Microsemi actually purchased Symmetricom back in ’13. That’s the company that actually developed them. They had been working on them for a long time. The “first generation” guys are the ones Microchip called “chip-scale atomic clocks.” The later generations (like the one you’re mentioning) are now called “miniature atomic clocks” but they’re virtually the same size, soooo apparently “miniature” = “chip scale.” Who knew.
“So supposedly they have worked the bugs out now, but since the modules are still unobtainium”
Amazingly the originals (the SA.45s) are still available to be purchased, and there are plenty of those in stock! I mean, they’d make exceptionally expensive Legos, I guess.
“If the bad modules you have are from the pre-fixed era, I suggest you contact Microsemi [5] and ask for FREE replacements for re-evaluation.”
Tried that! Crickets. I’d say they’re only usable as doorstops now, but they’re dang tiny.
SOLD OUT!!!!!!!!!!!!!!!!!!!!!!!!!!!
The Time4Pi board is actually still pre-release, but hopefully they’ll have it tested and in production in the next month or so.
There are also various GPS HATs you can make work with the PPS IN on the CM4 IO board; the GitHub issue has some great examples!
Speaking as someone who works with PTP regularly in a television broadcast environment, the statement about the purpose-built appliance grandmaster clocks being no longer relevant in 2022 is pretty far off the mark. What exactly is the replacement for it? A network card that has hardware timestamping, a precision oscillator, and a GPS receiver on it? You still need the accompanying software stack to provide PTP services out to the network, and once have that, you have a device that shouldn’t be used for anything else. Additionally, the appliances provide other forms of synchronization, like black burst or multiple forms/frequencies of pulsed output needed by devices that require precise time synchronization but aren’t going to run PTP. You can’t run a separate card in each device requiring a precision time lock to an external source either, since each one of those will need an antenna with visibility of the satellites. Also the priority might be for the devices to be locked to each other more than to an external time source. In those cases, loss of GPS signal isn’t as critical as long as all devices within the plant remain in lockstep with each other, and the internal oscillator on the grandmaster holds reasonably stable to prevent too much drift.
I can see it being true that for the synchronization of a single machine, you won’t need an external grandmaster. You can certainly then feed that back out to other devices in the network. But PTP is one of the more confusing implementations of multicast, and there’s a lot of constant two-way communication that goes on between server and clients. To scale out, you rapidly run into a need for boundary clocks, lest you start to severely load up the CPU on the grandmaster as the number of clients increases. Fine for small lab environments, and probably useful for servers hosting virtual machines in some circumstances, but the application really dictates the design of the clock. If you’re trying to synchronize multiple video/audio sources, or stock purchases on the trading room floor, or cell phone towers, or electrical phase in the power grid, you’re going to need multiple ruggedized, dedicated clocks. Nobody needed an appliance like a Raq or a Qube for very long, because it’s the sort of function general purpose machines could provide in their spare time. Synchronizing a network via PTP is a full time job in a production environment. Acting as an Internet router is something that doesn’t require purpose built appliances either, and yet we’re awash in those.
Good points; I think Lasse was using a little hyperbole in that statement, but from one perspective (smaller installations, edge deployment, that sort of thing) the dedicated equipment is wildly out of place compared to a more integrated solution.
He mentioned (in the video, this is not quoted in the blog post) that it’s not suitable for high-bandwidth applications, but it can be useful when you don’t have such needs (smaller studios, edge compute, remote datacenter are probably the best target currently).
I guess I’m just not fond of hyperbole in technology announcements. It makes it sound too much like a “computer programmer discovers weird timekeeping trick, Big Clock hates it!” kind of claim for my taste. Don’t get me wrong, I would absolutely love an affordable Pi module I could use to build a better PTP source for development and testing. Even an all-in-one (reasonably priced) PCI card would work. But considering the fact that I can get something like a Time Machines TM2000B for under US$600 (or a TM2500C at US$800 if I really need the PPS options), it’s sort of tough to claim that existing PTP appliances are wildly out of place in smaller deployments. PTP is an infrastructure level service, especially when the application can’t tolerate the kinds of general purpose precision levels you’ll find in protocols like NTP. The operationalization of the solution needs to be considered as well, especially in a production setting. Even in the small installation/edge deployment scenario, I would want to make sure the person who might have to diagnose and repair the item wouldn’t be up against something custom.
And let’s face it, if something like this Pi module gets rolled into a commercial offering, that offering will be sold as… an appliance.
The MAC-SA5X isn’t even available from suppliers yet, only directly from Microchip.
Rubidium is nowhere as radioactive as you try to make it look like in the video. At 670 Bq/g not much when we consider the ampoule we are presented contains gas, possible in the milligram range. So 0.7 Bq/vial, less than background radiation. I wish people would stop pretending something is dangerous just to scare viewers on youtube.
Now compare 0.7 Bq to the 4300 Bq of the 135 g potassium inside the human body, impossible to avoid and hitting cells. Is there a literate way to present science and not go straight for pop-science? A simple calculation showed 5 seconds for the interested reader instead of “boo hoo scary cancer clocks!”.
I think that was exactly the point made in the video (that it’s not dangerous, like “a banana”).
“You can see that they even get within a couple of ns from time to time.”
Yeah, my stopped watch is right twice a day as well.
Other than the lapsus with the best case offset, the video was well made. So does the hardware PTP time stamp uses an adjustable clock in the CM4 (afaiu that’s holding the BBB back).
I would like to find an *affordable* GPS disciplined 10 MHz source for the test gear on my bench. eBay seems to be crawling with similar-looking units sourced from China containing what appear to be recycled modules. Who knows if they actually function?, and there is probably little chance of getting a refund if you receive a DOA unit.
Quantify “affordable”.
You can get a Leo Bodnar GPSDO for about $160. Adjustable output frequency to boot.
You can roll your own for about half that. Projects for that are everywhere.
Leo Bodnar: http://www.leobodnar.com/shop/index.php?products_id=301
Why has everyone forgotten about LF clock standards?
WWVB (USA), DCF77 (Europe), RBU, RTZ (Russia), etc. ? To receive them you don’t even need any complicated equipment, it’s even possible to receive without ICs (if you only need frequency, without time and calendar)
As I remember, they are in 10-100 times precise whan NAVSTAR GPS onboard clocks
The first 5 minutes of the video are dedicated to broadcast time standards (WWV in particular). It is actually going to be an alternate option for time sync on a future version of the Time Card (in addition to GPS).
This is good! But a tiny clarification: ultra-long waves (near of 50-100kHz) meant here – 10MHz beacon will give you less accuracy than GPS.
By the way, one of the real applications of such accurate clock is CAP theorem solving (also named Brewer’s theorem): the more accurate sync of clocks then faster the synchronous data transfer to data centers will occur.
Has anybody tried GPS PPS+IEEE 1588 with a Banana Pi R1 which has a Broadcom BCM53125 and 5 ports? According to the docs it should support IEEE 1588 hardware timestamping, but I know this alone isn’t enough, the whole system (chip, board, driver) need to support that and there are also multiple levels of the support.
Hello. I wrote the PTP 1588 code for the BCM54210PE in the CM4. I am in Jeffy G’s video above. Since then we made a “sandwich” Timecard Mini for the CM4. I am here shamelessly plugging it… it’s ideal for PTP on the Raspberry PI #probably ;-)
https://store.timebeat.app/products/gnss-raspberry-pi-cm4-module
Cheapest version is $50
/Lasse