Re: Leveraging PTM

On Mon, Aug 21, 2023 at 10:56:16PM +0700, James Clark wrote:
> On Mon, Aug 21, 2023 at 3:28 PM Miroslav Lichvar <[email protected]> wrote:
> 
> > Do you have any recommendations for cheap switches with good PTP
> > support? That seems to be the most expensive part.
> 
> In the last year, FS.com has released some cheap switches with good
> PTP support. I would recommend the IES3110-8TF-R:
> 
> https://www.fs.com/products/148180.html

That looks great. How much asymmetry and jitter in PTP measurements do
you see when you compare a direct connection and different pairs of
ports of the switch?

> - the Raspberry Pi OS kernel doesn't include the igb driver; I could
> compile it myself, but that's not going to be a nice option for the
> average user; I thought of trying Fedora, but I'm not sure if its
> Raspberry Pi kernel includes the igb driver

It seems to be included:
https://src.fedoraproject.org/rpms/kernel/blob/f38/f/kernel-aarch64-fedora.config#_3029

> > As for the PTP-specific timestamping of the CM4 NIC, I'm interested in
> > tests using the NTP-over-PTP transport with the latest chrony version
> > (you might need to set hwtstimeout to 0.1 or even longer for the CM4).
> 
> That would be interesting. What kind of PTP support in a switch can
> the NTP-over-PTP transport take advantage of? I'm guessing E2E
> transparent.

Currently it doesn't take advantage of PTP support in switches (i.e.
the correction field). It would have to be a unicast one-step E2E
transparent clock. Do you know any switches that can do that?

The main point of NTP-over-PTP is to enable HW timestamping of NTP
messages on PTP-only NICs. NTP doesn't need HW support in switches to
perform well when the network is (moderately) loaded. You can still
get sub-microsecond accuracy with good (non-PTP) switches if there
are not too many of them between the server and client.

> > Verifying accuracy is much more difficult.
> 
> I was wondering if PTM could help here. The i225-T1 can do PTM and
> also output a PPS on its SDPs. So one approach to verifying the
> accuracy of
> would be to have a system with two NICs, one of which would be the
> i225-T1. Use chrony to sync the system clock using one NIC,
> and then sync the system clock to the PHC of the i225-T1 using PTM,
> and compare its PPS output to the PPS from a GPS using something like
> a TAPR TICC.

I think that would rely on PTM and PCIe between the CPU and the other
NIC having no asymmetry (including the asymmetry of the NIC reading
its HW clock), or at least them canceling out. I have not seen a
proper verification of PTM accuracy yet. The most accurate and still
somewhat practical way I know to measure accuracy of the system clock
is to run a program making timed random reads from memory to generate
a PPS signal on the memory bus, but that is not good enough to measure
PTM error.

With two identical NICs in the system you can measure the PCIe and
clock reading asymmetry between them by connecting them directly with
a cable and measuring the PTP or NTP offset. With different NICs that
is not possible if you don't know the packet timestamping errors. Both
NICs would both need to have a PPS input or output to measure the
timestamping errors, like the I210 and I225 do (and someone already
did the work and compensated the errors in the driver).

-- 
Miroslav Lichvar


-- 
To unsubscribe email chrony-users-request-kWFZVVI9zxvPqho9SqqRMmD2FQJk+8+b@public.gmane.org 
with "unsubscribe" in the subject.
For help email chrony-users-request-kWFZVVI9zxvPqho9SqqRMmD2FQJk+8+b@public.gmane.org 
with "help" in the subject.
Trouble?  Email [email protected]