Re: Leveraging PTM

On Tue, Aug 22, 2023 at 5:58 PM Miroslav Lichvar <[email protected]> wrote:

> I think that would rely on PTM and PCIe between the CPU and the other
> NIC having no asymmetry (including the asymmetry of the NIC reading
> its HW clock), or at least them canceling out. I have not seen a
> proper verification of PTM accuracy yet.
...
> With two identical NICs in the system you can measure the PCIe and
> clock reading asymmetry between them by connecting them directly with
> a cable and measuring the PTP or NTP offset.

This is a nice idea. I set up my system with 2 x i225-T1s (with a 12th
gen CPU). But it should also work on the 4-port box I mentioned
earlier.

My i225s are on enp4s0 and enp5s0, and I connected them with a short
patch cable. The steps I went through are:

1. Use phc_ctl to get the PHCs roughly right:

phc_ctl enp4s0 set
phc_ctl enp5s0 set

2. Start two instances of ptp4l to sync and measure the PHCs:

ptp4l -q -m -2 -i enp4s0
ptp4l -q -m -2 -i enp5s0

This will sync them and show the offsets, which become small (<10ns)
after it settles down. (I'm using -2 because they don't have IP
addresses.)

3. Stop chrony (forgot this the first time)

4. Get the system clock synced to the PHCs

 phc2sys -s enp4s0 -q -m -O 0

Let  this settle and then interrupt

5. Finally do

phc2sys -s CLOCK_REALTIME -c enp4s0 -c enp5s0 -O 0 -q -m --free_running=1

This prints out pairs of offsets for enp4s0 and enp5s0

phc2sys[333052.959]: enp5s0 sys offset     -1472 s0 freq      +0 delay      0
phc2sys[333052.959]: enp4s0 sys offset     -1480 s0 freq      +0 delay      0
phc2sys[333053.960]: enp5s0 sys offset     -1511 s0 freq      +0 delay      0
phc2sys[333053.960]: enp4s0 sys offset     -1515 s0 freq      +0 delay      0
phc2sys[333054.960]: enp5s0 sys offset     -1563 s0 freq      +0 delay      0
phc2sys[333054.960]: enp4s0 sys offset     -1561 s0 freq      +0 delay      0
phc2sys[333055.961]: enp5s0 sys offset     -1602 s0 freq      +0 delay      0
phc2sys[333055.961]: enp4s0 sys offset     -1611 s0 freq      +0 delay      0
phc2sys[333056.961]: enp5s0 sys offset     -1658 s0 freq      +0 delay      0
phc2sys[333056.961]: enp4s0 sys offset     -1665 s0 freq      +0 delay      0

which are within about 10ns of each other.

Doing

sudo lspci -vv | grep -i granularity

shows a PTM granularity of 4ns.  The clock rate of 2.5GBASE-T is
200MHz, which I think limits PTP accuracy to 5ns.

So if the difference of the phc2sys offsets are within 10ns, I believe
PTM is working as one would hope.

James

-- 
To unsubscribe email chrony-users-request-kWFZVVI9zxvPqho9SqqRMmD2FQJk+8+b@public.gmane.org 
with "unsubscribe" in the subject.
For help email chrony-users-request-kWFZVVI9zxvPqho9SqqRMmD2FQJk+8+b@public.gmane.org 
with "help" in the subject.
Trouble?  Email [email protected]