chrony not switching to stratum 10 after loss of references

Hi,

I'm having a problem with a system configured as a stratum 1 NTP server, running chrony, and I can't seem to be able to figure out what is happening and what might be the root cause of the problem.
(details about the setup are at the end of this message)


Basically, my stratum 1 servers takes a 1PPS and NMEA timecode on a serial port.  It also runs gpsd.  I've configured the SHM references as such in /etc/chrony.conf:

# Configure NMEA + 1PPS via shared memory:
refclock SHM 0 refid NMEA precision 1e-1 offset 0.000109
refclock SHM 1 refid PPS offset 0.0 precision 1e-7
<... snip ...>
local stratum 10

With this configuration, chrony is happy, sees the 2 signals (e.g., reach of 377 on each) and serves time as a stratum 1 server.  All good so far.



But then I intentionally stop the 1PPS and timecode signals to simulate a failure of the upstream reference.  Because of the "local stratum 10", I was expecting chrony to switch stratum 10 automatically when the reach value reaches 0 on those 2 references, but this is definitely not what I'm seeing.

Instead, it seems like chrony continues to be a stratum 1, long after the reference signal have been terminated.
For example, the chrony sources and tracking command output below was taken approx. 1 hour after termination of 1PPS and NMEA signal:

$ sudo date; sudo chronyc sources; sudo chronyc tracking
Wed Jun 14 08:23:00 PM UTC 2023
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
#? NMEA                          0   4     0   57m    -93us[  -93us] +/-  100ms
#? PPS                           0   4     0   57m    +98ns[ +346ns] +/-  101ns
Reference ID    : 50505300 (PPS)
Stratum         : 1
Ref time (UTC)  : Wed Jun 14 19:25:50 2023
System time     : 0.000000041 seconds fast of NTP time
Last offset     : +0.000000256 seconds
RMS offset      : 0.000000151 seconds
Frequency       : 12.835 ppm fast
Residual freq   : +0.006 ppm
Skew            : 0.008 ppm
Root delay      : 0.000000001 seconds
Root dispersion : 0.003475379 seconds
Update interval : 16.0 seconds
Leap status     : Normal

I've also confirmed that it still happily serves time at stratum 1 level if I query it from another system (using ntpdate tool for example from another system).

Is this normal and expected?  I was expecting chrony to fallback to stratum 10 but this is not what is happening.

There is probably something obvious I'm missing in my chrony.conf file to implement the desired behavior but I'm not able to figure it out.

Anyone else experienced this before?


System and setup info:

HP EliteDesk 800 G2 SFF
chrony version 4.3
gpsd 3.23.1-1.el9
AlmaLinux release 9.2 (Turquoise Kodkod)
Linux 5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 05:49:00 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

chrony.conf:

refclock SHM 0 refid NMEA precision 1e-1 offset 0.000109
refclock SHM 1 refid PPS offset 0.0 precision 1e-7
driftfile /var/lib/chrony/drift
leapsecmode system
makestep 1.0 3
rtcsync
hwtimestamp *
minsources 1
allow ***.***.0.0/16
local stratum 10
logdir /var/log/chrony
log measurements statistics tracking refclocks tempcomp




Any information or insights about this would be much appreciated.

Thanks,
  Andre


--
Andre Charbonneau

Frequency & Time
Metrology Research Centre
National Research Council Canada / Government of Canada
[email protected]<mailto:[email protected]> / 613-993-3129

Fréquence et temps
Centre de recherche en métrologie
Conseil national de recherches Canada / Gouvernement du Canada
[email protected]<mailto:[email protected]> / 613-993-3129