Bug#982950: ssh.service starts sshd before network is online: please switch to After=network-online.target instead of just After=network.target

On 2/17/21 10:14 AM, Colin Watson wrote:
> Control: severity -1 important
> 
> On Wed, Feb 17, 2021 at 09:36:15AM +0100, Thomas Goirand wrote:
>> Package: openssh-server
>> Version: 1:8.4p1-4
>> Severity: grave
> 
> No.  It may yet need to be sorted out before release, but this is a rare
> situation and I'm not having it being release-critical right now,
> especially because it's not clear that it's openssh-server's problem.

Let's not discuss the severity: let's try to fix the issue instead.

>> On a Sid/Testing system, currently we have in /lib/systemd/system/ssh.service:
>>
>> After=network.target auditd.service
>>
>> While this isn't a problem in most installation, it didn't work under our setup,
>> because we use "bgp-to-the-host" networking. In this setup, we need FRR (the
>> BGP routing daemon which is a fork of Quagga, if you didn't know) to provide
>> network connectivity to the server. Our configuration is something like this:
>>
>> # cat /etc/frr/frr.conf
>> [...]
>> !
>> int lo
>>  ip address 10.56.17.7/32
>> !
>> [...]
>>
>> This means that, until FRR is fully up and running, with the BGP session
>> established, the server IP (10.x.x.x/32 bound to the loopback interface) isn't
>> set yet on the server, and the ssh daemon cannot bind on the IP (as it's not
>> there yet).
> 
> Are you using ListenAddress in sshd_config?

Yes, with the same IP as above, in order to make sure ssh isn't
listening on a public IP (which would be a security concern for us).

> Normally sshd doesn't
> require network interfaces to be online - it just binds to 0.0.0.0 or
> [::] and that's enough for it to be bound to interfaces as they come up.
> 
> If lo has to be up for this to work (which is surprising to me, but
> maybe), then I think there's a decent argument for frr to be part of
> network.target on such systems.

The interface is up before FRR start. Though the IP on localhost is only
added when FRR has established a working BGP session with its peers
(here, the 2 switches the machine is connected to).

> See also
> https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/, which
> among other things (in general the tone of that page is that a
> well-written service should not use After=network-online.target) says:
> 
>   "If you write a server: listen on [::], [::1], 0.0.0.0 and 127.0.0.1
>   only.  These pseudo-addresses are unconditionally available."
> 
> That's what sshd does in its default configuration.  If it doesn't work,
> the systemd documentation suggests that something else is not fulfilling
> its end of a contract somewhere.

Maybe setting-up net.ipv4.ip_nonlocal_bind=1 (in sysctl.conf) would fix
my issue, no?

Your thoughts?
Cheers,

Thomas Goirand (zigo)