Re: Re: [Users] how FS is supposed to deal with problems further down the stack

On Tue, Mar 11, 2003 at 09:53:35AM -0500, John S. Denker wrote:
> On 03/10/2003 08:12 AM, martin f krafft wrote:
> 
> > maybe we should start discussing what is the ideal situation.
> 
> Right.  The current behavior is far from ideal.

Sorry to weigh in so late in this discussion...

> Initially:
> 
> > note the order of the interfaces for similar routing entries, eth 
> > before ipsec.
> 
> after ifdown + ifup:

Something asymmetric happens in this process which has only been touched
on by John below...  This is the place where the problem lies...

>  > now we have a problem. the routing entries for the 2.3.4/28 net
>  > have switched, and the default route is now dev ipsec0.
> 
> Yes, that's one problem among many.

I don't have an obvious solution for that...  but others have made
suggestions.

>  > in none of these cases was /usr/lib/ipsec/_updown ever called!
>  > it seems to be only called whenever a route is up'd or down'd with
>  > ipsec auto.
> 
> Yes, I had noticed that.

This is not an accident.

> > i do know what messes up the default route. my dhclient-script just 
> > adds a default route without the dev argument to /sbin/route.
> > changing that so that dev $interface is appended actually fixes the
> > default route problem, i can ping the outside world...
> 
> What does the _default_ dhclient-script do?
> If the default causes problems, this needs to
> be prominently documented.
> 
>  > but all my tunnels are dead, and apparently irrevivably.
> 
> Right, did you notice that all their eroutes have
> been wiped out?

This is where I have been confused all along...  I have not seen any
status output to prove this is the case.  The eroutes and the SAs don't
disappear.

>  > it won't work until i restart Pluto. ipsec auto {delete,add,up}
>  > doesn't work either, and neither does {unroute,route,up}, i always get
>  > the same error 22 as above.
> 
> You can learn a lot by invoking
> 	ipsec klipsdebug --all
> 
>    Note that whereas pluto logs to the authpriv facility
>    (so msgs typically up in /var/log/secure), klips
>    doesn't (so msgs typically wind up in /var/log/messages).
> 
> Anyway, when the device is downed you will see msgs like:
> klips_debug:ipsec_device_event: NETDEV_GOING_DOWN dev=eth0
> klips_debug:ipsec_device_event: NETDEV_DOWN dev=eth0 flags=2
> klips_debug:ipsec_tunnel_detach: physical device eth0 being detached 
> from virtual device ipsec0
> klips_debug:ipsec_device_event: device 'ipsec0' has been detached.
> 
> But (!) when the device comes back up, there is no
> attempt to re-attach it.
> 
> The "detach" occurs at a level deeper than any particular conn.  Think
> about the line "interfaces=ipsec0=eth0" in ipsec.conf, which affects
> all conns.  If this is broken, no amount of upping and downing
> individual conns is going to help.

Right, and this is exactly the place to fix it.  Running something that
looks in the config to see what virtual interfaces are connected to what
physical interfaces and re-do the ones you care about.  For example
above, the following command needs to be run at minimum to re-connect
it:

	ipsec tncfg --attach --virtual ipsec0 --physical eth0

This shoud be all that is necessary upon detection of re-connection of a
physical device that vanished.

That behaviour is there so that any physical devices that disappear
don't fail to clear when they come back.

>  > so how should it be?
>  >
>  > we have two options: ifdown causes all tunnels to be teared down and
>  > ifup causes them to be re-established. this is ugly and expensive.
> 
> Right.  That would be an improvement over what we've
> got now, but it would indeed be ugly and expensive.

It is, but it makes sure you start from a known state.

> On the other hand, we need to detect the case where the wild-side
> address changes to a new value, in which case all the affected tunnels
> will have to be re-established.  (One can dream of optimizing this to
> support mobility by providing for authenticated "I've been moved"
> messages that don't require full rekeying, but that's a discussion for
> another day.)

This one can't be fixed by tncfg since the change in addresses affects
the eroutes *and* the SAs.

>  > the other option is: ifdown goes unnoticed, FreeS/WAN does nothing.
> 
> Recall that not too long ago, klips would cause a kernel panic if a
> device went down.  So in this sense, it would be undesirable for
> ifdown to go completely unnoticed.  Perhaps it would be clearer to say
> that klips shouldn't over-react and shouldn't do anything
> irreversible.

For a while (I'm not sure if it is still the case...) there was a clue
kept in the ipsec device about what physical device to which it used to
be attached.

> It should decide on a packet-by-packet basis to throw away packets
> that are destined for an interface that is presently down.

It already does this.  It routes into klips, klips eroutes it, SAs
process it only if the virtual device is connected to a physical device.

> > if the tunnel stays down for too long, the SAs are going to die.
> > tough luck. but if the ifup happens within seconds afterwards, the
> > tunnels should just be able to live on.
> 
> Exactly.

This is not quite true.  It depends on the virtual device still being
connected to a physical device through which it can pass IKE packets.

>  > now i have been trying to hack this to oblivion but i am still stuck
>  > at one point that i can't figure out: why in the world don't the
>  > tunnels work anymore once i tore down the interface, brought it back
>  > up, and then manually restored the routing table to its original
>  > (working) state? answer that question for me and i'll continue diving
>  > into the depths of this problem.
> 
> 1) Fixing this is going to require nontrivial changes in
> ipsec_tunnel.c and perhaps elsewhere.

I don't think it is that big a fix.

> I suspect the current behavior was put in as a stopgap to avoid
> outright panics.  I suspect it was never tested.  It's hard to test
> something for desired behavior before any clear notion of "desired"
> behavior has been articulated.

As stated above, it was put in place to avoid information leakage.  It
is very deliberate.

> 2) Then there will need to be some userland daemon (pluto or
> friend-of-pluto) and/or hooks in various up/down/dhcp scripts to
> notice up/down events and provoke the correct response.

That is one way to do it, yes.

> The toughest case is where there are multiple ipsecN interfaces, and
> one of them has just come up (due to pcmcia card insertion or
> newly-granted DHCP address).  Then you have to up all the conns using
> that interface without disturbing the conns on other interfaces.

Connecting the device should not be a big challenge, but clearing out
old SAs and eroutes related to an old IP address are definitely a bigger
challenge.

	slainte mhath, RGB

-- 
Richard Guy Briggs           --    ~\                 Auto-Free Ottawa! Canada
<www.TriColour.net>            --    \@       @           <www.flora.org/afo/>
No Internet Wiretapping!        --   _\\/\%___\\/\%        Vote! -- <Green.ca>
<www.FreeSWAN.org>_______GTVS6#790__(*)_______(*)(*)_______<www.Marillion.com>