ipsec/ipfilter interaction problem

The docs on www.netbsd.org/Documentation/network/ipsec/#ipf-interaction
say that ipf looks at packets BEFORE IPSEC processing on inbound traffic 
and AFTER IPSEC processing on outbound traffic.

I seem to have a problem which contradicts this (sometimes):

There are two  NetBSD machines running an ipsec tunnel over the internet
(Both netbsd-1-6).
Basically the setup works, traffic is encrypted and shows up as
ESP packets on tcpdump.
   
Ipf is configured to pass ESP Protocol in and out between these
two machines.
   
But suddenly, (after some amount of time or bytes) when I try to
ssh from one to the other machine or when trying to send mail,
the SYN-ACK reply of the responding machine gets blocked by its
ipfilter:
   
-------------------------------------- 8< --------------------------------------
Sep 25 20:13:45 hostb ipmon[102]: 20:13:44.159219 fxp1 @0:18 b 1.2.3.4,22 -> 5.6.7.8,52161 PR tcp len 20 60 -AS 861376014 1945689524 16384 OUT 
-------------------------------------- 8< --------------------------------------

This means, ipf blocks the packet, before it is IPSEC processed.

Running /etc/rc.d/ipsec reload on that machines cures the problem.

The same hapens on both hosts.

I would be grateful, if anyone could help me track this down.

/etc/ipsec.confs look like follows:

hosta:
-------------------------------------- 8< --------------------------------------
#-------------------------------------------------------------------------------
# Add SA's
#-------------------------------------------------------------------------------
add 5.6.7.8 1.2.3.4 esp 25252 -E rijndael-cbc "SomeKeySomeKeySomeKeySomeKeyasdf";
add 1.2.3.4 5.6.7.8 esp 18181 -E rijndael-cbc "otherKeyotherKeyOtherKeyOtherKey";

#-------------------------------------------------------------------------------
# Add SP entries:
#-------------------------------------------------------------------------------

# Traffic hosta <-> hostb
spdadd 1.2.3.4 5.6.7.8 any -P out ipsec esp/transport//require;
spdadd 5.6.7.8 1.2.3.4 any -P in  ipsec esp/transport//require;

# Everything else may pass unencrypted
spdadd 0.0.0.0/0 0.0.0.0/0 any -P out none;
spdadd 0.0.0.0/0 0.0.0.0/0 any -P in  none;

#-------------------------------------------------------------------------------
# End of /etc/ipsec.conf
#-------------------------------------------------------------------------------


hostb:
-------------------------------------- 8< --------------------------------------
#-------------------------------------------------------------------------------
# Add SA's
#-------------------------------------------------------------------------------
add 5.6.7.8 1.2.3.4 esp 25252 -E rijndael-cbc "SomeKeySomeKeySomeKeySomeKeyasdf";
add 1.2.3.4 5.6.7.8 esp 18181 -E rijndael-cbc "otherKeyotherKeyOtherKeyOtherKey";

#-------------------------------------------------------------------------------
# Add SP entries:
#-------------------------------------------------------------------------------

# Traffic hostb <-> hosta
spdadd 5.6.7.8 1.2.3.4 any -P out ipsec esp/transport//require;
spdadd 1.2.3.4 5.6.7.8 any -P in  ipsec esp/transport//require;

# Everything else may pass unencrypted
spdadd 0.0.0.0/0 0.0.0.0/0 any -P out none;
spdadd 0.0.0.0/0 0.0.0.0/0 any -P in  none;

#-------------------------------------------------------------------------------
# End of /etc/ipsec.conf
#-------------------------------------------------------------------------------

Thanks in advance
Chris

-- 
----------------------------------------------------------------------
Christoph Kaegi                                           [email protected]
----------------------------------------------------------------------