Re: reconnect question

Le vendredi 02 mai 2008 à 09:26 -0500, LC Bruzenak a écrit :
> On Fri, 2008-05-02 at 09:16 +0200, Yoann Vandoorselaere wrote:
> > Le mardi 29 avril 2008 à 08:22 -0500, LC Bruzenak a écrit : 
> > > On Tue, 2008-04-29 at 10:33 +0200, Yoann Vandoorselaere wrote:
> > > > Le lundi 28 avril 2008 à 09:34 -0500, LC Bruzenak a écrit : 
> > > > > On Mon, 2008-04-21 at 19:33 +0200, Yoann Vandoorselaere wrote:
> > > > > > Le jeudi 17 avril 2008 à 19:05 -0700, LC Bruzenak a écrit :
> > > > > > > In testing, when the prelude-manager on my collection machine is stopped the
> > > > > > > clients are unable to reconnect on their own.
> > > > > > > I am forced to go restart those services in order to reestablish connection.
> > > > > > > I realize that the collection server should never go away; it should be a
> > > > > > > high-availability type of system.
> > > > > > 
> > > > > > This is what is supposed to happen: once an agent notice the connection
> > > > > > loss, it will wait for ten seconds then attempt a reconnection. 
> > > > > > 
> > > > > > Each time the reconnection attempt fail, the connection timeout is
> > > > > > increased by a factor of two, up to one hour total.

[...]

> > > > > We left the connection disabled for ~23 minutes, after which time your
> > > > > algorithm would be at or near the 1-hour reconnect time.
> > > > > 
> > > > > It gets to a long time rather quickly this way...the first 3 checks in
> > > > > the series happen in < 1 minutes, while the last 3 are at the 21.2
> > > > > minutes, 42.4 minutes and 1 hour mark IIUC.
> > > > > 
> > > > > This is a long time - the reconnect time dominates the downtime.
> > > > > 
> > > > > Is there a reason for the incremental reconnect delay? 
> > > > 
> > > > Not really, we always achieved this behavior, and never got any negative
> > > > feedback concerning it. Anyone out there would also like to see shorter
> > > > reconnection time?
> > > > 
> > > > What about the same incremental delay, with the maximum timeout being
> > > > ten minutes?
> > > 
> > > Perhaps a configurable maximum?
> > 
> > That would be possible, although it could be a pain to implement, since
> > the API is used by a lot of Prelude modules.
> 
> OK.
> 
> > 
> > At first, it would be easier to have nicer default values. Suggestions?
> > 
> 
> Yoann, is there some downside to a faster constant reconnect time?
> From what you said before, there was no real requirement for the
> incremental reconnect delay. I can test here to check the network and
> the machine's behavior during reconnect attempts, but in your opinion is
> there some technical reason to not just keep trying once every minute?
> 
> Or did it just seem illogical to just keep sending connection requests?


I can see two downside:

- Your log file size will increase, with a lot of reconnection attempt.

- The connect() time in case the peer is ignoring TCP packets on the
Prelude port [although this should not happen] could be very long. This
could be a problem in case the Prelude client does not use asynchronous
timer, because the data analysis will be blocked until the connect call
expire.

These look like minors issues to me, but that might be more problematic
to other users?

> I do have a requirement for the child sensors to reconnect quickly. What
> that means in terms of time is inexact, but more on the order of
> magnitude of a minute or less vice an hour. 
> 
> So for my case, I think I'll just need to ensure that there is another
> parent manager ready and use the failover option. If that is also
> unavailable, once the parent manager is back online I'll need to
> manually restart all the child managers to get them to reconnect faster
> than one hour. This is an acceptable work-around.
> 
> Anything more than a 1-minute reconnect is not going to help me and not
> worth your time if no one else has this requirement.

As a temporary workaround, you might want to edit libprelude
src/prelude-connection-pool.c, and modify the MAXIMUM_EXPIRATION_TIME
definition to fit your needs.

Regards,

-- 
Yoann Vandoorselaere | Responsable R&D / CTO | PreludeIDS Technologies
Tel: +33 (0)8 70 70 21 58                  Fax: +33(0)4 78 42 21 58
http://www.prelude-ids.com

_______________________________________________
Prelude-user site list
[email protected]
http://lists.prelude-ids.org/mailman/listinfo/prelude-user