Re: track_file function ?

Damien,
I've looked at this a bit further, and I don't think adding a delay
will resolve the problem. At around line 927 of
keepalived/core/keepalived_netlink.c there is an explanation of what
appears to be happening:                                /* It appears
the if we add a large number of interfaces,
then                                 * within a few seconds of starting
up the kernel can decide to                                 * notify us
again of all the new interfaces (or it may have
something                                 * to do with becoming
master). In any event, if we add a                                 *
kernel_netlink_poll() at every stage of the process, we
suddenly                                 * go from having nothing to
receive on nl_kernel to ENOBUFS. Since
it                                 * relates to the interfaces, and we
have already got the information                                 *
about the interfaces, it appears that we aren't losing any useful info.
*/
I need to change this to:                                /* It appears
the if we add a large number of interfaces,
then                                 * while adding the interfaces the
kernel can decide to suddenly                                 * notify
us again of all the new interfaces, and this notification
causes                                 * use to receive ENOBUFS on the
nl_kernel socket. Since it                                 * relates to
the interfaces, and we have already got the
information                                 * about the interfaces, it
appears that we aren't losing any useful information. */
If you run:ip monitor link address routewhile keepalived is starting
up, you will see in the output of that, and the same point as
keepalived logs the Receive message overrun that ip monitor shows
netlink messages for all the interfaces we have created so far (it is
interesting that it is the interfaces that keepalived had created
except the first one, and not the interfaces that had been created
before keepalived started). It is almost as though it is an issue with
a circular buffer wrapping around and it then sends the whole buffer
contents again.
I'm not sure yet how ip monitor manages to avoid a buffer overrun but I
will investigate that further.
Quentin
On Thu, 2018-02-01 at 14:36 +0000, Damien Clabaut wrote:
> Hello again Quentin,
> 
> So far this is the only issue I have with latest beta:
> Feb  1 15:32:56 bz-perflab-6-vr Keepalived_vrrp[10486]: (4137):
> Success creating VMAC interface vrrp4.137.1
> Feb  1 15:32:56 bz-perflab-6-vr Keepalived_vrrp[10486]: Netlink:
> Received message overrun - (No buffer space available)
> 
> This results in:
> +----------+-------------+-----------------------------+---------+---
> -------+-------------+---------+----------------------------+
> > Instance |  Interface  |          Addresses          | Version |
> > Priority | Master prio |  State  |      Last Transition       |
> 
> +----------+-------------+-----------------------------+---------+---
> -------+-------------+---------+----------------------------+
> >   4137   | vrrp4.137.1
> > |         10.0.137.254        |    2    |   100    |      0      |
> > Unknown | 2018-02-01 15:32:57.263674 |
> 
> Is there any chance we can set a very short delay (few milliseconds)
> between each interface creation ?
> 
> Regards,
> 
> On Wed, 2018-01-31 at 18:37 +0000, Damien Clabaut wrote:
> > Hello Quentin,
> > For the record, we compiled the beta branch this morning and tested
> > Keepalived with 398 instances (half version 2, IPv4, the rest
> > version 3, IPv6).
> > For that we used the templates provided in tools/json_tracking on
> > the beta branch.
> > Everything is fine for now, though we do see Netlink Buffer Overrun
> > errors once in a while.
> > Regards,
> > 
> > On Tue, 2018-01-16 at 15:18 +0000, Damien Clabaut wrote:
> > > Hello Quentin and thank you for your answer,
> > > I packaged the beta branch and started the deployment in our lab.
> > > We will do extensive testing in the following weeks so we can put
> > > it in
> > > production once it is merged to Master on your side.
> > > I tested the track_file function a while back, and it worked
> > > fine, even
> > > with hundreds of VRRP interfaces.
> > > Regards,
> > > 
> > > On Tue, 2018-01-16 at 10:05 +0000, Quentin Armitage wrote:
> > > > On Mon, 2018-01-15 at 20:23 +0000, Damien Clabaut wrote:
> > > > > Hello,
> > > > > Do you know when the track_file function will leave beta to
> > > > > neter
> > > > > the
> > > > > main Keepalived branch ?
> > > > > Regards,
> > > > > 
> > > > 
> > > > There is no plan to backport any of the functionality from the
> > > > beta
> > > > branch into the master branch. Rather, at some point, and I
> > > > believe
> > > > this is currently intended to be February, the beta branch will
> > > > be
> > > > merged into the master branch and hence become master.
> > > > 
> > > > More feedback on testing the beta branch would be very welcome
> > > > -
> > > > reported at https://github.com/acassen/keepalived/issues/686 .
> > > > So far
> > > > only one person has provided any feedback. Positive feedback,
> > > > as well
> > > > as negative feedback would be most welcome; it's very helpful
> > > > to know
> > > > that if it is being successfully used, as well as any problems.
> > > > 
> > > > Quentin Armitage
> > > > -------------------------------------------------------------
> > > > ------
> > > > -----------
> > > > Check out the vibrant tech community on one of the world's most
> > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > > > _______________________________________________
> > > > Keepalived-devel mailing list
> > > > [email protected]
> > > > https://lists.sourceforge.net/lists/listinfo/keepalived-devel

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
Keepalived-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/keepalived-devel