AW: Re: Patches for glibc

Hi Oliver,

looks like there'S still a problem with the patches: as reported yesterday, after a few hours uptime I start getting lots of error messages in cdbd_log on all nodes:

Wed Apr 24 13:42:06.650 webc2 cdbd  - fs2d new connection setup failed

This seems to be triggered by fs2d_locate_process, or, to be more exact, the condition 

        if (fd >= 0 &&
            fd < FD_SETSIZE) {

round line 70; checking the list of open filehandles for the cdbd process shows > 1024 used descriptors after some time, in contrast to about 60 right after startup.

So, it looks like cdbd is leaking filedescriptors somewhere, and starting to be fail when the number of open descriptors hits 1024 (default value of FD_SETSIZE, I believe).

here's what lsof shows for the cdbd process:

stage 1: lots of processes loke these (obviously they must've been normal, established connections before..)
  cdbd 16549 root 64u IPv4 190499416 TCP localhost.localdomain:614->localhost.localdomain:8423 (CLOSE_WAIT)
stage 2: these turn into
  cdbd 16549 root 64u sock  0,0 190499416 can't identify protocol

entries which hang around indefinitely and increase in number at a constant rate.

Any idea if this could actually be connected with your patch?

Thanks, Martin