Re: [Patch 01/12] CLD: fix crash in retransmissions

Jeff Garzik <[email protected]> Sun, 18 Apr 2010 23:46:07 -0400
Newsgroups org.kernel.vger.hail-devel
Message-ID <[email protected]>
On 04/18/2010 12:37 AM, Pete Zaitcev wrote:
> For a longest time I was plagued by (very infrequent) crashes like this:
>
> Program received signal SIGSEGV, Segmentation fault.
> sess_retry_output (timer=0x92070c0) at session.c:532
> 532                     if (!next_retry || (op->next_retry<  next_retry))
> (gdb) info threads
> * 1 Thread 0xb72f96c0 (LWP 22417)  sess_retry_output (timer=0x92070c0) at session.c:532
> (gdb) where
> #0  sess_retry_output (timer=0x92070c0) at session.c:532
> #1  session_retry (timer=0x92070c0) at session.c:565
> #2  0x08049aee in cld_timers_run (tlist=0x8056630) at ../lib/libtimer.c:95
> #3  0x0804e9cc in main_loop (argc=5, argv=0xbff70bd4) at server.c:983
> #4  main (argc=5, argv=0xbff70bd4) at server.c:1138
>
> The crash happens because op is NULL. As it turned out, this happens
> if a packet retransmit and a session expiration occur simultaneously
> (in the same pass of timers_run). The scenario is:
>   - timers_run collects expired timers at exec list
>   - timers_run expires session
>   - two timer_del are called, but one of them is on exec list already,
>     so it's ineffective
>   - session is freed, this zeroes ->data in lists (later op)
>   - timers_run continues along the exec list, invokes the retransmission
>     callback, and that crashes with NULL op.
>
> The proposed solution is to rework the timers_run, again. But this
> time, we'll make it simpler by observing that timers are ordered by
> expiration time. Therefore, we can pull next timer off the list,
> expire it, and loop until expiration time is greater than the current
> time. No execution list is kept. The integrity of the main list
> is assured by never walking it and always referring to the head
> anew at each iteration.
>
> This patch appears to fix the problem and stands up to use that
> crashed the old code.
>
> Signed-off-by: Pete Zaitcev<[email protected]>
>
> ---
>   include/cld_common.h |   10 ++++++----
>   lib/libtimer.c       |   41 ++++++++++++++++++++---------------------
>   2 files changed, 26 insertions(+), 25 deletions(-)

applied 1-4

Note that I change the email subject line prefix (which is normally 
copied by automated tools directly into git) from "CLD" to "libcldc", 
when committing to git.  I am trying to follow (and encourage others to) 
the kernel's method of using a prefix to indicate the subsystem or 
section within the current git repo to which a change applies.

To be fully friendly with automated tools, an ideal Project Hail subject 
line might read

	[cld patch 1/1] update Makefile.am
	 	or
	[tabled patch 1/1] update Makefile.am
		or
	[cld patch 1/1] libcldc: add nncld API, the new new CLD API

Not a big deal, just noting what is most friendly to the git automated 
tools, when I'm importing each Hail patch.

	Jeff