Re: Reconsidering libevent

On 09/21/2010 01:51 PM, Jeff Garzik wrote:
> Today's Project Hail hacking involves moving tcp_write_* code from
> itd.git to libhail.  tcp_write_* is the well-debugged TCP async write
> queueing code from tabled, with cosmetic changes to provide a nice API
> encapsulation.  I did this when moving the code originally from tabled
> ->  itd.  By moving this code into libhail, with (yet more) cosmetic
> changes, the same TCP write queueing code may be used in both tabled
> and itd pkgs.  chunkd'd async TCP write code is highly similar, and
> can be merged into libhail as well.
>
> (if you're curious why I'm doing this, it's one step in the road to
> updating itd to use chunkd's GET_PART)
>
> An interesting side effect:  both tabled and itd use libevent, which
> means that libhail must now link with libevent.  Is this a bad thing?
> Recently libevent was purged from cld (a simple use case, where poll
> sufficed) and chunkd (obscure concerns about multi-threading), but
> linking libevent into libhail is causing me to rethink the matter.
>
> My concerns about chunkd, libevent and M-T centered around libevent's
> use of a single, mutex-unprotected event_base into which events are
> queued etc.  In one particular thread model, the set of listened-to
> events might be manipulated from multiple threads concurrently,
> thereby implying a commitment to libevent implies a commitment to
> libevent wrappers which add locks.
>
> However, it turns out that thread model is quite suboptimal compared
> to other methods.  Assigning an incoming TCP connection to a thread,
> and having that thread manage that connection for its entire lifetime,
> is both much more efficient and quite compatible with libevent's API.
> One merely assigns an event_base to each thread.  In Apache-land, this
> is called the "prefork" model (yes, I know, fork != thread; I didn't
> invent the terminology)
>
> So, I am reluctantly considering moving chunkd back to libevent.  That
> will enable chunkd, tabled and itd to all share the same async TCP
> write code, and as a nice side effect, reduce our minor reinvention of
> libevent (c.f. libtimer and some of the network poll code).
>
> Comments welcome, as always.
>
>        Jeff
> --
> To unsubscribe from this list: send the line "unsubscribe hail-devel" in
> the body of a message to [email protected]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

libevent version 2 has proper mutual exclusion, but the code needs some 
work.

regards
-steve