Re: benchmarking various event loops with and without anyevent

On Apr 27, 2008, at 00:56, Marc Lehmann wrote:
> On Sat, Apr 26, 2008 at 08:13:27AM -0400, Rocco Caputo =
<[email protected]=20
> > wrote:
>> each event watcher.  Anyone who knows POE can tell you this is one of
>> the least efficient designs possible.
>
> It is the only design that I could get working, even after =20
> consulting a
> few people and implementing some workarounds for the bugs in POE.
>
> In any case, you have to consider that most people on this planet
> don't know POE, and even if, they don't know it that well. Since the
> documentation for POE is in such a bad state, thats the obvious way =20=

> to fix
> that.

Most people on the planet don't know Perl, or even how to program a =20
computer.  No amount of documentation will help them.  :)

In the future, you may wish to include me on your list of people to =20
consult about designing applications for POE.  I may known a thing or =20=

two about the topic. :)

Are you aware that I'm gradually rewriting POE's documentation?  If =20
you could describe what you don't like in a useful way, I may be able =20=

to do something about it.

>> In fact, this design is worse than the average for first-time POE =20
>> users.
>
> If a better design is possible, it is not known to me, and you haven't
> suggested one either, so talk is cheap. I'd be happy to get a more
> efficient design for POE but nobody could come up with one that also
> worked reliably through multiple iterations of run and also does not =20=

> keep
> the POE kernel from returning.

I would love to have the opportunity to suggest a different design, =20
but most of my discretionary time is spent addressing the constant =20
misunderstandings between us.  If we can first resolve them, I'll have =20=

that much more time to work on the design.

Obviously I cannot expect you to know everything about POE.  Likewise, =20=

you cannot expect me to magically know when you started writing =20
AnyEvent::Impl::POE.  Even if you announced it somewhere, I may not =20
have been looking.  The first I heard of it was here, when you =20
announced your benchmarks.

In general, if you need someone's attention online, the most effective =20=

and polite way is to contact them directly.  If you did contact me =20
about this, then I'm sorry that I missed it.  Are you sure your =20
message wasn't lost in transit?

As for the design:

First-time POE users tend to design programs where the number of =20
sessions scales linearly with the number of objects that handle =20
events.  If S(1) is the total overhead imposed by a single session, =20
then S(N) is the overhead imposed by the average na=EFve POE user.  N is =
=20
the number of objects handling events.

AnyEvent::Impl::POE creates a new POE::Session for every event =20
watcher.  It's not uncommon for an object to use more than one event =20
watcher (I/O and timeout, for example).  So we can model the session =20
overhead in an AnyEvent::Impl::POE program as S(N*M), where N is the =20
number of objects handling events, and M is the average number of =20
event watchers per object.

Assuming that N is the same between the equivalent POE and =20
AnyEvent::Impl::POE program:

S(N*M) > S(N) for M > 1.

QED  :P

> If a better design *is* possible (which I don't really doubt), then it
> needs to be vastly more complex, or it needs some non-obvious trick. =20=

> I can
> only imagine making some very complex on-demand instantiating and re-=20=

> check
> wether the session still exists on each watcher creation.

Your imagination comes up with such incredible things.  Don't lose =20
that. :)

The "trick" is to minimize the number of sessions used.  Your =20
benchmarks and comments in your documentation implied that you knew =20
that sessions imposed overhead.  I wrongly assumed the solution would =20=

be obvious.

In your case, I would create a single persistent POE::Session instance =20=

that serviced all the watchers.  The watchers themselves would be =20
small proxies that controlled POE::Kernel watchers within that =20
session's context.

I use this design in POE::Stage.  If you're not averse to looking at =20
experimental code, you can find it on the CPAN.  It also does a lot of =20=

other, unrelated things, so you may have difficulty separating the =20
magic you need from the voodoo you don't.  Comments are welcome, if =20
they're useful.

> (It is possible that I was fooled by the docs as well, so if there =20
> is a
> better way, it likely isn't documented, so who could blame me).

I won't blame you.  But I will point out that your documentation says =20=

you're already familiar with using undocumented POE features. :)

> Instead of going around and accuse people of making bad designs, it =20=

> would
> be much better to improve the documentation for POE, so that said =20
> people
> don't have to go around and start guessing...

I don't appreciate your vague, negative comments about the =20
documentation.  Please describe problems in a useful way, or your =20
expectation that they be fixed is unreasonable.

You seem to be saying that your design for AnyEvent::Impl::POE is =20
based on guesswork.  If so, I overestimated your knowledge of POE =20
based on your documentation.  Therefore, please consider the chance =20
that you may know less about POE than your documentation implies.

> Oh, and while you fix the docs, could you fix the other bugs as well

I certainly can, but you'll need to describe them in a useful way =20
first.  http://search.yahoo.com/search?p=3Duseful+bug+reports seems to =20=

link to good tips.

And please post them to POE's rt.cpan.org queue.  The Perl community =20
has been kind enough to establish a convenient, central place to =20
report problems with CPAN distributions.  It would be most cooperative =20=

of you to use it.

> (the race condition with sigchld is really annoying,


Please describe the race condition in a useful way, so that I can =20
investigate and possibly fix it.

POE checks for child processes between calls to application code.  If =20=

you fork your process and set sig_child() in the same event handler, =20
POE won't be able to call waitpid() between them.  If this is the race =20=

condition you mean, then it simply shouldn't be possible.

On the other hand, I haven't actually documented this behavior.  It =20
hadn't occurred to me because all my programs follow the reliable =20
pattern.  Based on your feedback, I added this to the description of =20
sig_child().  Please let me know how/if I can improve it:

"Programs that wish to reliably reap child processes should be sure to =20=

call sig_child() before returning from the event handler that forked =20
the process.  Otherwise POE::Kernel may have an opportunity to call =20
waitpid() before an appropriate event watcher has been registered."

In the event that you've found an actual bug, I regret that your =20
description of the problem is not useful.  As we say in IRC:

<purl> Look buddy, doesn't work is a strong statement. Does it sit on =20=

the couch all day? Is it making faces at you? Does it want more money? =20=

Is it sleeping with your girlfriend? Please be specific!

> and the nag messages are as well).

I understand from your documentation that you dislike the "ugly" "nag" =20=

messages.  I'm sorry, but I cannot help you there.  They were added as =20=

the result of useful feedback from other users.  And contrary to your =20=

documentation, they can be silenced.

The "run() method was never called" message can be silenced by calling =20=

run().  Not all applications require run().  But if you call it =20
without any POE::Session instances, it will return immediately and =20
satisfy POE::Kernel.

For example, this generates the warning:

perl -wle 'use POE; POE::Session->create(inline_states =3D> { _start =3D> =
=20
sub{} });'

While this does not:

perl -wle 'use POE; POE::Kernel->run; POE::Session-=20
 >create(inline_states =3D> { _start =3D> sub{} });'

I hadn't considered to document the technique because it seemed =20
obvious to me.  By now I hope it's clear that a lot more things seem =20
obvious to me than are actually obvious to the average user.  You =20
should probably call them to my attention if you'd like them to be =20
documented.

Speaking of which, I added this to the docs.  Please let me know how/=20
if I can improve it:

"POE::Kernel will print a strong message if a program creates sessions =20=

but fails to call run().  If the lack of a run() call is deliberate, =20
you can avoid the message by calling it before creating a session.  =20
run() at that point will return immediately, and POE::Kernel will be =20
satisfied."

...

You complain about a message that occurs when a program leaks a child =20=

process.  Programs that leak processes are broken in a way that is =20
difficult to detect without the warning.  In extreme cases, process =20
leakage can quietly strangle a machine to death.

POE can detect this silent but deadly condition.  It notifies users =20
out of courtesy to them, and also because I was tired of handling "POE =20=

crashed my machine!" reports.  In this case, the one who smelled it =20
did not deal it.

If you receive this message, it can be silenced by fixing the process =20=

leak.

>> I have specific issues with your docs.  As a courtesy to you and the
>> list, I'll send them to you directly.
>
> Not sure why it would be a courtesy to me, do you have anything to =20
> hide
> or do you want to insult me even more in private so nobody sees your =20=

> real
> self or something? (just guessing... :)

Wow, you really do expect the worst from me.  I don't know where that =20=

came from.

The courtesy to you would be that I brought the issues to you =20
directly.  I could post them somewhere public and unexpected and wait =20=

for you to find them, but that's just rude.

Likewise, posting long, limited-interest e-mail to a public =20
distribution list is a violation of common etiquette.  To those who =20
are still reading, I'm sorry.

I kind of thought those points would be obvious.

If you still insist on doing this publicly, please indicate your =20
intent by posting your documentation for AnyEvent::Impl::POE to the =20
mailing list (new thread, please).  That will make it easier for me to =20=

respond.  For the record, I still think this is a bad idea.

> I do fix my bugs and am open to suggestions and improvements. But
> correctness comes first, performance second.


OMG, we agree on something!  The end times are here:

--=20
Rocco Caputo - [email protected]=