Re: How does POE speed up operation in reality?

On Sun, May 3, 2009 at 12:17 AM, Roy M. <[email protected]> wrote:
> hello,
>
> On Sun, May 3, 2009 at 2:02 PM, Rocco Caputo <[email protected]> wrote:
>> You should consult the component's
>> author directly if he doesn't respond on the mailing list.
>>
>
> you are right, I will send to author off the list then.
>
>
>> I don't understand the question.  Do you mean to imply that an event-driven
>> program cannot consume data from multiple producers?
>>
>
> I think at least if you want to access blocking method in the main
> event loop, then thread is needed.
>
> e.g. in python twisted, they have threadpool
> http://twistedmatrix.com/projects/core/documentation/howto/threading.html
>
>
> Just wonder if have the same thing in POE.
>

My opinion is that you are making this problem harder than it has to be.

I've written solutions to this exact problem of collecting data from a
large number of hosts concurrently using FTP.  The trivially easy way
to do it is to manage a moderate number of single host transfer
processes.  This can be done in Perl as using Parallel::ForkManager or
in POE  using POE::Component::JobQueue  and POE::Wheel::Run to launch
and monitor the FTP sub processes.

Accounting can be handled either by instrumenting the FTP sub
processes to update some score card or by letting the management
process manage the score card itself.

JMHO
cfedde