Strategy in designing a scaleable robot with POE::Component::Client::HTTP?

[email protected] (Ryan Chan)
Newsgroups perl.poe
Message-ID <[email protected]> 
Hello,


Assume I only have a dual core server, with limited 1GB memory , I
want to build a web robot to crawl 1000 pre-defined web sites.

Anyone can provide a basic strategy for my tasks?

Should I  create 1000 sessions at the same time, to archive the max
network throughput?





Thanks.
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.