Re: multi curl working very slow when threads > 10

On Mon, 24 Jan 2011, far Fraser wrote:

> Curl: 7.21.3

Using which resolver backend in libcurl?

> I'm trying simple multi curl script (GET list of URLs in 10-100 threads).

The libcurl multi interface doesn't use threads at all. It does multiple 
transfers in a single thread.

> It does work on any windows, but when I try the same code on CentOS: if 
> threads = 10, everything works almost fine, but when I set threads = 50 or 
> 100 the process is almost stuck, it takes minutes for "multi_curl_exec"

That sounds like you're using the stock resolver, as that is synchronous all 
the other transfers stall during each name resolve. 100 name resolves in 
serial can take a long time. libcurl for windows defaults to build with a 
threader resolver.

-- 

  / daniel.haxx.se
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php