Re: kill a socket

Gerd Stolpmann wrote:
> Am Montag, den 03.03.2008, 11:08 +0100 schrieb Peter Halacsy:
>> Hello,
>>
>> I use httpclient to create something similar to the example called 
>> spider (downloading a lot of urls)
>>
>> I've three problems
>>
>> 1. if I use a pipeline with a length of 1500, I got Segmentation Fault . 
>> With max 1000 connection everything works fine
> 
> The segfault is strange. Note that there's a limit of usually 1024
> descriptors per process, but that shouldn't crash if it is reached.
> 


Do you know how I can set this number?
>> 2. At the end  of running I see two connections not closed by the 
>> library (I use Http_client.connection_timeout = 5.0). There is no way to 
>> kill the connections.
> 
> Do you use aggressive connection caching? This is known not to work
> properly in all cases.

No. I use default caching.
> 
>> 3. Is there any way to ask httpclient not to download pages larger then 
>> a given limit?
> 
> You can do that with set_response_body_storage and providing a custom
> out_obj_channel that receives the body. You can assume that the response
> header is available when the first byte of the body arrived. To stop the
> download, raise Bad_message from one of the out_obj_channel methods.
> 
> Gerd


Thanks, I'll try this. But I checked: i could use the content-length 
header. Does every HTTP 1.1 server send this header attribute?

peter


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/