Re: Seeking advice for a couple of issues

Am 29.03.2013 01:11:39 schrieb(en) John Derrick:
> Thanks for the reply, Gerd.
> 
> First, I should mention I'm using Ocaml 4.001, the Mingw build on  
> Win7.  I'm
> using Ocamlnet version 3.6.1 (or newer).
> 
> I tried using ~enable_auth_in_advance as you suggested but was unable  
> to get
> it working.  I must be doing something wrong.
> Here is my latest attempt:

Confirmed, this does not work. I had this wrong in my mind -  
auth_in_advance does not work when the server never generates a  
challenge.

I've added a new mode skip_challenge, e.g.

let auth = new Http_client.basic_auth_handler ~skip_challenge:(Some  
"*") keys

This is set to "*" or to the domain URL (e.g. "http://localhost/") to  
which it applies.

> method private do_run_pipe call =
>   let keys = new Http_client.key_ring() in
>   let auth = new Http_client.basic_auth_handler  
> ~enable_auth_in_advance:true
> keys in
>   let key = Http_client.key ~user:"admin" ~password:"admin" ~realm:""
>     ~domain:["localhost"] in

This is also a problem in the current code: you would have to set  
domain to "http://localhost:80/". Not really well documented. The new  
version also allows now "*", and you can omit the port number.

That the domain is a full URL has to do with the definitions in the  
HTTP protocol.

>   keys#add_key key;
>   let pipeline = new Http_client.pipeline in
>   pipeline # add_auth_handler auth;
>   let hdr = call#request_header `Base in
>   hdr#update_field "Content-type" "application/json";
>   pipeline#add call;
>   pipeline#run();
>   call#response_status_code
> 
> Packet capture shows that no Authorization header was added to a PUT  
> request
> with the above block of code.  Return code is 401.
> 
> On the second issue, I have TCP_NODELAY set to true in the CouchDb  
> config.
> The process is using heavy CPU while it runs.  Do you think the issue  
> could
> be related to OS and/or OCaml build?

Turns out this is in deed a problem with event handling, namely that  
there was some progress counting, and this resulted in a performance  
bug when the chain of event handling gets long.

The new version is not yet released, but you can grab it from the svn  
server: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/

Gerd


> Thanks again for your help,
> John
> 
> -----Original Message-----
> From: Gerd Stolpmann
> Sent: Thursday, March 28, 2013 9:06 AM
> To: John Derrick
> Cc: [email protected]
> Subject: AW: [Ocamlnet-devel] Seeking advice for a couple of issues
> 
> Am 27.03.2013 23:41:36 schrieb(en) John Derrick:
> > Hi All,
> >
> > I've been working with Http_client to develop a no-frills CouchDb
> > driver. I'm relatively new to OCaml, but have many years of  
> experience
> > with various imperative langs.
> >
> > I'm looking for help with a couple of issues. My first issue is  
> Basic
> > Authentication. I've noticed that Basic Auth and Basic Auth Session
> > do not include the "Authorization:" header when making GET or PUT
> > requests UNLESS a challenge is received. Is this the expected
> > behavior? Not a serious issue as it's easy to add my own
> > Authorization entry to the header. Apparently CouchDb does not  
> send  an
> > auth challenge.
> 
> It is in deed possible to enable this behavior (it's off by default):
> 
>   - Create the authentication handler like this:
> 
>     let keys = new Http_client.key_ring()
>     let auth = new Http_client.basic_auth_handler
>                   ~enable_auth_in_advance:true keys
> 
>     The enable_auth_in_advance option makes it working.
> 
>   - Add credentials:
> 
>     let key = Http_client.key ...
>     keys # add_key key
> 
>   - Add this to the pipeline:
> 
>     pipeline # add_auth_handler auth
> 
> Well, this behavior has always been a matter of discussion. Normally,
> you don't want to send credentials before you know that the server is
> really the server (remember that you can run http over ssl, and the  
> ssl
> connection authenticates the server), and this way it is specified in
> RFC 2617. Historically, though, browsers have always implemented some
> form of sending the credentials without prior challenge, namely when
> the web page was visited the second time.
> 
> > The second issue is chunked request performance. When peforming a
> > query against CouchDb (say _all_docs for example) via a GET request,
> > Ocamlnet seems to have issues processing the returned data. CouchDb
> > returns many small blocks of data in chunked format. Processing a
> > collection of 1000 docs can take several minutes. I'm not sure if
> > this relates to the event system performance or socket performance.
> > Does any one know of a solution to this issue?
> 
> I don't think it has to do with the event processing, as the OS  
> buffers
> data up if the client is not fast enough, so for the next read() the
> client gets many chunks at once. Also, 1000 events (if they are really
> generated) are not really much, and should not take minutes, but only
> milliseconds.
> 
> Does the client consume a lot of CPU? Or is it just waiting?
> 
> In theory there could be a bad interaction with the TCP buffering,
> especially when CouchDB disables the Nagle algorithm (TCP_NODELAY),
> which it probably does. But you report a really bad behavior, so I've
> actually no clue here.
> 
> Gerd
> 
> 
> >
> > Thanks to all who contributed to the development of Ocamlnet. It's  
> an
> > impressive collection of work.
> >
> > Regards,
> > John
> >
> >
> >
> > -------------------------------------------------
> >
> >
> >
> >
> >
> > VFEmail.net - http://www.vfemail.net
> >
> >
> > $14.95 ONETIME Lifetime accounts with Privacy Features!
> >
> >
> > 15GB disk! No bandwidth quotas!
> >
> >
> > Commercial and Bulk Mail Options!
> >
> >
> >
> 
> ------Zitierte Anlage------
> >  
> ------------------------------------------------------------------------------
> > Own the Future-Intel® Level Up Game Demo Contest 2013
> > Rise to greatness in Intel's independent game demo contest.
> > Compete for recognition, cash, and the chance to get your game
> > on Steam. $5K grand prize plus 10 genre and skill prizes.
> > Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d
> 
> ------Zitierte Anlage------
> > _______________________________________________
> > Ocamlnet-devel mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> >
> 
> 
> 
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    [email protected]
> Creator of GODI and camlcity.org.
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> ------------------------------------------------------------
> 
> 
> -------------------------------------------------
> 
> VFEmail.net - http://www.vfemail.net
> $14.95 ONETIME Lifetime accounts with Privacy Features!
> 15GB disk! No bandwidth quotas!
> Commercial and Bulk Mail Options!
> 
> ------------------------------------------------------------------------------
> Own the Future-Intel(R) Level Up Game Demo Contest 2013
> Rise to greatness in Intel's independent game demo contest. Compete
> for recognition, cash, and the chance to get your game on Steam.
> $5K grand prize plus 10 genre and skill prizes. Submit your demo
> by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2
> _______________________________________________
> Ocamlnet-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/ocamlnet-devel
> 
> 



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    [email protected]
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------
------------------------------------------------------------------------------
Own the Future-Intel(R) Level Up Game Demo Contest 2013
Rise to greatness in Intel's independent game demo contest. Compete 
for recognition, cash, and the chance to get your game on Steam. 
$5K grand prize plus 10 genre and skill prizes. Submit your demo 
by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2