Miscellaneous questions on OCamlNet + news

Hi,

In developing the alternative interface I proposed, I stumbled over
the following questions/suggestions/wishlist written as I go:

- Netmime.storage : it would be nice that if the user decided for
  `Memory but the data is too large to fit in a string, `File is
  automatically chosen instead of blowing up.  From scanning Netmime,
  a strategy could be that if the Netbuffer blows up, one copies its
  content to a file and continues with it -- I haven't really thought
  of the messy details though (not duplicating nor losing data).

- The FCGI Sys.max_string_length POST limitation is claimed to be
  corrected.  However, in netcgi_fcgi.ml, the function
  [fcgi_read_stream] accumulates the input stream into a buffer.  (I
  probably missed something.)

- Mimestring.read_multipart_body could also be useful in the form of
  "fold" (left and right): e.g.

  Mimestring.fold_left_multipart_body :
    ('a -> Netstream.in_obj_stream -> 'a) -> 'a ->
    string -> Netstream.in_obj_stream -> 'a

- When the request method is not allowed or when it is unknown, the
  library raises Failure.  This is not a correct behavior however,
  instead a reply with the appropriate status (e.g. `Not_implemented)
  should be sent (and this should not be the task of the user: exn
  launched by the lib must be handled by it if it can).

  The way I solved this is to define an internal HTTP exception with a
  status code and an error message (see Netcgi_common).  This
  exception is caught by connectors and appropriate messages are sent
  back to the client.  Note that your exn [Resources_exceeded] is now
  a particular case of this (`Request_entity_too_large).

- It does not seem that #finalize uses reference counting on the
  files.  Concretely, it means that a MIME body can only be shared
  between different MIME messages (as they will if one "clones" an
  arg) with great care.

  Also, since in the new interface all handlers are callbacks, what
  about automatically finalizing the arguments when the callback
  returns (normally or otherwise)?

- The Netmime.mime_header_ro defines

    method content_type : unit -> string * (string * Mimestring.s_param) list

  while Netcgi_env.cgi_environment says

    method input_content_type : (string * (string * Mimestring.s_param) list)

  It would be nice for the two to be unified.  The same goes for
  [content_length].

- It would be nice to be able to turn a mime_header read-only.
  Indeed, it is good to have it rw to fill it and then lock it ro.  Or
  maybe a solution alike [Netmime.storage ~ro:true] can be envisioned.
  A simple solution is to make [mime_header] a class that takes a
  argument of type [mime_header_ro] -- it will still define a type so
  it does not influence the rest of the code (but appropriate doc
  should be provided).

- One of the arguments for separating connectors in their own module
  (which I agree with), was to reduce the code size.

  [Mimestring], [Netmime], [Nethttp] and [Neturl] call [Netencoding]
  for its [Base64], [QuotedPrintable], [Q] and [Url] modules.  However
  [Netencoding] also has a [Html] module (not used for cgi/) which
  links with [Netconversion], one of the bigger modules!  While I
  agree it is the right place for [Html], this side effect is a bit
  unfortunate.  Maybe a splitting [Netencoding_base64],... would be
  nice.

- It is not clear to me whether

    let net_db_dir = "/usr/local/lib/ocaml/3.08.3/netstring" ;;

  in Netdb is correctly set up at compile time.


I'll appreciate comments on these.

About the new interface, you can browse the interface at
<http://ocaml-cgi.sourceforge.net/netcgi/>.  Notice that there is a
Netcgi_compat module (implemented) which allows to convert between the
new and old objects.  The documentation of this module also gather the
motivation for the changes.

I am also happy to announce that I have started the implementation of
the other modules (they depend on netstring)
<http://cvs.sourceforge.net/viewcvs.py/ocaml-cgi/ocamlnet/netcgi/>.
Highlights are:

- Cookie module that supports Netscape, RFC 2109, and RFC 2965 cookies
  both for sending and receiving (including quoted values).  Care has
  been taken send cookies so that old browsers understand them (this
  has not been tested however).

  Remark: Cookies are just strings, contrarily to an earlier proposal.

- Correct responses to several errors by way of the [HTTP] exception
  and handlers.

  There is a default handler for user generated exceptions but it is
  possible to override it.

- All exceptions are acted upon correctly -- the user does not have to
  write code to handle them (although he can if he wishes to).

- One can decide to drop file arguments one knows we will not use --
  save space.  See Netcgi.arg_store.

- Arguments are finalized on normal termination of the script or on
  error.

- Netcgi_cgi and Netcgi_test: complete implementation with the new tools.

- Netcgi_mod: "mod_caml" connector (complete).  The necessary subset
  of mod_caml is included in the sources for convenience (but one may
  as well use the standard mod_caml.so).

- Netcgi_fcgi:

  * An input object is created to handle incoming data without size
    restrictions.

  * out_obj_channel now outputs correctly empty strings or strings
    longer than 0xFFFF bytes.  Output is padded to reach a multiple of
    8 bytes for performance.  Also, closing the output channel does
    not close the error one -- they are independent.

  * All FCGI roles (Responder, Authorizer, Filter) are supported.

  * Management records are handled appropriately.

  The module has not been tested so it is likely to still contain
  minor errors.  I am quite busy now so I'll do that in 3 weeks or so.

- Netcgi_ajp: AJP 1.3 is planned but I have no time to do it in the
  coming weeks (shouldn't be that difficult).

- The "run" functions has been kept for lack of a better name
  (suggestions welcome).  Netcgi_fcgi and Netcgi_ajp are ready to
  handle concurrent connections once a good model is found (for the
  other connectors, the script has no control about its concurrent
  behavior).

- Netcgi_common shelter the main development functions while Netcgi is
  the user interface.  I hope you like the design.

Misc.

- Drop in replacement of Url: 10 * faster and never throws exceptions
  (always decode).

- Some examples have been ported to the new interface.

As usual, all remarks are welcome.

Best regards,
ChriS


---
P.S.  It would be nice if the lines in *.mli files were not wider than
the usual 80 chars, for compatibility with all screen layouts and
because long lines are tiresome to read.  (Emacs M-q is a great help
for that.)


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf