Re: chunked transfer encoding and HTTP::Daemon

[email protected] (Andreas J. Koenig) writes:

> >>>>> On 07 Dec 2005 10:17:36 -0800, Gisle Aas <[email protected]> said:
> 
>   > [email protected] (Andreas J. Koenig) writes:
>  >> Does anybody have (or can point me to) a working example of
>  >> HTTP::Daemon that implements a server that is capable to handle
>  >> "Transfer-Encoding: chunked"? (I mean really capable, which means it
>  >> should work without a Content-Length header and for inputs that tickle
>  >> in in small chunks.)
> 
>   > The $c->get_request method should handle "Transfer-Encoding: chunked"
>   > just fine, but it will not return until all the content data has been
>   > received.  If you want to process chunked data as it is received, then
>   > you will have to tell it to only read past the headers with
>   > $c->get_request(1) and then do the parsing of the chunked body
>   > yourself.
> 
> Thanks for coming back to my question. In the meantime I have verified
> that $c->get_request() does it right. I append a testscript for that;
> hope you like it.

I did't find anything appended.

> What I had problems to get to work was the $c->get_request(1) + "do
> the parsing yourself" variant. It would be a great help if there were
> a sample implementation (maybe it would fit into my testscript?) that
> is more or less bullet proof.

Inside the get_request method you will find a block that goes like this:

    if ($te && lc($te) eq 'chunked') {
        # Handle chunked transfer encoding
        my $body = "";
      CHUNK:
        while (1) {
            print STDERR "Chunked\n" if $DEBUG;
            if ($buf =~ s/^([^\012]*)\012//) {
                my $chunk_head = $1;
                unless ($chunk_head =~ /^([0-9A-Fa-f]+)/) {
                    $self->send_error(400);
                    $self->reason("Bad chunk header $chunk_head");
                    return;
                }
                my $size = hex($1);
                last CHUNK if $size == 0;

                my $missing = $size - length($buf) + 2; # 2=CRLF at chunk end
                # must read until we have a complete chunk
                while ($missing > 0) {
                    print STDERR "Need $missing more bytes\n" if $DEBUG;
                    my $n = $self->_need_more($buf, $timeout, $fdset);
                    return unless $n;
                    $missing -= $n;
                }
                $body .= substr($buf, 0, $size);
                substr($buf, 0, $size+2) = '';

            }
            else {
                # need more data in order to have a complete chunk header
                return unless $self->_need_more($buf, $timeout, $fdset);
            }
        }
        $r->content($body);

        # pretend it was a normal entity body
        $r->remove_header('Transfer-Encoding');
        $r->header('Content-Length', length($body));

        my($key, $val);
      FOOTER:
        while (1) {
            if ($buf !~ /\012/) {
                # need at least one line to look at
                return unless $self->_need_more($buf, $timeout, $fdset);
            }
            else {
                $buf =~ s/^([^\012]*)\012//;
                $_ = $1;
                s/\015$//;
                if (/^([\w\-]+)\s*:\s*(.*)/) {
                    $r->push_header($key, $val) if $key;
                    ($key, $val) = ($1, $2);
                }
                elsif (/^\s+(.*)/) {
                    $val .= " $1";
                }
                elsif (!length) {
                    last FOOTER;
                }
                else {
                    $self->reason("Bad footer syntax");
                    return;
                }
            }
        }
        $r->push_header($key, $val) if $key;

    }

This is basically what you need to do :)

--Gisle