Re: mp2] [mod_proxy] [filter] mod_proxy is not playing with my request filter.

I'm still having issues attempting to modify the body of an http request -
this time using a FilterConnectionHandler.

I've had to park this due to deadlines and write it using LWP in a
PerlResponseHandler. This is far from ideal as some of the responses are
upwards of 5MB, and what I'm trying to do does feel very doable in mod_perl.

What I really want is to be able to do is pass the request off to mod_proxy
(which I assume is better setup for streaming responses through Apache ... I
may be wrong) and then through mod_deflate to compress the results.

I've attached the code and configuration if anybody has a second and could
shed some light over what I'm doing wrong.

I'm basically trying to convert a GET request to 'http://hostname/echo' into
a POST request and attach a payload, however as it hangs mod_proxy becomes
largely academic.

Again any help would be much appreciated.
Thanks, James.


PerlModule Example::Echo
PerlModule Example::ConnectionFilter

PerlInputFilterHandler Example::ConnectionFilter::forward_get_as_post

<Location /echo>
    SetHandler modperl
    PerlResponseHandler Example::Echo
</Location>



package Example::Echo;

use strict;
use warnings;

use Apache2::Const -compile => qw(OK);
use Apache2::RequestIO;
use Apache2::RequestRec;


sub handler
{
    my $r = shift;

    $r->content_type('text/plain');
    $r->read(my $buffer, 1024);
    $r->print("received post data: '".$buffer."'");

    return Apache2::Const::OK;
}


1;



package Example::ConnectionFilter;

use strict;
use warnings;

use base qw/Apache2::Filter/;

use Apache2::Connection;
use Apache2::Const -compile => qw(OK DECLINED);
use Apache2::FilterRec;
use Apache2::Log;
use Apache2::RequestRec;
use Apache2::RequestIO;

use APR::Const -compile => ':common';
use APR::Brigade;
use APR::Bucket;
use APR::BucketType;
use APR::Error;


sub forward_get_as_post :FilterConnectionHandler
{
    my ($f, $bb, $mode, $block, $readbytes) = @_;
    my $ctx = $f->ctx || { 'state' => 'waiting_for_request_line' };

    warn "state = ".$ctx->{'state'}."\r\n";

    # check whether we need to process this request.
    return Apache2::Const::DECLINED if ($ctx->{'state'} eq 'ignore');

    # read into a tmp brigade.
    my $connection = $f->c;
    my $tmp_bb = APR::Brigade->new($connection->pool,
$connection->bucket_alloc);
    my $rv = $f->next->get_brigade($tmp_bb, $mode, $block, $readbytes);

    return $rv unless $rv == APR::Const::SUCCESS;

    while (!$tmp_bb->is_empty)
    {
        # pop buckets from this brigade.
        my $bucket = $tmp_bb->first;
        $bucket->remove();

        if ($ctx->{'state'} eq 'waiting_for_request_line')
        {
            # assumes request line is first bucket.
            $bucket->read(my $request_line);
            my ($method, $uri, $version) = ($request_line =~ m|^(.*?) (.*?)
HTTP/(.*?)\r\n$|);

            if (defined ($method) and $method eq "GET" and $uri =~
m|^/echo|)
            {
                my $new_uri = 'POST '.$uri.' HTTP/'.$version."\r\n";
                my $new_uri_bucket =
APR::Bucket->new($connection->bucket_alloc, $new_uri);

                $bb->insert_tail($new_uri_bucket);

                my $bucket2 = APR::Bucket->new($connection->bucket_alloc,
"Content-Type: application/x-www-form-urlencoded\r\n");
                $bb->insert_tail($bucket2);

                my $bucket3 = APR::Bucket->new($connection->bucket_alloc,
"Content-Length: 9\r\n");
                $bb->insert_tail($bucket3);

                $ctx->{'state'} = 'waiting_for_end_of_headers';
            }
            else
            {
                $bb->insert_tail($bucket);
                $ctx->{'state'} = 'ignore';
            }
        }
        elsif ($ctx->{'state'} eq 'waiting_for_end_of_headers')
        {
            $bucket->read(my $header);
            warn "received header ... ".$header."\r\n";

            if ($header =~ m|^\r\n$|)
            {
                warn "detected end_of_headers\r\n";

                my $post_data = &get_post_data();


                ### as soon as I add 'data=test' to this bucket the request
appears to hang.
                ############################################


                my $end_of_headers_bucket =
APR::Bucket->new($connection->bucket_alloc, "\r\ndata=test");

                $bb->insert_tail($end_of_headers_bucket);
                $ctx->{'state'} = 'finished';
            }
            else
            {
                $bb->insert_tail($bucket);
            }
        }
    }

    # set context.
    $f->ctx($ctx);

    return Apache2::Const::OK;
}


1;







On 25 August 2010 09:41, James Lee <[email protected]> wrote:

> Hi Andre, thanks for the response.
>
> I don't actually want to use a PerlResponseHandler, I was just using that
> to make sure my filter did what I wanted it to do.
>
> I actually wanted the request filter to add the POST body expecting then
> that mod_proxy would do the rest. I expected mod_proxy to kick in around
> PerlTransHandler time but wasn't sure when my request filter got called.
> Either way it wasn't working :)
>
> I may see whether I can get the connection filter to add the content I was
> just after a little clarification when filters were called in relation to
> mod_proxy.
>
> Thanks again.
>
>
>
> On 25 August 2010 07:12, André Warnier <[email protected]> wrote:
>
>> James Lee wrote:
>>
>>> Hi mod_perl community, this is my first post so be gentle with me.
>>>
>>> I'm trying to create something which translates a GET request into a POST
>>> (creating the body dynamically) and then hand off the POST to another
>>> server.
>>>
>>> I have created a mod_perl connection filter to change the GET to a POST
>>> in
>>> the request line. It also adds Content-Length and Content-Type headers
>>> and
>>> this works fine.
>>>
>>> I then have a request filter which creates the POST body. This works when
>>> the request is handled by a PerlResponseHandler but I'm trying to offload
>>> the request to mod_proxy as the responses can be quite large but I just
>>> can't get it to play ball. I see the request filter being called but the
>>> POST body never arrives at the target server. I suspect this is a timing
>>> issue, ie: mod_proxy kicking in before my request filter but I'm not
>>> certain.
>>>
>>
>> Exactly. By the time your response handler is invoked, the time for
>> mod_proxy is long past.
>>
>> One approach would be : have your PerlResponseHandler send the request
>> directly to the back-end server, using LWP, read the response, and return
>> this response as the response for your response handler.  In other words, do
>> yourself what mod_proxy would do.
>>
>>
>>
>>> Can anybody shed some light on this or correct my approach. I was going
>>> to
>>> add the POST body in the connection filter but it seemed cleaner/easier
>>> to
>>> do it in a request filter.
>>>
>>
>> In the mod_perl documentation, there is somewhere a schema of the
>> different Apache phases, and when the different handlers get invoked.  I
>> beliebe mod_proxygets invoked somewhere around the Fixup phase, so if you
>> want to beat it, you have to be earlier than that.
>>
>>
>>
>>> I've looked on the mailing list and found a few things that touch on this
>>> (
>>> http://tech.groups.yahoo.com/group/modperl/message/54541) but nothing
>>> that's
>>> close enough to help.
>>>
>>> Config below ... I've not included perl code as this message is quite
>>> long
>>> anyway. Please let me know if it would be helpful.
>>> Thanks in advance, James.
>>>
>>>
>>> ***
>>> httpd.conf extract:
>>>
>>>
>>> PerlInputFilterHandler Sample::RequestTweaker::change_get_to_post
>>>
>>> <Location /reports>
>>>   PerlInputFilterHandler Sample::RequestTweaker::inject_post_data
>>>   ProxyPass http://appserver/reports-engine
>>> </Location>
>>>
>>>
>>
>