Re: Fwd: Idea for a Gofer transport for translating SQL - primarily intended for testing

Thinking about how middleware could be implemented in DBD::Gofer - see 
below ...

Tim Bunce wrote:
> On Thu, Sep 08, 2011 at 01:19:17PM +0100, Andrew Ford wrote:
>   
>>    and I would quite like to get this module onto CPAN and would like some feedback as to
>>    whether "the expert" thinks it is a bad idea.  I think it would be a useful addition to the testing
>>    armoury, but then I am obviously biased.
>>     
>
> The general idea is certainly useful. I'd like to generalise it though.
>
> There's a conceptual similarity with PSGI/Plack middlewares that I'd
> like to build on.  Specifically, rather than creating a new transport,
> I'd like to see DBD/Gofer/Transport/Base.pm extended to call one or more
> 'Gofer client middlewares' just before it calls transmit_request_by_transport().
>
> The calling code could be something like:
>
> +   my $mw = $self->middleware;
>     my $transmit_sub = sub {
>         my $response = eval {
>             ...
> -          $self->transmit_request_by_transport($request)
> +          $mw->($self, $request);
>         }
>         ...
>     }
>
> The middleware attribute would default to:
>
>     sub { shift->transmit_request_by_transport(@_) }
>
> new middleware layer would be added by doing:
>
>     my $mw = $self->middleware;
>     $self->middleware( sub {
>         my ($self, $request) = @_;
>         ...do something with $request...
>         my $response = $mw->($self, $request);
>         ...do something with $response...
>         return $response;
>     } );
>
> [This is all off-the-top-of-my-head, and I'm not very familar with Plack
> internals so I may well be missing important issues.]
>
> Then we just need a way to add middlewares via the environment.
>   
> I'd be delighted if you would work up a patch to add that to the DBI.
>   
Currently DBD::Gofer::dr:connect() contains code to set up the transport:

            my $transport_class = delete $go_attr{go_transport}
                or return $drh->set_err($DBI::stderr, "No transport=
    argument in '$orig_dsn'");
            $transport_class = "DBD::Gofer::Transport::$transport_class"
                unless $transport_class =~ /::/;
            _load_class($transport_class)
                or return $drh->set_err($DBI::stderr, "Can't load
    $transport_class: $@");
            my $go_transport = eval { $transport_class->new(\%go_attr) }
                or return $drh->set_err($DBI::stderr, "Can't instanciate
    $transport_class: $@");


I would imagine that we could have a "middleware" keyword that was a 
comma-separated list of names that were assumed to be in the 
DBDx::GoferMiddleware namespace - these would be installed in sequence 
with a similar block of code that follows on from the snippet above, 
looking something like:

            my $middleware_classes = delete $go_attr{go_middleware};
            foreach my $middleware_class (split /,/,
    $middleware_classes) {      
                $middleware_class =
    "DBDx::GoferMiddleware::$middleware_class"
                    unless $middleware_class =~ /::/;
                _load_class($middleware_class)
                    or return $drh->set_err($DBI::stderr, "Can't load
    $middleware_class: $@");
                $go_transport->middleware( sub { return
    $middleware_class->HANDLER(shift, $go_transport->middleware) } );
            }

Each middleware handler would take a request object and a reference to 
the next handler in the chain.

We would need to have a default method name for the middleware handler 
(I've used the placeholder HANDLER in the pseudo-code above).

 The DSNs would then look like:

     dbi:Gofer:transport=null;middleware=adaptor;transform=MyMapper;dsn=dbi:XXX...

There are a few holes in this that I can think of:

    * We might want to instantiate a middleware object for each
      component and use that in installing the middleware handler
    * I am not sure how we could specify a handler name other than the
      default for individual middleware components
    * Also not sure how attributes for each middleware component could
      be specified such that the component for which the attribute is
      intended could be specified - the example DSN above assumes that
      the attribute (transform in the example DSN) is just dumped into
      the general "go_" attribute hash)
    * The go_middleware attribute could be set in the DBI connect
      attribute hash to a list of subroutine references (code would need
      to differentiate between a comma-separated list value and a list
      reference value)
    * Should the middleware components be listed innermost first (as
      implemented in the pseudo-code above) or outermost first?
    * We might want to add some convenience methods to the request class
      to build response objects (as suggested in my
      DBD::Gofer::Transport::adaptor code)


I'll have a go at getting this coded over the next few days, but it 
looks as if the changes required to implement middleware are relatively 
trivial.

> Your module could then use this mechanism. If you wanted to release it
> to CPAN then a name like DBDx::GoferMiddleware::FOO would be good.
> (Note the DBDx not DBIx since this is client-side. We may well end up
> with server-side gofer middlewares as well.)
>   
I think I would take the name DBDx::GoferMiddleware::adaptor for my 
transformation middleware, unless anyone has a better idea (are we 
keeping the final class name component for the middleware class in lower 
case, as it is with the transport classes?).

Andrew