Re: contrib: hack for psycopg2 performance

On 12/05/10 20:22, Daniele Varrazzo wrote:
> On Wed, May 12, 2010 at 6:04 PM, Michael Tharp
> <gxti-1pawZKhx9Om5WRpDikjj11aTQe2KTcn/@public.gmane.org> wrote:
>> On 05/10/2010 05:54 PM, Daniele Varrazzo wrote:
>> Obviously any completely transparent replacement, regardless of
>> implementation, would need to be able to parse enough SQL to reliably split
>> statements. I don't suppose that would be terribly difficult because SQL
>> quoting rules are fairly simple, but it's still more work to be done.
> 
> It is also kinda brittle in the case of new statements being
> introduced in the syntax, but probably working at tokenizer level,
> without syntax knowledge, is enough for a split algorithm. But I'd
> rather have a keyword parameter on cursor.execute() to say to use
> *Param version (when it's true, the query is assumed containing a
> single statement) than going down the road of tokenizing the query.

It'd be interesting to see how big is the performance improvement of
using PQExecParams for a typical case. If it's a clear win that maybe it
would we worthwhile to invest some effort in trying to make it the
default behaviour. If it's a big win only for some queries, then I think
having a kwarg that selects this type is the appropriate thing (and you
don't get all the fancy custom typecasters, it's your responsability not
to send multiple statements, etc).

Cheers,
Jan

PS: The issue of PQExecParams being safer vs not should not matter: if
PQExec the way psycopg2 does it is vulnerable, then it should be fixed
immediately. BTW, I think it is not likely to be vulnerable and that
most SQL injections come from people doing stupid things like
constructing their queries by hand instead of relying on psycopg2's
casting code, e.g. calling c.execute("select * where x = '%s'" % val)

J