Re: contrib: hack for psycopg2 performance

On Wed, May 12, 2010 at 11:53 AM, Karsten Hilbert
<[email protected]> wrote:
> On Tue, May 11, 2010 at 12:24:26AM +0100, Daniele Varrazzo wrote:

>> So I think potentially there are some improvements that can be done,
>> but you will have to fight against the fact that psycopg is a very
>> generic library (I think PQexec was chosen in first instance exactly
>> for being more generic than the *Param equivalent) and trying to
>> replicate the effect of a generic function by composition of less
>> generic ones is not an easy task.
>
> If I remember correctly the *Params library call was
> invented to *get rid* of the SQL injection problem (because
> query assembly doesn't happen before it has arrived *inside*
> the server).

Definitely, this is the reason why it was introduced in the libpq. I
meant PQexec was chosen *by psycopg* because it allows the user to
send more statements in a single shot. Or at least this is how I
reverse-engineered fog design reading the libpq documentation (when I
read P. Christeas mail I thought "cool, let's use the *Param!", then I
went to the docs, read the just-one-query story and, with some
disappointment, arrived at the conclusion that it is too limiting for
the .execute() implementation)

> The non-*Params variant only makes it hard to
> avoid injection if using the libpq based quoting (which
> AFAIR psycopg2 does).
>
> But maybe I'm wrong here.

In a generic application is definitely a risk using the non *Params
variant: there may be queries sprouting everywhere, there may be weak
assumption ("this filed will never be exposed to the UI and only takes
a ZIP code" ecc) that may lead to skip an escaping. But psycopg has a
controlled path between .execute() and PQexec, where all the
parameters are converted into strings and filtered by
PQescapeStringConn: the boundary we have to protect is not
indefinitely large.

-- Daniele