Re: contrib: hack for psycopg2 performance

On Wed, May 12, 2010 at 6:04 PM, Michael Tharp
<gxti-1pawZKhx9Om5WRpDikjj11aTQe2KTcn/@public.gmane.org> wrote:
> On 05/10/2010 05:54 PM, Daniele Varrazzo wrote:
>>
>> PQexecParams has the shortcoming of being able to send only a query at
>> time, whereas PQexec allows to send any number in a single string,
>> separated by semicolons. Many people use this feature and dropping it
>> would be an incompatible change. How would you work around that?
>
> Can you "pipeline" statements in libpq? e.g. send three statements as
> separate commands but without waiting for the replies to the first two.

I fear not: PQexec (and the different variations) blocks until the
result is returned. These functions are blocking wrappers around the
non-blocking counterparts (e.g. PQsendQuery) together with a wait
loop, but the docs for the latter says "PQsendQuery cannot be called
again (on the same connection) until PQgetResult has returned a null
pointer, indicating that the command is done", so I guess no luck
here.

> Obviously any completely transparent replacement, regardless of
> implementation, would need to be able to parse enough SQL to reliably split
> statements. I don't suppose that would be terribly difficult because SQL
> quoting rules are fairly simple, but it's still more work to be done.

It is also kinda brittle in the case of new statements being
introduced in the syntax, but probably working at tokenizer level,
without syntax knowledge, is enough for a split algorithm. But I'd
rather have a keyword parameter on cursor.execute() to say to use
*Param version (when it's true, the query is assumed containing a
single statement) than going down the road of tokenizing the query.

-- Daniele