Re: contrib: hack for psycopg2 performance

On 5/12/10, Daniele Varrazzo <[email protected]> wrote:
> On Wed, May 12, 2010 at 6:04 PM, Michael Tharp
>  <gxti-1pawZKhx9Om5WRpDikjj11aTQe2KTcn/@public.gmane.org> wrote:
>  > On 05/10/2010 05:54 PM, Daniele Varrazzo wrote:
>  >>
>  >> PQexecParams has the shortcoming of being able to send only a query at
>  >> time, whereas PQexec allows to send any number in a single string,
>  >> separated by semicolons. Many people use this feature and dropping it
>  >> would be an incompatible change. How would you work around that?
>  >
>  > Can you "pipeline" statements in libpq? e.g. send three statements as
>  > separate commands but without waiting for the replies to the first two.
>
>
> I fear not: PQexec (and the different variations) blocks until the
>  result is returned. These functions are blocking wrappers around the
>  non-blocking counterparts (e.g. PQsendQuery) together with a wait
>  loop, but the docs for the latter says "PQsendQuery cannot be called
>  again (on the same connection) until PQgetResult has returned a null
>  pointer, indicating that the command is done", so I guess no luck
>  here.

Yeah, the problem is that libpq does not support pipelining.

>  > Obviously any completely transparent replacement, regardless of
>  > implementation, would need to be able to parse enough SQL to reliably split
>  > statements. I don't suppose that would be terribly difficult because SQL
>  > quoting rules are fairly simple, but it's still more work to be done.
>
>
> It is also kinda brittle in the case of new statements being
>  introduced in the syntax, but probably working at tokenizer level,
>  without syntax knowledge, is enough for a split algorithm. But I'd
>  rather have a keyword parameter on cursor.execute() to say to use
>  *Param version (when it's true, the query is assumed containing a
>  single statement) than going down the road of tokenizing the query.

Why make things complicated?  How about the following:

1) No arguments - PQexec
2) Has arguments - PQexecParams

How common are multi-statement parametrized queries anyway?

-- 
marko