Re: In re: improving async support in psycopg

 > I think this is not how async support should be implemented.
 > It is too invasive.

I wouldn't call this an async support implementation, it's more like how 
I put together psycopg and io-event-scheduled coroutines. It does, 
however, have some things common with async support.

 > Instead psycopg should just provide a *minimal* API (like libpq) to
 > allow external applications to take advantage of an asynchronous
 > environment.
 >
 > If you want a connection that it is always async, you can
 > write a Python wrapper on the top of psycopg.

Why is it that people tend to not notice the overhead that such wrappers 
bring? If it's not visible under your loads, why even bother with 
coroutines or async frameworks? Plain old threads would presumably suffice.

 > It should not be hard to write a custom SQLAlchemy dialect, too.

It's not needed as long as libpq wrapper stays DB-API compatible and 
coroutine switches are hidden from end-user code. Currently I use 
vanilla SQLAlchemy, and there is one obscure corner case where default 
SQLAlchemy connection pool may need a monkeypatch: when many requests 
all demand connections and db is dead or severely lagging. Given that 
app dies anyway as load ramps up even more, I just don't bother.

 > How a socket is checked for ready status should not be done in the
 > psycopg implementation,

It's not done in psycopg implementation, as you can see. coev_wait() is 
external to it. Theoretically one can replace libucoev with an 
implementation that calls back to Python/'standard event/reactor API' 
code instead.

 > it should be done by external applications (and
 > using greenlets and an standard event/reactor API, you can write a
 > flexible and reusable wrapper).

Checking a socket for ready status is a performance-critical operation, 
thus it should not be exported through a bunch of hoops to some unknown 
implementation, it should sit in core framework, and be written in C.



But all that is beside the point, I posted originally just to show how I 
did what I did, taking no compromise on performance and reliability, 
while still staying source-compatible but for a couple of module names, 
solving some of the same problems async support faces.


-- 

./lxnt