Re: Copy_from gets stuck with bigger input file

Federico Di Gregorio ha scritto:
> I just commited to the trunk a change that allows psycopg to raise an
> error instead of locking when there is an error during the COPY
> operation. There is definitely something bad going on here:
> 
> The call to PQputCopyData() returns -1 (error) and calling
> PQerrorMessage prints the following error:
> 
> server closed the connection unexpectedly
> 	This probably means the server terminated abnormally
> 	before or while processing the request.
> 
> Uh?! Anyway that explains why any successive send operation fails and
> trying to get something from the server just hangs forever. The problem
> is that PQstatus() returns CONNECTION_OK and PQtransactionStatus()
> returns PQTRANS_ACTIVE, so there is no way to cleanup cleanly. So the
> fix is to simply mark any connection that fails in COPY as
> "bad" (closed) forcing the client code to cope with the exception and
> re-open it.
> 
> Someone with more time than me should really write to PostgreSQL devel
> mailing list asking:
> 
>      1. Why if the server says "server closed the connection
>         unexpectedly" the connection is not marked as CONNECTION_BAD?
>      2. Why the COPY fails in the first place?
> 
> Comments?
> 

I don't know the details, but here is the culprit:

/*
  * pqSendSome: send data waiting in the output buffer.
  *
  * len is how much to try to send (typically equal to outCount, but may
  * be less).
  *
  * Return 0 on success, -1 on failure and 1 when not all data could be sent
  * because the socket would block and the connection is non-blocking.
  */
static int
pqSendSome(PGconn *conn, int len)
{

    /* ... */

    sent = pqsecure_write(conn, ptr, len);

    if (sent < 0)
    {
        /*
	* Anything except EAGAIN/EWOULDBLOCK/EINTR is trouble. If it's
	* EPIPE or ECONNRESET, assume we've lost the backend connection
	* permanently.
	*/
	switch (SOCK_ERRNO)
	{
             /* ... */
            case EPIPE
#ifdef ECONNRESET
	   case ECONNRESET:
#endif

             /* ... set the error message ... */

             /*
	     * We used to close the socket here, but that's a bad idea
              * since there might be unread data waiting (typically, a
              * NOTICE message from the backend telling us it's
	     * committing hara-kiri...).  Leave the socket open until
              * pqReadData finds no more data can be read.  But abandon
              * attempt to send data.
	     */
	     conn->outCount = 0;
	     return -1;

      /* ... */
}


Hope it helps.

> federico
> 
> 



Manlio Perillo
_______________________________________________
Psycopg mailing list
[email protected]
http://lists.initd.org/mailman/listinfo/psycopg