Re: Getting DB error = 20017 - Unexpected EOF from the server

[email protected] wrote:
> 
> It appears that the change did not have the desired effect:
> 
> DB error = 20004,  Severity code = 9
> DB error msg Read from the server failed
> OS error # 54
> OS error msg connection reset by peer

http://h71000.www7.hp.com/doc/82final/6529/6529pro_030.html#app_error_codes

54 ECONNRESET	 SS$_CONNECFAIL	 Connection reset by peer

Hi Jeff, 

I think the problem originates on the server, or seems to from the
client's perspective.  

I was hoping Craig would weigh in, because he's done the most to make
FreeTDS work under VMS.  You would certainly be one of only a small number
of people using 0.82 with VMS.  

When I read your post on Friday, I wanted to suggest looking at the server
and the network.  Freddy suggested that select(2) may say the write socket
was ready when it wasn't.  He's right, but as you found there's no fix if
the server's really gone.  

select(2) and poll(2) can return positive -- meaning a socket is "ready"
-- for four reasons: 

1.  data can be read/written
2.  connection is closed
3.  accept/connect completed
4.  socket error pending

We know #3 doesn't apply.  #1 is the good case: the i/o succeeds as
exepected and life goes on.  #2 and #4 are handled when the next i/o
operation returns an error and sets errno appropriately.  If the operation
returns 0, we emit TDSESEOF, else that the read/write failed.  In your
first post, you were getting SYBESEOF, which is  #2 of read(2).  

I think it's a read (not write) failure for two reasons.  

1.  tdserror was called during sqlok(), not sqlsend().  

2.  Once you masked it with Frediano's suggested changes, you got a hard
read error, much as you would from reading from a file after hitting EOF. 

Case #2 results from the peer -- here, the server -- closing the
connection.  The reader is waiting for the sender to send something, and
instead is told the other end has disconnnected.  

For the sake of anyone trying to follow this, let's review.  select(2) (or
poll(2)) returns a count of ready sockets.  In tds_select(), that count
will only ever be 1 at most.  select(2) says the socket is ready, but by
"ready" means it was closed by the server.  When recv(2) or read(2)
returns 0, we know we have an eof condition and say so.  

This is an exceptional condition.  It's not a network failure: By
definition "closed by peer" means we received the server's FIN packet. 
But why?  Normally only clients disconnect.  

Remember the server isn't supposed to disconnect abruptly and, when it
does, nothing is going to restore the connection.  It is not just resting.
 The connection is dead, expired, pushing up daisies, an ex-connexion!  

It's not due to a TDS protocol violation; we know that from the packet you
sent.  I would check the server logs, looking for resource exhaustion or
other reasons it would close the connection.  Also, the 24-hour thing is
suspicious.  Maybe some backup script or cleanup utility is closing
"stale" connections or otherwise resetting the network interface.  Or
maybe a firewall shuts down connections after a period of inactivity. 
Keep in mind it might be half-closed.  

In looking into this, I noticed a subtle mistake in tds_goodwrite(): it
should never return TDSESEOF.  For a closed connection, read(2) returns 0,
but write(2) returns -1.  No special meaning attaches to write(2)
returning zero.  

When write(2) returns zero, tds_goodwrite() returns TDSESEOF.  It should
simply retry and let the timeout take care of things.  It'll take a while,
sending the data zero bytes at a time, but that's no reason to claim it
can't be done.  

The attached patch fixes this.  It won't apply cleanly to your modified
code, but I hope you'l be able to use it as a guide.  By preventing
tds_goodwrite() from emitting TDSESEOF, we'll eliminate some ambiguity.  

One by-the-way suggestion: in your error handler, call strerror(3) instead
of printing out the raw error code.  According to the documentation
Frediano found, 54 is ECONNRESET; if it actually means something else,
we're on a wild goose chase.  

In sum:

1.  It looks like the server is disconnecting for reasons unknown. 
Removing TDSESEOF from tds_goodwrite() will help clarify matters.  

2.  It does not look like a network problem.  

3.  Server disconnections are abnormal, particularly when SQL Server
itself doesn't crash.  

4.  The culprit would seem to be neither the client, nor the network, nor
SQL Server.  Other server processes and/or a firewall seems likely.  

HTH.  

--jkl

P.S. Anyone know why we use MSG_NOSIGNAL on systems that support it?  It
means that the application requirements vary by OS: those that don't
support MSG_NOSIGNAL need a signal handler.  I think that's subtle and
unhelpful.  Furthermore, we should *never* receive SIGPIPE.  If the server
closes first, the first send/write should fail with ECONNRESET or similar.
 We then mark the connection DEAD.  Only writing *again* provokes SIGPIPE.
 The API should prevent the client from ever trying to send data over a
DEAD a connection.

_______________________________________________
FreeTDS mailing list
[email protected]
http://lists.ibiblio.org/mailman/listinfo/freetds

Index: src/tds/net.c
===================================================================
RCS file: /cvsroot/freetds/freetds/src/tds/net.c,v
retrieving revision 1.88
diff -u -r1.88 net.c
--- src/tds/net.c	28 Feb 2009 17:38:50 -0000	1.88
+++ src/tds/net.c	15 Mar 2009 21:03:30 -0000
@@ -692,18 +692,19 @@
 #else
 			nput = WRITESOCKET(tds->s, p, remaining);
 #endif
-			if (nput < 0 && sock_errno == EAGAIN)
+			if (0 == nput || nput < 0 && sock_errno == EAGAIN)
 				continue;
-			/* detect connection close */
-			if (nput <= 0) {
-				tdserror(tds->tds_ctx, tds, nput == 0 ? TDSESEOF : TDSEWRIT, sock_errno);
+
+			if (nput < 0) {
+				tdsdump_log(TDS_DBG_NETWORK, "send(2) failed: %d (%s)\n", err, strerror(err));
+				tdserror(tds->tds_ctx, tds, TDSEWRIT, sock_errno);
 				tds_close_socket(tds);
 				return -1;
  			}
 		} else if (rc < 0) {
 			if (sock_errno == EAGAIN) /* shouldn't happen, but OK, retry */
 				continue;
-			tdsdump_log(TDS_DBG_NETWORK, "TDS: Write failed in tds_write_packet\nError: %d (%s)\n", err, strerror(err));
+			tdsdump_log(TDS_DBG_NETWORK, "select(2) failed: %d (%s)\n", err, strerror(err));
 			tdserror(tds->tds_ctx, tds, TDSEWRIT, sock_errno);
 			tds_close_socket(tds);
 			return -1;
@@ -713,9 +714,13 @@
 			case TDS_INT_CONTINUE:
 				continue;
 			case TDS_INT_TIMEOUT:
-				/* FIXME we are not able to send a packet and we want to send a packet ?? */
+				/* 
+				 * "Cancel the operation ... but leave the dbproc in working condition." 
+				 * We must try to send the cancel packet, else we have to abandon the dbproc.  
+				 * If it can't be done, a harder error e.g. ECONNRESET will bubble up.  
+				 */
 				tds_send_cancel(tds);
-				continue; /* fixme: or return? */
+				continue; 
 			default:
 			case TDS_INT_CANCEL:
 				tds_close_socket(tds);