Re: Connection reset

Peter Jones - JavaSoft East <[email protected]> Fri, 17 Mar 2006 17:43:03 -0500
Newsgroups gmane.comp.java.sun.rmi
Message-ID <20060317224303.GE12767@east>
> I'm getting the exception below, 2 hours after running
> my command. The server-side processing should take
> about 30 seconds, although the return value is a bunch
> of HashMaps which may take a while to serialize.
>
> It looks to me like the server thinks it has finished
> the command, but the client hasn't received all the
> data it was expecting and so it leaves the connection
> open.  After 2 hours the server gets bored and resets
> the connection.
>
> Is this the correct interpretation? If so, what would
> cause it?  Should I be able to shorten the timeout
> period with the property
> sun.rmi.transport.tcp.readTimeout
>                              ?  Changing this property
> doesn't seem to have the desired effect (JDK 1.5.0_06,
> client and server are RH Linux FC3/4).

That system property's default value is indeed two hours, but it
doesn't seem related to this failure.  It controls the server-side
socket read timeout, so it only applies when the server is expecting
data from a connection (like a new remote invocation request, or the
remainder of the current request).  It appears that the client was in
the middle of unmarshalling a HashMap contained in the return value of
a "getDataSet" invocation when the exception occurred.  As the client
was trying to read from the connection, if the server closed its
socket because of this timeout, the client should get an EOFException,
not a connection reset indication.

The two hour delay and the connection reset failure might suggest that
the TCP keepalive mechanism triggered the failure.  But that would
seem to indicate that TCP segments between the client and server
machines were getting dropped at some point.  Does other communication
between them seem to work throughout this period?

How reproducible is this failure-- always, occasionally, etc.?

Have you gotten thread dumps from the client and server VMs during
this two hour period?  "netstat" output on the client and server
machines during this period?

> java.lang.RuntimeException: RMI exception
>         at com.xxx.data.database.AbstractDataDatabase_Stub.getDataSet(AbstractDataDatabase_Stub.java:366)
>         at com.xxx.data.delegatedatabase.DelegateDataDatabase.getDataSet(DelegateDataDatabase.java:54)
> Caused by: java.rmi.UnmarshalException: error unmarshalling return; nested exception is:
>         java.net.SocketException: Connection reset
>         at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:157)
>         at com.nexgenfs.data.database.AbstractDataDatabase_Stub.getDataSet(AbstractDataDatabase_Stub.java:359)
>         ... 7 more
> Caused by: java.net.SocketException: Connection reset
>         at java.net.SocketInputStream.read(SocketInputStream.java:168)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>         at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
>         at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2217)
>         at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2230)
>         at java.io.ObjectInputStream$BlockDataInputStream.readInt(ObjectInputStream.java:2722)
>         at java.io.ObjectInputStream.readHandle(ObjectInputStream.java:1398)
>         at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1279)
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:339)
>         at java.util.HashMap.readObject(HashMap.java:1013)
>         at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:585)
>         at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:919)
>         at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1813)
>         at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1713)
>         at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1299)
>         at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1912)
>         at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1836)
>         at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1713)
>         at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1299)
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:339)
>         at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:290)
>         at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:139)
>         ... 8 more

-- Peter

===========================================================================
To unsubscribe, send email to [email protected] and include in the body
of the message "signoff RMI-USERS".  For general help, send email to
[email protected] and include in the body of the message "help".

For a list of frequently asked RMI questions please refer to:
http://java.sun.com/j2se/1.3/docs/guide/rmi/faq.html

To view past RMI-USERS postings, please see:
http://archives.java.sun.com/archives/rmi-users.html