Re: Software Caused Connection Abort (10053) on Client

Ok, I think I am confusing the good and bad sequences when a stale
server stub is encountered by the agent. I have a lot of activity going
on and the lab has not sync'd their clocks so it's been a little tough.
But I think I have it now.

When everything works well, things go something like this:

        Client -> proxy: server.invoke
        Server -> trans: NoSuchObjectException
        Client -> proxy: catch NoSuchObjectException
        Server -> trans: unkown op
        Client -> proxy: retry server.invoke -> OK

When things don't work well I get:

        Client -> proxy: server.invoke
        Server -> trans: NoSuchObjectException
        Client -> trans: IOException marshalling arguments:
        Client -> trans: SSLException: Connection has been shutdown
        Client -> proxy: catch MarshalException (nesting the
SSLException)
        Server -> trans: unkown op
        Client -> no retry

Does that seem a little more correct? I would have expected to get a
server side transport trace message saying something like "exception
thrown marshalling exception: ...". But I guess if I did then
"serviceCall" would have returned false and I wouldn't have had this
problem?

About 9 times out of 10 (on the same call and arguments) I will get the
correct sequence. The SSLException eventually nests the "Software caused
connection abort: socket write error".

So this is good, I should be able to just change my #shouldRetry
predicate to include both NoSuchObjectException and MarshalException.

R.



-----Original Message-----
From: Bob Scheifler [mailto:[email protected]] 
Sent: Thursday, December 01, 2005 3:49 PM
To: Robert DiFalco
Cc: [email protected]
Subject: Re: Software Caused Connection Abort (10053) on Client

> The first time the Proxy invokes a method on that cached stub I will 
> get this error on the server side:
> 
>  java.rmi.NoSuchObjectException: no such object in table

That makes sense.

> This results in a NoSuchObjectException on the agent (as you would 
> expect).

Are you positive that a NoSuchObjectException is caught by the
SmartProxy?  Can you provide a client-side stack trace for it?

> So the proxy determines our stub is out of date and uses the Registry 
> on the server to get a fresh one and retries the method invocation. 
> This time there is an invalid op error in the TCPTransport layer of 
> the server and the client gets this exception:

This doesn't match my belief of what should happen.  If the previous
call really resulted in a NoSuchObjectException being caught by the
SmartProxy, the connection should be closed on the client side, and
won't be reused on the next call.  So that next call should not see the
"invalid op" exception that results from misparsing the previous call's
arguments.

If I accept your sequence, that there are two remote calls rather than
one, then server-side logging should be showing two separate exceptions:
the NoSuchObjectException from the first call, and then some other
failure for the second call.
I'm inferring from what you've supplied so far that you're only seeing a
single server-side exception being reported, which would be consistent
with only a single remote call.

Another set of evidence that would be useful here is to set the
client-side sun.rmi.transport.tcp logging level to FINEST, and capture
console logging interleaved with your SmartProxy stack trace prints.
This should tell more clearly if a connection is being reused or not.

> Now all of this seems to be inline with bug #4716483.

Well, it seems in line to me, but with a different sequence of events
than you have described.

> But what I don't
> understand is this. Don't tons of people use Reloading Proxies for 
> service stubs? Why isn't this bug rampant with them?

One possibility is that most people use dynamically assigned ports, and
they get connection refused exceptions rather than
NoSuchObjectExceptions.  Or if they are using static ports (I assuming
you are, is that correct?), that their calls happen to have sufficiently
small data that they reliably get NoSuchObjectExceptions.

> Also, for your scrutiny, here is the offending proxy code, is there 
> something inherently wrong with our initial retry logic?

From a quick scan, no, but the code seems consistent with my sequence
where the first call throws UnmarshalException and the SmartProxy gives
up without retrying (but reloads the proxy so a subsequent call will
succeed).

- Bob

===========================================================================
To unsubscribe, send email to [email protected] and include in the body
of the message "signoff RMI-USERS".  For general help, send email to
[email protected] and include in the body of the message "help".

For a list of frequently asked RMI questions please refer to:
http://java.sun.com/j2se/1.3/docs/guide/rmi/faq.html

To view past RMI-USERS postings, please see:
http://archives.java.sun.com/archives/rmi-users.html