OpenJMS server seems to lock up
"Eric Kreiser" <[email protected]>
| Newsgroups | gmane.comp.java.openjms.devel |
|---|---|
| Message-ID | <[email protected]> |
Intro: We are using two different versions of OpenJMS in production environments, .5 and .7.2. The server is running under the RMI mode in both versions, and use it to handle Persistent Messages with Durable Consumers. All Sessions are managed as Client Acknowledged. The persistence is going to an Oracle Database. The problem we have experienced is that under both versions (.5 and .7.2) we have had instances where the JMS server has locked up. When we get a thread dump as part of killing the server after it has stopped responding everything seems to be inside core JMS code and none inside my business logic consumers. Does anyone have any suggestions as to what I can look at/test for/etc... The following is a subset of the thread dump from the time they went down. Thread dump from version .5 ---------------------------------------------- Full thread dump: "RMI TCP Connection(2602)-204.183.205.88" daemon prio=5 tid=0x7ed5f8 nid=0xfde waiting for monitor entry [0xb7c00000..0xb7c01a28] at org.exolab.jms.messagemgr.MessageDispatcher.add(MessageDispatcher.java:167) at org.exolab.jms.messagemgr.MessageMgr.add(MessageMgr.java:335) at org.exolab.jms.server.JmsServerSession.sendMessage(JmsServerSession.java:296) at org.exolab.jms.server.rmi.RmiJmsServerSession.sendMessage(RmiJmsServerSession.java:165) at java.lang.reflect.Method.invoke(Native Method) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:241) at sun.rmi.transport.Transport$1.run(Transport.java:142) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:139) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:443) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:643) at java.lang.Thread.run(Thread.java:484) "RMI RenewClean-[172.20.4.8:61723]" daemon prio=5 tid=0x742ff0 nid=0xec3 waiting on monitor [0xc0f01000..0xc0f01a28] at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:495) at java.lang.Thread.run(Thread.java:484) "RMI TCP Connection(1637)-172.20.5.7" daemon prio=5 tid=0x5e8a38 nid=0xb28 waiting for monitor entry [0xddf00000..0xddf01a28] at oracle.jdbc.driver.OracleConnection.remove_statement(OracleConnection.java:773) at oracle.jdbc.driver.OracleStatement.close(OracleStatement.java:1396) at oracle.jdbc.driver.OraclePreparedStatement.close(OraclePreparedStatement.java:1646) at org.exolab.jms.persistence.RDBMSAdapter.getLastId(RDBMSAdapter.java:355) at org.exolab.jms.util.IdMessageGenerator.getId(IdMessageGenerator.java:131) at org.exolab.jms.messagemgr.PersistentCache.add(PersistentCache.java:180) at org.exolab.jms.messagemgr.MessagePool.add(MessagePool.java:126) at org.exolab.jms.messagemgr.MessageDispatcher.add(MessageDispatcher.java:168) at org.exolab.jms.messagemgr.MessageMgr.add(MessageMgr.java:335) at org.exolab.jms.server.JmsServerSession.sendMessage(JmsServerSession.java:296) at org.exolab.jms.server.rmi.RmiJmsServerSession.sendMessage(RmiJmsServerSession.java:165) at java.lang.reflect.Method.invoke(Native Method) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:241) at sun.rmi.transport.Transport$1.run(Transport.java:142) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:139) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:443) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:643) at java.lang.Thread.run(Thread.java:484) "jcp-2" prio=5 tid=0x35d438 nid=0x33 runnable [0xdf701000..0xdf701a28] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:413) at java.net.ServerSocket.implAccept(ServerSocket.java:243) at java.net.ServerSocket.accept(ServerSocket.java:222) at allaire.jrun.NetworkService.accept(NetworkService.java:213) at allaire.jrun.jrpp.ProxyService.createRunnable(ProxyService.java:31) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "Pool Manager Deamon" daemon prio=1 tid=0x474f90 nid=0x2e waiting on monitor [0xdd801000..0xdd801a28] at java.lang.Thread.sleep(Native Method) at tyrex.resource.ResourcePoolManagerImpl.run(ResourcePoolManagerImpl.java:412) at java.lang.Thread.run(Thread.java:484) "XADataSource Timeout Daemon" daemon prio=1 tid=0x474a68 nid=0x2d waiting for monitor entry [0xdd901000..0xdd901a28] at oracle.jdbc.driver.OraclePreparedStatement.close(OraclePreparedStatement.java:1643) at oracle.jdbc.driver.OracleConnection.close_statements(OracleConnection.java:783) at oracle.jdbc.driver.OracleConnection.close(OracleConnection.java:350) at tyrex.jdbc.xa.XADataSourceImpl.run(XADataSourceImpl.java:453) at java.lang.Thread.run(Thread.java:484) "RMI LeaseChecker" daemon prio=5 tid=0x45e4d0 nid=0x2a waiting on monitor [0xddc01000..0xddc01a28] at java.lang.Thread.sleep(Native Method) at sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java:299) at java.lang.Thread.run(Thread.java:484) "Scheduler" prio=5 tid=0x45a2e8 nid=0x26 waiting on monitor [0xde001000..0xde001a28] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:420) at org.exolab.jms.messagemgr.Scheduler.next(Scheduler.java:210) at org.exolab.jms.messagemgr.Scheduler.run(Scheduler.java:171) at java.lang.Thread.run(Thread.java:484) "ThreadManagerReaper" prio=5 tid=0x45ad88 nid=0x25 waiting on monitor [0xde101000..0xde101a28] at java.lang.Thread.sleep(Native Method) at org.exolab.jms.lease.LeaseManager.run(LeaseManager.java:293) at java.lang.Thread.run(Thread.java:484) "Thread-4" prio=5 tid=0x445c48 nid=0x1a waiting on monitor [0xdec01000..0xdec01a28] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:420) at org.exolab.core.util.FifoQueue.waitWhileEmpty(FifoQueue.java:308) at org.exolab.core.util.FifoQueue.get(FifoQueue.java:207) at org.exolab.core.threadPool.ThreadPoolWorker.runWork(ThreadPoolWorker.java:175) at org.exolab.core.threadPool.ThreadPoolWorker.access$0(ThreadPoolWorker.java:161) at org.exolab.core.threadPool.ThreadPoolWorker$1.run(ThreadPoolWorker.java:116) at java.lang.Thread.run(Thread.java:484) "Pool Manager Deamon" daemon prio=1 tid=0x444da0 nid=0x19 waiting on monitor [0xded01000..0xded01a28] at java.lang.Thread.sleep(Native Method) at tyrex.resource.ResourcePoolManagerImpl.run(ResourcePoolManagerImpl.java:412) at java.lang.Thread.run(Thread.java:484) "ThreadLocal Cleanup Daemon" daemon prio=1 tid=0x447c50 nid=0x18 waiting on monitor [0xdee01000..0xdee01a28] at java.lang.Thread.sleep(Native Method) at tyrex.util.FastThreadLocal.run(FastThreadLocal.java:379) at java.lang.Thread.run(Thread.java:484) "XADataSource Timeout Daemon" daemon prio=1 tid=0x3764a8 nid=0x17 waiting on monitor [0xdef01000..0xdef01a28] at java.lang.Thread.sleep(Native Method) at tyrex.jdbc.xa.XADataSourceImpl.run(XADataSourceImpl.java:428) at java.lang.Thread.run(Thread.java:484) "RMI TCP Accept-2" daemon prio=5 tid=0x3603a8 nid=0x16 runnable [0xdf001000..0xdf001a28] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:413) at java.net.ServerSocket.implAccept(ServerSocket.java:243) at java.net.ServerSocket.accept(ServerSocket.java:222) at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:309) at java.lang.Thread.run(Thread.java:484) "GC Daemon" daemon prio=2 tid=0x360280 nid=0x15 waiting on monitor [0xdf101000..0xdf101a28] at java.lang.Object.wait(Native Method) at sun.misc.GC$Daemon.run(GC.java:103) "RMI Reaper" prio=5 tid=0x360158 nid=0x14 waiting on monitor [0xdf201000..0xdf201a28] at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:284) at java.lang.Thread.run(Thread.java:484) "RMI TCP Accept-1" daemon prio=5 tid=0x35fc30 nid=0x13 runnable [0xdf301000..0xdf301a28] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:413) at java.net.ServerSocket.implAccept(ServerSocket.java:243) at java.net.ServerSocket.accept(ServerSocket.java:222) at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:309) at java.lang.Thread.run(Thread.java:484) "Thread-2" prio=5 tid=0x2b398 nid=0x1 runnable [0..0xffbee360] "jcp-0" prio=5 tid=0x3605f8 nid=0x12 runnable [0xdf401000..0xdf401a28] at java.net.SocketInputStream.socketRead(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:86) at java.io.BufferedInputStream.fill(BufferedInputStream.java:186) at java.io.BufferedInputStream.read1(BufferedInputStream.java:225) at java.io.BufferedInputStream.read(BufferedInputStream.java:280) at allaire.jrun.jrpp.ProxyEndpoint.readFully(ProxyEndpoint.java:310) at allaire.jrun.jrpp.ProxyEndpoint.readFully(ProxyEndpoint.java:302) at allaire.jrun.jrpp.ProxyEndpoint.readInt(ProxyEndpoint.java:320) at allaire.jrun.jrpp.ProxyEndpoint.readRequest(ProxyEndpoint.java:188) at allaire.jrun.jrpp.ProxyService.swapRunnable(ProxyService.java:48) at allaire.jrun.ThreadPool.swapRunnable(ThreadPool.java:223) at allaire.jrun.WorkerThread.run(WorkerThread.java:77) "scheduler-0" prio=5 tid=0x167558 nid=0x11 waiting on monitor [0xdf501000..0xdf501a28] at java.lang.Object.wait(Native Method) at allaire.jrun.scheduler.SchedulerService.createRunnable(SchedulerService.java:160) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "Thread-1" prio=5 tid=0x16a020 nid=0x10 waiting on monitor [0xdf601000..0xdf601a28] at java.lang.Thread.sleep(Native Method) at com.knowledgeplanet.messaging.DelayedMessageServer.run(DelayedMessageServer.java:28) "scheduler-0" prio=5 tid=0x3cb830 nid=0xd waiting on monitor [0xdf901000..0xdf901a28] at java.lang.Object.wait(Native Method) at allaire.jrun.scheduler.SchedulerService.createRunnable(SchedulerService.java:160) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "control-0" prio=5 tid=0x3be058 nid=0xc runnable [0xdfa01000..0xdfa01a28] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:413) at java.net.ServerSocket.implAccept(ServerSocket.java:243) at java.net.ServerSocket.accept(ServerSocket.java:222) at allaire.jrun.NetworkService.accept(NetworkService.java:213) at allaire.jrun.control.ControlService.createRunnable(ControlService.java:169) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "scheduler-0" prio=5 tid=0x3b9d40 nid=0xb waiting on monitor [0xdfb01000..0xdfb01a28] at java.lang.Object.wait(Native Method) at allaire.jrun.scheduler.SchedulerService.createRunnable(SchedulerService.java:160) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "Signal Dispatcher" daemon prio=10 tid=0xb7b70 nid=0x9 runnable [0..0] "Finalizer" daemon prio=8 tid=0xb4f38 nid=0x7 waiting on monitor [0xfae81000..0xfae81a28] at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at java.lang.ref.Finalizer$FinalizerWorker$FinalizerThread.run(Finalizer.java:120) "Reference Handler" daemon prio=10 tid=0xb2fc0 nid=0x6 runnable [0xfaf81000..0xfaf81a28] at java.lang.Object.notifyAll(Native Method) at java.lang.ref.ReferenceQueue.spliceQueues(ReferenceQueue.java:67) at java.lang.ref.Reference$ReferenceHandler.distributeBatch(Reference.java:117) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:135) ---------------------------------------------- Thread dump from version .7.2 ---------------------------------------------- Full thread dump: "RMI TCP Connection(2602)-204.183.205.88" daemon prio=5 tid=0x7ed5f8 nid=0xfde waiting for monitor entry [0xb7c00000..0xb7c01a28] at org.exolab.jms.messagemgr.MessageDispatcher.add(MessageDispatcher.java:167) at org.exolab.jms.messagemgr.MessageMgr.add(MessageMgr.java:335) at org.exolab.jms.server.JmsServerSession.sendMessage(JmsServerSession.java:296) at org.exolab.jms.server.rmi.RmiJmsServerSession.sendMessage(RmiJmsServerSession.java:165) at java.lang.reflect.Method.invoke(Native Method) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:241) at sun.rmi.transport.Transport$1.run(Transport.java:142) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:139) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:443) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:643) at java.lang.Thread.run(Thread.java:484) "RMI RenewClean-[172.20.4.8:61723]" daemon prio=5 tid=0x742ff0 nid=0xec3 waiting on monitor [0xc0f01000..0xc0f01a28] at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:495) at java.lang.Thread.run(Thread.java:484) "RMI TCP Connection(1637)-172.20.5.7" daemon prio=5 tid=0x5e8a38 nid=0xb28 waiting for monitor entry [0xddf00000..0xddf01a28] at oracle.jdbc.driver.OracleConnection.remove_statement(OracleConnection.java:773) at oracle.jdbc.driver.OracleStatement.close(OracleStatement.java:1396) at oracle.jdbc.driver.OraclePreparedStatement.close(OraclePreparedStatement.java:1646) at org.exolab.jms.persistence.RDBMSAdapter.getLastId(RDBMSAdapter.java:355) at org.exolab.jms.util.IdMessageGenerator.getId(IdMessageGenerator.java:131) at org.exolab.jms.messagemgr.PersistentCache.add(PersistentCache.java:180) at org.exolab.jms.messagemgr.MessagePool.add(MessagePool.java:126) at org.exolab.jms.messagemgr.MessageDispatcher.add(MessageDispatcher.java:168) at org.exolab.jms.messagemgr.MessageMgr.add(MessageMgr.java:335) at org.exolab.jms.server.JmsServerSession.sendMessage(JmsServerSession.java:296) at org.exolab.jms.server.rmi.RmiJmsServerSession.sendMessage(RmiJmsServerSession.java:165) at java.lang.reflect.Method.invoke(Native Method) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:241) at sun.rmi.transport.Transport$1.run(Transport.java:142) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:139) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:443) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:643) at java.lang.Thread.run(Thread.java:484) "jcp-2" prio=5 tid=0x35d438 nid=0x33 runnable [0xdf701000..0xdf701a28] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:413) at java.net.ServerSocket.implAccept(ServerSocket.java:243) at java.net.ServerSocket.accept(ServerSocket.java:222) at allaire.jrun.NetworkService.accept(NetworkService.java:213) at allaire.jrun.jrpp.ProxyService.createRunnable(ProxyService.java:31) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "Pool Manager Deamon" daemon prio=1 tid=0x474f90 nid=0x2e waiting on monitor [0xdd801000..0xdd801a28] at java.lang.Thread.sleep(Native Method) at tyrex.resource.ResourcePoolManagerImpl.run(ResourcePoolManagerImpl.java:412) at java.lang.Thread.run(Thread.java:484) "XADataSource Timeout Daemon" daemon prio=1 tid=0x474a68 nid=0x2d waiting for monitor entry [0xdd901000..0xdd901a28] at oracle.jdbc.driver.OraclePreparedStatement.close(OraclePreparedStatement.java:1643) at oracle.jdbc.driver.OracleConnection.close_statements(OracleConnection.java:783) at oracle.jdbc.driver.OracleConnection.close(OracleConnection.java:350) at tyrex.jdbc.xa.XADataSourceImpl.run(XADataSourceImpl.java:453) at java.lang.Thread.run(Thread.java:484) "RMI LeaseChecker" daemon prio=5 tid=0x45e4d0 nid=0x2a waiting on monitor [0xddc01000..0xddc01a28] at java.lang.Thread.sleep(Native Method) at sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java:299) at java.lang.Thread.run(Thread.java:484) "Scheduler" prio=5 tid=0x45a2e8 nid=0x26 waiting on monitor [0xde001000..0xde001a28] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:420) at org.exolab.jms.messagemgr.Scheduler.next(Scheduler.java:210) at org.exolab.jms.messagemgr.Scheduler.run(Scheduler.java:171) at java.lang.Thread.run(Thread.java:484) "ThreadManagerReaper" prio=5 tid=0x45ad88 nid=0x25 waiting on monitor [0xde101000..0xde101a28] at java.lang.Thread.sleep(Native Method) at org.exolab.jms.lease.LeaseManager.run(LeaseManager.java:293) at java.lang.Thread.run(Thread.java:484) "Thread-4" prio=5 tid=0x445c48 nid=0x1a waiting on monitor [0xdec01000..0xdec01a28] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:420) at org.exolab.core.util.FifoQueue.waitWhileEmpty(FifoQueue.java:308) at org.exolab.core.util.FifoQueue.get(FifoQueue.java:207) at org.exolab.core.threadPool.ThreadPoolWorker.runWork(ThreadPoolWorker.java:175) at org.exolab.core.threadPool.ThreadPoolWorker.access$0(ThreadPoolWorker.java:161) at org.exolab.core.threadPool.ThreadPoolWorker$1.run(ThreadPoolWorker.java:116) at java.lang.Thread.run(Thread.java:484) "Pool Manager Deamon" daemon prio=1 tid=0x444da0 nid=0x19 waiting on monitor [0xded01000..0xded01a28] at java.lang.Thread.sleep(Native Method) at tyrex.resource.ResourcePoolManagerImpl.run(ResourcePoolManagerImpl.java:412) at java.lang.Thread.run(Thread.java:484) "ThreadLocal Cleanup Daemon" daemon prio=1 tid=0x447c50 nid=0x18 waiting on monitor [0xdee01000..0xdee01a28] at java.lang.Thread.sleep(Native Method) at tyrex.util.FastThreadLocal.run(FastThreadLocal.java:379) at java.lang.Thread.run(Thread.java:484) "XADataSource Timeout Daemon" daemon prio=1 tid=0x3764a8 nid=0x17 waiting on monitor [0xdef01000..0xdef01a28] at java.lang.Thread.sleep(Native Method) at tyrex.jdbc.xa.XADataSourceImpl.run(XADataSourceImpl.java:428) at java.lang.Thread.run(Thread.java:484) "RMI TCP Accept-2" daemon prio=5 tid=0x3603a8 nid=0x16 runnable [0xdf001000..0xdf001a28] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:413) at java.net.ServerSocket.implAccept(ServerSocket.java:243) at java.net.ServerSocket.accept(ServerSocket.java:222) at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:309) at java.lang.Thread.run(Thread.java:484) "GC Daemon" daemon prio=2 tid=0x360280 nid=0x15 waiting on monitor [0xdf101000..0xdf101a28] at java.lang.Object.wait(Native Method) at sun.misc.GC$Daemon.run(GC.java:103) "RMI Reaper" prio=5 tid=0x360158 nid=0x14 waiting on monitor [0xdf201000..0xdf201a28] at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:284) at java.lang.Thread.run(Thread.java:484) "RMI TCP Accept-1" daemon prio=5 tid=0x35fc30 nid=0x13 runnable [0xdf301000..0xdf301a28] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:413) at java.net.ServerSocket.implAccept(ServerSocket.java:243) at java.net.ServerSocket.accept(ServerSocket.java:222) at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:309) at java.lang.Thread.run(Thread.java:484) "Thread-2" prio=5 tid=0x2b398 nid=0x1 runnable [0..0xffbee360] "jcp-0" prio=5 tid=0x3605f8 nid=0x12 runnable [0xdf401000..0xdf401a28] at java.net.SocketInputStream.socketRead(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:86) at java.io.BufferedInputStream.fill(BufferedInputStream.java:186) at java.io.BufferedInputStream.read1(BufferedInputStream.java:225) at java.io.BufferedInputStream.read(BufferedInputStream.java:280) at allaire.jrun.jrpp.ProxyEndpoint.readFully(ProxyEndpoint.java:310) at allaire.jrun.jrpp.ProxyEndpoint.readFully(ProxyEndpoint.java:302) at allaire.jrun.jrpp.ProxyEndpoint.readInt(ProxyEndpoint.java:320) at allaire.jrun.jrpp.ProxyEndpoint.readRequest(ProxyEndpoint.java:188) at allaire.jrun.jrpp.ProxyService.swapRunnable(ProxyService.java:48) at allaire.jrun.ThreadPool.swapRunnable(ThreadPool.java:223) at allaire.jrun.WorkerThread.run(WorkerThread.java:77) "scheduler-0" prio=5 tid=0x167558 nid=0x11 waiting on monitor [0xdf501000..0xdf501a28] at java.lang.Object.wait(Native Method) at allaire.jrun.scheduler.SchedulerService.createRunnable(SchedulerService.java:160) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "Thread-1" prio=5 tid=0x16a020 nid=0x10 waiting on monitor [0xdf601000..0xdf601a28] at java.lang.Thread.sleep(Native Method) at com.knowledgeplanet.messaging.DelayedMessageServer.run(DelayedMessageServer.java:28) "scheduler-0" prio=5 tid=0x3cb830 nid=0xd waiting on monitor [0xdf901000..0xdf901a28] at java.lang.Object.wait(Native Method) at allaire.jrun.scheduler.SchedulerService.createRunnable(SchedulerService.java:160) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "control-0" prio=5 tid=0x3be058 nid=0xc runnable [0xdfa01000..0xdfa01a28] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:413) at java.net.ServerSocket.implAccept(ServerSocket.java:243) at java.net.ServerSocket.accept(ServerSocket.java:222) at allaire.jrun.NetworkService.accept(NetworkService.java:213) at allaire.jrun.control.ControlService.createRunnable(ControlService.java:169) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "scheduler-0" prio=5 tid=0x3b9d40 nid=0xb waiting on monitor [0xdfb01000..0xdfb01a28] at java.lang.Object.wait(Native Method) at allaire.jrun.scheduler.SchedulerService.createRunnable(SchedulerService.java:160) at allaire.jrun.ThreadPool.createRunnable(ThreadPool.java:175) at allaire.jrun.WorkerThread.run(WorkerThread.java:72) "Signal Dispatcher" daemon prio=10 tid=0xb7b70 nid=0x9 runnable [0..0] "Finalizer" daemon prio=8 tid=0xb4f38 nid=0x7 waiting on monitor [0xfae81000..0xfae81a28] at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at java.lang.ref.Finalizer$FinalizerWorker$FinalizerThread.run(Finalizer.java:120) "Reference Handler" daemon prio=10 tid=0xb2fc0 nid=0x6 runnable [0xfaf81000..0xfaf81a28] at java.lang.Object.notifyAll(Native Method) at java.lang.ref.ReferenceQueue.spliceQueues(ReferenceQueue.java:67) at java.lang.ref.Reference$ReferenceHandler.distributeBatch(Reference.java:117) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:135) ----------------------------------------------