Re: [jgroups-users] RejectedExcecutionException in TP

Bonjour Bruce, ca va bien? :-)
Switching to English now...

On 11/04/16 17:32, Questions/problems related to using JGroups wrote:
> Bonjour à tous,
>
> I hit this exception in a test run with v3.6.7.
>
> java.util.concurrent.RejectedExecutionException: Task
> org.jgroups.protocols.TP$3@3485cdbb rejected from
> java.util.concurrent.ThreadPoolExecutor@70067f19[Running, pool size = 4,
> active threads = 4, queued tasks = 500, completed tasks = 1717]
> at
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
> at
> org.jgroups.util.ShutdownRejectedExecutionHandler.rejectedExecution(ShutdownRejectedExecutionHandler.java:33)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
> at org.jgroups.protocols.TP.loopback(TP.java:1522)
> at org.jgroups.protocols.TP.down(TP.java:1486)
> at org.jgroups.stack.Protocol.down(Protocol.java:438)
>
> It looks like I could increase the size of the internal thread pool in TP

How do you know it was the internal thread pool? The thread pool picked 
for looping back a message is based on the message itself, ie. if the 
message is OOB, then the OOB pool will be picked, else the regular pool.

> but shouldn't it have some exception handling and retry for this?

No, the stragegy for handling of full thread pools is to abort, ie. 
throw an exception, increment num_rejected_msgs and drop the message. 
Unreliable messages will get dropped, reliable messages retransmitted.

I don't want to compound the problem by sending even more messages.

You could, however, replace the behavior by setting your own rejection 
policy, or even replacing the entire thread pool altogether.

> I don't have a way to control the number of threads that might send
> messages requiring loopback at any point in time.  I would set
> loopback_separate_thread to false but this setting warns "Don't use
> this; might get removed without warning".

True. If the pool in question has no queue enabled and a high 
max-threads, then you should be fine.

In general, dropping messages is not as bad as it might seem, as this 
leads to retransmission and possibly throttling of the sender as it 
won't get credits as quickly as it liked.

> Bruce Schuchardt
> Apache Geode project


-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
javagroups-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/javagroups-users