[jgroups-dev] Buffer starvation or something else?
Jack <[email protected]> Thu, 30 Jan 2014 14:28:50 -0800
| Newsgroups | gmane.comp.java.javagroups.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format.
--===============7713029430147681085==
Content-Type: multipart/alternative;
boundary="------------050008040007070308050505"
This is a multi-part message in MIME format.
--------------050008040007070308050505
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
I hope I'm using the correct forum for this question.
I originally found this problem with the Infinispan's
InitialStateTransfer but the problem is easily reproduced with UUPerf.
When I run UUPerf with the default 4.5M data size and 1 msg I see the
following error mixed randomly in the log:
14:16:27.607 ERROR (Invoker-1) [org.jgroups.protocols.TP.down(TP.java:1344)] JGRP000029: jfrost-multi-8014:failed sending message to jfrost-multi2-58507 (30079 bytes):
java.lang.Exception: dest=/192.168.10.193:58055 (30082 bytes),
headers: FRAG2: [id=1, frag_id=64, num_frags=151], UNICAST3: DATA, seqno=67, conn_id=1, UDP: [channel_name=uuperf]
I tracked in with the debugger and found the root cause is an
IOException with the message "No buffers available" returned by
MulticastSocket.send(). I've seen other postings that the "No buffers
available" can be a bit of a red herring when it comes to the exact
problem(full ARP table under Linux for example). I'm worried I'm
chasing the wrong problem.
I can post trace logs if they will help. From the logs it also appears
that the problem gets progressively worse as buffer problems will do.
So it could be that once the error state is entered buffers don't get
returned in a timely manner.
At first I thought the problem was specifically the frag_size as the
error seemed to occur at the frag_size boundary. For frag_size=20K I
would see errors with size 20082. But I was told by someone else that
the size was normal(data+overhead).
I've tried adjusting UDP buffers and max_bundle_size, FRAG2 frag_size
and UFC/MFC max_credits and threshold. But I can't seem to find a magic
combination of values that will alleviate the problem.
I note below that my test machines are VMWare based but I've been able
to try the test on hardware based FreeBSD and the results are the same.
My sysadmin did some checking and it doesn't seem like the system buffer
pool gets anywhere near depletion. The systems aren't really do
anything else besides my test.
Any insights or suggestions are welcome.
/Jack
------------------------------------------------------------------------
I'm running:
2 nodes running FreeBSD hosted under VMWare.
Nodes are on the same LAN segment, no routers involved.
jgroups 3.4.1 (The problem show up with 3.3.1 as well but the
error message is slightly different, more explicit actually)
openjdk 1.7.0_25
This is one of the last configurations I tried. With this config there
seems to be a sensitivity to logging. With UDP set to DEBUG it only
takes one 4.5M RPC to show the error. With TRACE it may not show up
without raising the RPC count.
<config xmlns="urn:org:jgroups"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:org:jgroupshttp://www.jgroups.org/schema/JGroups-3.4.xsd">
<UDP
mcast_addr="${jgroups.udp.mcast_addr:228.6.7.8}"
mcast_port="${jgroups.udp.mcast_port:46655}"
tos="8"
ucast_recv_buf_size="32m"
ucast_send_buf_size="32m"
mcast_recv_buf_size="32m"
mcast_send_buf_size="32m"
loopback="true"
max_bundle_size="21k"
ip_ttl="${jgroups.udp.ip_ttl:2}"
enable_diagnostics="false"
bundler_type="old"
thread_naming_pattern="pl"
thread_pool.enabled="true"
thread_pool.min_threads="2"
thread_pool.max_threads="30"
thread_pool.keep_alive_time="60000"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="100"
thread_pool.rejection_policy="Discard"
oob_thread_pool.enabled="true"
oob_thread_pool.min_threads="2"
oob_thread_pool.max_threads="30"
oob_thread_pool.keep_alive_time="60000"
oob_thread_pool.queue_enabled="false"
oob_thread_pool.queue_max_size="100"
oob_thread_pool.rejection_policy="Discard"
internal_thread_pool.enabled="true"
internal_thread_pool.min_threads="1"
internal_thread_pool.max_threads="10"
internal_thread_pool.keep_alive_time="60000"
internal_thread_pool.queue_enabled="true"
internal_thread_pool.queue_max_size="100"
internal_thread_pool.rejection_policy="Discard"
/>
<PING timeout="3000" num_initial_members="3"/>
<MERGE2 max_interval="30000" min_interval="10000"/>
<FD_SOCK/>
<FD_ALL timeout="15000" interval="3000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK2
xmit_interval="1000"
xmit_table_num_rows="100"
xmit_table_msgs_per_row="10000"
xmit_table_max_compaction_time="10000"
max_msg_batch_size="100"/>
<UNICAST3
xmit_interval="500"
xmit_table_num_rows="20"
xmit_table_msgs_per_row="10000"
xmit_table_max_compaction_time="10000"
max_msg_batch_size="100"
conn_expiry_timeout="0"/>
<pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="1m"/>
<pbcast.GMS print_local_addr="false" join_timeout="3000" view_bundling="true"/>
<tom.TOA/> <!-- the TOA is only needed for total order transactions-->
<UFC max_credits="2m" min_threshold="0.40" />
<MFC max_credits="2m" min_threshold="0.40" />
<FRAG2 frag_size="20k" />
<RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" />
</config>
--------------050008040007070308050505
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size: 13px;
font-style: normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<div class="moz-signature"><span style="color: rgb(0, 0, 0);
font-family: arial; font-size: 13px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;
display: inline !important; float: none;">I hope I'm using the
correct forum for this question.</span><br style="color: rgb(0,
0, 0); font-family: arial; font-size: 13px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">I originally found this problem with the
Infinispan's InitialStateTransfer but the problem is easily
reproduced with UUPerf.</span><br style="color: rgb(0, 0, 0);
font-family: arial; font-size: 13px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">When I run UUPerf with the default 4.5M data size
and 1 msg I see the following error mixed randomly in the log:</span><br>
<br>
<pre>14:16:27.607 ERROR (Invoker-1) [org.jgroups.protocols.TP.down(TP.java:1344)] JGRP000029: jfrost-multi-8014:failed sending message to jfrost-multi2-58507 (30079 bytes):
java.lang.Exception: dest=/192.168.10.193:58055 (30082 bytes),
headers: FRAG2: [id=1, frag_id=64, num_frags=151], UNICAST3: DATA, seqno=67, conn_id=1, UDP: [channel_name=uuperf]
</pre>
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">I tracked in with the debugger and found the root
cause is an IOException with the message "No buffers available"
returned by MulticastSocket.send().</span> <span style="color:
rgb(0, 0, 0); font-family: arial; font-size: 13px; font-style:
normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;"> I've seen other postings that the "No buffers
available" can be a bit of a red herring when it comes to the
exact problem(full ARP table under Linux for example). I'm
worried I'm chasing the wrong problem.</span><br style="color:
rgb(0, 0, 0); font-family: arial; font-size: 13px; font-style:
normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">I can post trace logs if they will help. From
the logs it also appears that the problem gets progressively
worse as buffer problems will do. So it could be that once the
error state is entered buffers don't get returned in a timely
manner.</span><br style="color: rgb(0, 0, 0); font-family:
arial; font-size: 13px; font-style: normal; font-variant:
normal; font-weight: normal; letter-spacing: normal;
line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">At first I thought the problem was specifically
the frag_size as the error seemed to occur at the frag_size
boundary. For frag_size=20K I would see errors with size
20082. But I was told by someone else that the size was
normal(data+overhead).</span><br style="color: rgb(0, 0, 0);
font-family: arial; font-size: 13px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">I've tried adjusting UDP buffers and
max_bundle_size, FRAG2 frag_size and UFC/MFC max_credits and
threshold. But I can't seem to find a magic combination of
values that will alleviate the problem.</span><br style="color:
rgb(0, 0, 0); font-family: arial; font-size: 13px; font-style:
normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">I note below that my test machines are VMWare
based but I've been able to try the test on hardware based
FreeBSD and the results are the same.</span><br style="color:
rgb(0, 0, 0); font-family: arial; font-size: 13px; font-style:
normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">My sysadmin did some checking and it doesn't seem
like the system buffer pool gets anywhere near depletion. The
systems aren't really do anything else besides my test. </span><br
style="color: rgb(0, 0, 0); font-family: arial; font-size: 13px;
font-style: normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">Any insights or suggestions are welcome.</span><br
style="color: rgb(0, 0, 0); font-family: arial; font-size: 13px;
font-style: normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">/Jack</span><br style="color: rgb(0, 0, 0);
font-family: arial; font-size: 13px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<hr style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;" size="2" width="100%"><br
style="color: rgb(0, 0, 0); font-family: arial; font-size: 13px;
font-style: normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">I'm running:</span><br style="color: rgb(0, 0, 0);
font-family: arial; font-size: 13px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px; -webkit-text-stroke-width:
0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;"> 2 nodes running FreeBSD hosted under
VMWare.<br>
Nodes are on the same LAN segment, no routers involved.<br
style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
</span><span style="color: rgb(0, 0, 0); font-family: arial;
font-size: 13px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px; display:
inline !important; float: none;"> jgroups 3.4.1 (The
problem show up with 3.3.1 as well but the error message is
slightly different, more explicit actually)</span><br
style="color: rgb(0, 0, 0); font-family: arial; font-size: 13px;
font-style: normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;"> openjdk 1.7.0_25</span><br style="color:
rgb(0, 0, 0); font-family: arial; font-size: 13px; font-style:
normal; font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<span style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; display: inline !important;
float: none;">This is one of the last configurations I tried.
With this config there seems to be a sensitivity to logging.
With UDP set to DEBUG it only takes one 4.5M RPC to show the
error. With TRACE it may not show up without raising the RPC
count.</span><br style="color: rgb(0, 0, 0); font-family: arial;
font-size: 13px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br style="color: rgb(0, 0, 0); font-family: arial; font-size:
13px; font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;">
<pre style="color: rgb(0, 0, 0); font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><config xmlns="urn:org:jgroups"
xmlns:xsi=<a class="moz-txt-link-rfc2396E" href="http://www.w3.org/2001/XMLSchema-instance">"http://www.w3.org/2001/XMLSchema-instance"</a>
xsi:schemaLocation="urn:org:jgroups <a class="moz-txt-link-freetext" href="http://www.jgroups.org/schema/JGroups-3.4.xsd">http://www.jgroups.org/schema/JGroups-3.4.xsd</a>">
<UDP
mcast_addr="${jgroups.udp.mcast_addr:228.6.7.8}"
mcast_port="${jgroups.udp.mcast_port:46655}"
tos="8"
ucast_recv_buf_size="32m"
ucast_send_buf_size="32m"
mcast_recv_buf_size="32m"
mcast_send_buf_size="32m"
loopback="true"
max_bundle_size="21k"
ip_ttl="${jgroups.udp.ip_ttl:2}"
enable_diagnostics="false"
bundler_type="old"
thread_naming_pattern="pl"
thread_pool.enabled="true"
thread_pool.min_threads="2"
thread_pool.max_threads="30"
thread_pool.keep_alive_time="60000"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="100"
thread_pool.rejection_policy="Discard"
oob_thread_pool.enabled="true"
oob_thread_pool.min_threads="2"
oob_thread_pool.max_threads="30"
oob_thread_pool.keep_alive_time="60000"
oob_thread_pool.queue_enabled="false"
oob_thread_pool.queue_max_size="100"
oob_thread_pool.rejection_policy="Discard"
internal_thread_pool.enabled="true"
internal_thread_pool.min_threads="1"
internal_thread_pool.max_threads="10"
internal_thread_pool.keep_alive_time="60000"
internal_thread_pool.queue_enabled="true"
internal_thread_pool.queue_max_size="100"
internal_thread_pool.rejection_policy="Discard"
/>
<PING timeout="3000" num_initial_members="3"/>
<MERGE2 max_interval="30000" min_interval="10000"/>
<FD_SOCK/>
<FD_ALL timeout="15000" interval="3000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK2
xmit_interval="1000"
xmit_table_num_rows="100"
xmit_table_msgs_per_row="10000"
xmit_table_max_compaction_time="10000"
max_msg_batch_size="100"/>
<UNICAST3
xmit_interval="500"
xmit_table_num_rows="20"
xmit_table_msgs_per_row="10000"
xmit_table_max_compaction_time="10000"
max_msg_batch_size="100"
conn_expiry_timeout="0"/>
<pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="1m"/>
<pbcast.GMS print_local_addr="false" join_timeout="3000" view_bundling="true"/>
<tom.TOA/> <!-- the TOA is only needed for total order transactions-->
<UFC max_credits="2m" min_threshold="0.40" />
<MFC max_credits="2m" min_threshold="0.40" />
<FRAG2 frag_size="20k" />
<RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" />
</config></pre>
<br>
</div>
</body>
</html>
--------------050008040007070308050505--
--===============7713029430147681085==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
------------------------------------------------------------------------------
WatchGuard Dimension instantly turns raw network data into actionable
security intelligence. It gives you real-time visual feedback on key
security issues and trends. Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
--===============7713029430147681085==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Javagroups-development mailing list
--===============7713029430147681085==--