Re: [jgroups-users] JGroups-3.6.6-Final causes memory usage to grow gradually even when there is no traffic/activity (for a single node cluster)
Questions/problems related to using JGroups <[email protected]>
| Newsgroups | gmane.comp.java.javagroups.general |
|---|---|
| Message-ID | <[email protected]> |
Your configuration looks ok. Memory exhaustion can happen when people
mistakenly remove or comment STABLE.
I suggest enable TRACE on org.jgroups. If the node sits there just
idling, then all you should see is periodic STABLE, MERGE3 and FD activity.
When you connect a memory profiler to the process (e.g. jvisualvm), what
does it show as culprit? Are you sure it's JGroups? Or could it be your
application?
JGroups 3.6.x is used in hundreds if not thousands of installations
world-wide; if there's a memory leak, a lot of people would have noticed
and filed bug reports when 3.6.6 was released, but this wasn't the case.
I just ran 2 nodes (UPerf) on the 3.6 branch (3.6.10-SNAPSHOT); each
node sending 50 million 8K messages, and memory was stable:
invoking 50000000 RPCs of 8KB, sync=true, oob=false, msg_bundling=true
.........
done (in 1315752 ms)
======================= Results: ===========================
belasmac-15961: 37995.48 reqs/sec (39499863 gets, 10500137 puts, get RTT
556.41 us, put RTT 1036.25 us)
belasmac-4716: 38001.08 reqs/sec (39503180 gets, 10496820 puts, get RTT
557.06 us, put RTT 1034.77 us)
Throughput: 37998.28 reqs/sec/node (303.99MB/sec)
Roundtrip: gets avg = 556.73 us, puts avg = 1035.51 us
On 22/06/16 00:53, Questions/problems related to using JGroups wrote:
> Hello,
>
> I'm relatively new to JGroups. I've successfully got it working in our
> solution.
> However, recently we noticed that memory usage continuously grows and
> reaches maximum available memory approx in a week and then crashes due
> to lack of memory.
>
> I'm running 3.6.6-Final version.
>
> I tried to reproduce the same issue with just one node and absolutely no
> data activity. Even with no data activity - just starting the
> application (a single node) and waiting, I can see the memory usage
> gradually grow in size and take max allocated memory for the application.
>
> When I comment out JChannel.connect(..) in the code (meaning am not
> connecting my single node to a cluster at all and therby not forming a
> cluster), memory of the application remains stable.
>
> I'm using MPING in my config for node discovery.
>
> The following is my config:
> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns="urn:org:jgroups" xsi:schemaLocation="urn:org:jgroups <a href="
> http:="" www.jgroups.org <http://www.jgroups.org>="" schema=""
> jgroups.xsd"="">http://www.jgroups.org/schema/jgroups.xsd">
> <TCP bind_port="7800"
> recv_buf_size="${tcp.recv_buf_size:5M}"
> send_buf_size="${tcp.send_buf_size:5M}"
> max_bundle_size="64K"
> max_bundle_timeout="30"
> use_send_queues="true"
> sock_conn_timeout="300"
>
> timer_type="new3"
> timer.min_threads="4"
> timer.max_threads="10"
> timer.keep_alive_time="3000"
> timer.queue_max_size="500"
>
> thread_pool.enabled="true"
> thread_pool.min_threads="2"
> thread_pool.max_threads="8"
> thread_pool.keep_alive_time="5000"
> thread_pool.queue_enabled="true"
> thread_pool.queue_max_size="10000"
> thread_pool.rejection_policy="discard"
>
> oob_thread_pool.enabled="true"
> oob_thread_pool.min_threads="1"
> oob_thread_pool.max_threads="8"
> oob_thread_pool.keep_alive_time="5000"
> oob_thread_pool.queue_enabled="false"
> oob_thread_pool.queue_max_size="100"
> oob_thread_pool.rejection_policy="discard"/>
>
> <MPING ip_ttl="8"
> mcast_addr="224.1.2.3"
> mcast_port="7801"
> send_on_all_interfaces="true"
> receive_on_all_interfaces="true"/>
>
> <MERGE3 min_interval="10000"
> max_interval="30000"/>
> <FD_SOCK client_bind_port="40718" start_port="40718"/>
> <FD timeout="3000" max_tries="3" />
> <VERIFY_SUSPECT timeout="1500" />
> <BARRIER />
> <!--<pbcast.NAKACK2 use_mcast_xmit="false"
> discard_delivered_msgs="true"/> -->
> <pbcast.NAKACK2 xmit_interval="500"
> xmit_table_num_rows="10"
> xmit_table_msgs_per_row="200"
> xmit_table_max_compaction_time="30000"
> max_msg_batch_size="500"
> use_mcast_xmit="false"
> discard_delivered_msgs="true"/>
> <UNICAST3 xmit_interval="500"
> xmit_table_num_rows="10"
> xmit_table_msgs_per_row="200"
> xmit_table_max_compaction_time="60000"
> conn_expiry_timeout="0"
> max_msg_batch_size="500"/>
> <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
> max_bytes="4M"/>
> <pbcast.GMS print_local_addr="true" join_timeout="5000"
> view_bundling="true"/>
> <MFC max_credits="2M"
> min_threshold="0.4"/>
> <FRAG2 frag_size="60K" />
> <!--RSVP resend_interval="2000" timeout="10000"/-->
> <pbcast.STATE_TRANSFER/>
>
> </config>
>
> It would be very helpful if you could take a look at this issue and
> share some your thoughts on what could be going wrong.
>
> I appreciate your help .
>
> Thank You.
>
>
> --
> *Regards,
> *Venkat Subramanian
>
>
>
>
> ------------------------------------------------------------------------------
> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
> Francisco, CA to explore cutting-edge tech and listen to tech luminaries
> present their vision of the future. This family event has something for
> everyone, including kids. Get more information and register today.
> http://sdm.link/attshape
>
>
>
> _______________________________________________
> javagroups-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>
--
Bela Ban, JGroups lead (http://www.jgroups.org)
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape