Re: [jgroups-users] OOM issues?

Thanks for the help on this -- I know it's not a well defined question!

On Tue, Oct 30, 2018 at 3:46 AM Questions/problems related to using JGroups
<[email protected]> wrote:

>
>
> You said 'a simple test of our code': does this occur with a simple
> cluster of standalone JGroups nodes (e.g. Draw) too?
>

I haven't tried that, and am not sure how I'd reproduce my stack in code
with it (I haven't looked at Draw in forever). To help narrow things down
instead of jumping into looking at everything in memory, will write a
standalone app that does nothing but connect and then 'main' exits. That's
basically what my 'simple' case does, a "witness" node in a database
monitoring system, but can simplify it further in case the problem is
somewhere else in my code. I have no reason to think there's a problem in
jgroups; but wanted to ask in case anyone on this list was seeing problems
these days that they didn't use to.


> There's no way around attaching jmc to a running node and seeing which
> class increases memory. JConsole might help, too, but jmc has more
> detailed info.
>

Will look into that, thanks. I'm getting jstat info from customers, so
should get familiar with that also.


>
> > Anyway, am just curious for any stories of similar things with recent
> > JDK versions or some random thing I might have missed. Our jchannel
> > stack (which I can share again) is unchanged for a while now.
>
> Yes, please post it. A simple misconfig (e.g. omitting STABLE) could
> easily lead to this...
>

 Added below, thanks. Would be great if that's the problem, since what
we're seeing is mostly with our code after we made the jgroups 3->4 change
and we could have messed it up.



> The only thing you may have to look into is the GC configuration, but I
> don't think this is the root cause.
>

The only param we run with is -Xmx32m, now 128m. Wondering if we should be
running with "-server" also. It's been > decade since I've thought about
that kind of thing.

Thanks,
Bobby

Our stack, all inline comments added for this email. Note that we do have a
custom Auth protocol here, but I'm seeing the memory rise with a stable set
of nodes -- once everyone joins I just leave it alone.

        List<Protocol> stack = new ArrayList<>();
        stack.add(new TCP()
            // method below pulls address out of a Properties object
            .setValue("bind_addr",
InetAddress.getByName(getBindingAddress()))
            .setValue("bind_port", bindingPort)
            .setValue("thread_pool_min_threads", 1)
            .setValue("thread_pool_keep_alive_time", 5000)
            .setValue("send_buf_size", 640000)
            .setValue("sock_conn_timeout", 300)
            .setValue("recv_buf_size", 5000000));
        stack.add(new TCPPING()
            // method below returns List<IpAddress> from a config file
            .setValue("initial_hosts", parseHostList())
            .setValue("send_cache_on_join", true)
            .setValue("port_range", 0));
        stack.add(new MERGE3()
            .setValue("min_interval", 10000)
            .setValue("max_interval", 30000));
        stack.add(new FD_ALL()
            // method below returns Integer.parseInt (for some reason)
            // from Properties object
            .setValue("timeout", getJGroupsTotalTimeout() * 1000L));
        stack.add(new VERIFY_SUSPECT()
            .setValue("timeout", 1500));
        stack.add(new BARRIER());
        stack.add(new NAKACK2()
            .setValue("use_mcast_xmit", false));
        stack.add(new UNICAST3());
        stack.add(new STABLE()
            .setValue("desired_avg_gossip", 50000)
            .setValue("max_bytes", 4000000));
        // method below returns an AUTH protocol, 'param' changed here is
        // object that extends ReceiverAdapter implements RequestHandler
        stack.add(createAuthProtocol(param));
        stack.add(new GMS()
            .setValue("join_timeout", 3000));
        stack.add(new MFC()
            .setValue("max_credits", 2000000)
            .setValue("min_credits", 800000));
        stack.add(new FRAG2());
        stack.add(new STATE_TRANSFER());
        return new JChannel(stack);

_______________________________________________
javagroups-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/javagroups-users