Re: Parfait agent, docs, rpm updates

Hi,

On 2017-01-10 03:25, Nathan Scott wrote:
> ----- Original Message -----
> 
>>  Does there
>> exist a design that eventually will not rely on JVM-side regular
>> internal polling of JMX,
> 
> Parfait doesn't rely on JMX, so this statement doesn't really make sense.
> 
> FWLIW, there is no evidence that the memory copying that you're asserting is
> expensive, actually is.  This code is being used on many production machines
> and is frequently evaluated in load-testing environments ... the "polling"
> is in-process, so just involves in-memory copying - no syscalls, no IPC.

Let's try to be a bit more specific and look at concrete examples. We
have metrics like VmVersion which are static and never change during the
lifetime of the JVM but on the other side of the spectrum metrics like
OpenFileDescriptorCount are something that are calculated on demand
basis and since it's the OS/UNIX kernel which provides the fd
information, after calling the Java method [1] there will be a call to
the corresponding JVM native function which in turn calls the readdir(2)
system call. After some processing the current value is then propagated
through the layers to the caller, either over MMV or RMI.

When we're talking about one or few metrics the MMV vs RMI overhead
difference might be noticeable but as we already saw last year e.g. with
WildFly there are thousands of metrics available, so depending on how
much it takes effort to calculate the current values the overhead of
transporting the values may not be that relevant anymore. (We saw last
year that if querying for all the metrics from a WildFly instance its
CPU usage goes momentarily trough the roof and that's certainly not due
to RMI/socket traffic.) In general, I'd guesstimate that most metrics
cost-wise are somewhere between VmVersion and OpenFileDescriptorCount.

Considering the above, with pure JMX the overhead is easy to understand:
if JMX is disabled or JMX is enabled but nobody is polling for the
metrics, then there is no notable overhead. If I start to poll those
metrics (with Jconsole or pmval -t ... or such) the values are
calculated at the internal configured on the client (and that can be
naturally changed trivially on the client side).

It's still a bit unclear to me how Parfait/MMV gets values like open fd
count which need to be calculated and merely poking a memory area does
not give the current/real value? Does the query for the metric (from
outside of Parfait) trigger a request to calculate the value which can
then be accessed over MMV? Or is there an interval which needs be
defined to get the values updated (even if nobody is polling for them)?
How is Parfait/JMX different in this regard?

1)
https://docs.oracle.com/javase/8/docs/jre/api/management/extension/com/sun/management/UnixOperatingSystemMXBean.html

>> regardless of the presence or absence of PCP/MMV consumers of the data?
> 
> This could certainly be done in the existing Parfait, yes, if there was a
> need (but there is not AFAIK).  The application could be poked independently
> of the PCP infrastructure to tune any refresh interval, and/or set of values
> exported.

So, continuing with the above example, I'd hate to see a situation where
we keep updating OpenFileDescriptorCount and hundreds of other metrics,
say, every 10 seconds for days and weeks even if nobody is polling for
them. Also, what happens if there is a crisis and I determine I need to
see the updated values every one second instead?

> However noone has ever requested this and its not something that tends to be
> needed in real world environments, as these tend to use an "always-on" style
> of monitoring of applications in my experience.

It's also a bit unclear what kind of "real world environments" we're
talking about here; after all, it seemed to come as a surprise to all of
us that the agent didn't work with WildFly which is very, very much used
in real world environments in my experience.

Thanks,

-- 
Marko Myllynen

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links:

You receive all messages sent to this group.

View/Reply Online (#14963): https://groups.io/g/pcp/message/14963
View All Messages In Topic (9): https://groups.io/g/pcp/topic/2984744
Mute This Topic: https://groups.io/mt/2984744?uid=174580
New Topic: https://groups.io/g/pcp/post
-=-=-
pcp mailing list
[email protected]
https://groups.io/g/pcp/messages
-=-=-
Change Your Subscription: https://groups.io/g/pcp/editsub?uid=174580
Group Home: https://groups.io/g/pcp
Contact Group Owner: [email protected]
Terms of Service: https://groups.io/static/tos
Unsubscribe: https://groups.io/g/pcp/leave/354243/563757577/xyzzy
-=-=-=-=-=-=-=-=-=-=-=-