Re: Parfait agent, docs, rpm updates

Hi,

On 2017-01-11 12:48, Nathan Scott wrote:
> ----- Original Message -----
>>> [...]
>> Let's try to be a bit more specific and look at concrete examples.
> 
> *nod* in the Aconex environment, hundreds of metrics (& tens of thousands of
> values) have been used with good success for many years, so there's existence
> proof this is readily achievable.  And that's actual, 24x7, 365-days-a-year
> use, solving actual production performance problems - not just a few queries
> of JMX values by us developers poking at JVMs and guesstimating ...

I wonder what are these metrics, since currently with what we have here
we only get a dozen or so metrics? If they have hundreds of metrics why
don't we have even full JVM coverage?

>> (We saw last
>> year that if querying for all the metrics from a WildFly instance its
>> CPU usage goes momentarily trough the roof
> 
> OK - what was that due to though?

It was never investigated in detail as the overall discussion
concentrated in other areas. I think we can continue this investigation
when we get something out of WildFly with the current/emerging tooling.

>> and that's certainly not due to RMI/socket traffic.)
>             ^^^^^^^^^^^
> How are we certain?  What *is* it from then, OOC?  (needs someone to go
> profile it methinks, to be certain about what it is or is not)

Based on general experience sending a limited amount of data over a
socket doesn't cause the process to eat huge amount of CPU. But of
course there might be something else going on before the actual socket
operation so I agree that we'd need to dig a bit deeper to be 100% certain.

>> In general, I'd guesstimate that most metrics
>> cost-wise are somewhere between VmVersion and OpenFileDescriptorCount.
> 
> We can't guesstimate when doing performance analysis...

In this case I think it's pretty fair assumption that most metrics are
somewhere between a) a metric which is a completely static,
never-changing string (=VmVersion) and b) a metric that needs to go down
all the way to the kernel to get the value (=OpenFileDescriptorCount).

> From the Aconex environment, using Parfait & MMV, querying metric values
> introduces no measurable overhead and this includes all the JMX values
> we access now in parfait-agent, and a bunch of application-specific ones
> that we of course don't.  This has been measured, alot - there is a bunch
> of code in Parfait for benchmarking itself, too - not to mention the load
> testing they do regularly.

You said we should not guesstimate but how do you know e.g. WildFly
metrics won't cause measurable overhead if it hasn't even been tested?
What other OSS components were in use in addition to JVM and Tomcat?

>> How is Parfait/JMX different in this regard?
>> [...] Also, what happens if there is a crisis and I determine I need to
>> see the updated values every one second instead?
> 
> Again from that last mail, this would not be difficult to add to Parfait.
> All the ground work is there, the intervals are already parameterised &
> already use Java system properties, so easily externalised.  I've not seen
> evidence this dynamic-live-sampling is needed though so I'll certainly not
> be working on it.
> 
> And again, some other agent talking to Parfait could be used to drive those
> intervals, (or MMV could be extended, or...) but someone who believes in it
> would need to step forward to do that work.  For me, at this stage, it's a
> hypothetical problem and there are many other real problems to be worked on.

Quoting the other mail:

> Have a read of the man page and see why.

I don't see it, care to explain? The man page version I have says:

       -i/--interval
            Delay between sampling (JMX values in particular) to
            refresh the values exported to PCP.

Since in the default mode this is defined at JVM startup, how you would
change this later on?

>> It's also a bit unclear what kind of "real world environments" we're
>> talking about here; after all, it seemed to come as a surprise to all of
>> us that the agent didn't work with WildFly which is very, very much used
>> in real world environments in my experience.
> 
> Those are, unsurprisingly, Red Hat centric real world environments though.

WildFly / JBoss are being used by tens of thousands of organizations
around the world. Parfait is being used at Aconex, anywhere else?

> Parfait came from the Aconex production environment, long before Red Hat
> interest in PCP - and its (was at the time) a Tomcat-based Java environment
> with a variety of JVM versions over the years.  It is the best historical
> test bed we have - we don't need to make up performance problems, they've
> seen them all already.

Sorry, but how can you say "they've seen them all already" if they even
don't use some of the most ubiquitous components in the middleware market?

New paradigms and technologies are emerging faster than ever, thinking
that "we know it all already" is not the way to thrive in such of an age.

> Or not.  As I said - if the current parfait-agent is not floating peoples
> boats - those people could go right ahead and extend it to suit, to use
> different backends, make it more dynamic/adaptable - anything is possible
> with a little creativity and duct tape.

Going back to more concrete level, what are the short/long term plans
for improvements around this area, I don't see anything in Trello?

Thanks,

-- 
Marko Myllynen

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links:

You receive all messages sent to this group.

View/Reply Online (#14966): https://groups.io/g/pcp/message/14966
View All Messages In Topic (12): https://groups.io/g/pcp/topic/2984744
Mute This Topic: https://groups.io/mt/2984744?uid=174580
New Topic: https://groups.io/g/pcp/post
-=-=-
pcp mailing list
[email protected]
https://groups.io/g/pcp/messages
-=-=-
Change Your Subscription: https://groups.io/g/pcp/editsub?uid=174580
Group Home: https://groups.io/g/pcp
Contact Group Owner: [email protected]
Terms of Service: https://groups.io/static/tos
Unsubscribe: https://groups.io/g/pcp/leave/354243/563757577/xyzzy
-=-=-=-=-=-=-=-=-=-=-=-