Re: RFC: metric labels as first-class PCP concept

nathans wrote:

> [...]  OK, see attached patch - v2.  This is a fairly large update,
> starting to spec out more of the PMDA side of things now as well and
> incorporate the feedback so far.

Thank you, you're obviously working hard on this.  You have sketched out
more aspects of the representation and a couple aspects of data flow.

> (not yet profiles or archives though)

The profiles part is important, because it would encompass ahead-of-time
filtering.  I'd like to see not really the detailed proposed API, but
the control/data flow you envision.  In what form would a client
indicate its interest or disinterest in labels?  Who would evaluate
those conditions, and when? 


> New design & documentation in this patch ...
>
> - outline the initial PMDA interfaces
> - extends the initial PMAPI interfaces
> [...]

If I read & imagine correctly, in this model, labels are kind of like
instantaneous metric values, in the sense that a pmapi client must
request a snapshot of label values whenever it is interested, and the
several pmdas would then compute & return json blocks.

This looks very simple, which is good.  But how do you imagine this
being used by tools other than trivial one-time-dumper tools like
pminfo?  Like pmchart or pmlogger or pmval?  Surely they are not simply
to call pmLookupLabels() after every pmFetch() - that'd be huge new
overhead.  Or are you imaginging the PMCD_LABEL_CHANGE pmFetch
result-flag indicating --any-- labeling change?

If the latter, how do you imagine pmcd/pmdas would compute that flag?
As I understand the model, labels seen by a client wouldn't persist
anywhere within pmcd proper, nor is a history of metrics a client has
seen.  Nor would a PMDA notify anyone about a label change it initiates,
if all it does with labels is returns a current set on demand.  So I'm
really not sure how you imagine this working, but it's a critical part
so please describe it.

For comparison with the alternative model I started to outline, if one
were to treat labels at the wire-protocol / pmcd / pmda level as
first-class SEM_DISCRETE string metrics, then we can ask pmlogger & pals
to fetch those labeling PMIDs along with the payload PMIDs, every time.
A PMDA can answer numval=0 if the metric-level labels for that metric
have not changed for that client, etc.


> - switches space-separated name=value pairs to JSONB

What does "JSONB" mean in this context?  The textual representation of
JSONB (as used in e.g. postgresql) is JSON, which are exactly what your
examples show.  There is no "json binary" file format in common use, and
such would be an impediment to anyone who tried to create these
pcp/label.d files.  Even in postgresql, the indexable JSONB
representation is an internal implementation detail, and is used in
documentation only to *explain* certain data-non-preservation
consequences (whitespace, key ordering).

I suggest sticking to plain JSON in all of our text, and probably
ditching the "llen" length field (== strlen(labels) anyway).


> - support for user-defined labels via a pmlabel(1) command

I don't understand how you envision this command being used.  It looks
much like a sibling of pmstore, which is something that apprx. only the
pcpqa uses now.  Is it a one-time machine provisioning step run by a
sysadmin - but one that couldn't instead set pcp/labels.d files?  Is it
by ordinary users - but then they can interfere.  Is it for use by
tooling - authenticated how?


> - worked examples of data structure layouts for the new PMAPI
>   interfaces, and handling multi-dimensional instance domains
>   in the PMDA interfaces.  \o/

Regardless of anything else, I am glad that generalization of
single-dimensional indoms is being pondered!


- FChE

-=-=-=-=-=-=-=-=-=-=-=-
pcp mailing list
[email protected]
https://groups.io/g/pcp/messages
-=-=-
Groups.io Links:

You receive all messages sent to this group.

View/Reply Online (#14867): https://groups.io/g/pcp/message/14867
View All Messages In Topic (18): https://groups.io/g/pcp/topic/3188520
Mute This Topic: https://groups.io/mt/3188520?uid=174580
New Topic: https://groups.io/g/pcp/post

Change Your Subscription: https://groups.io/g/pcp/editsub?uid=174580

Group Home: https://groups.io/g/pcp
Contact Group Owner: [email protected]

Terms of Service: https://groups.io/static/tos

Unsubscribe: https://groups.io/g/pcp/leave/354243/563757577/xyzzy
-=-=-=-=-=-=-=-=-=-=-=-