pcp updates - merge kenj, mgoodwin, dave, liming, marko, nathan
"Mark Goodwin" <[email protected]>
| Newsgroups | gmane.comp.sysutils.pcp |
|---|---|
| Message-ID | <[email protected]> |
This is pretty much it for 3.6.11, apart from any QA updates and release
docs (changelog etc). There will be some QA updates for sure - I'm seeing a
failure in the new pmrep qa test (1069).
Note - I actually only merged collectl2pcp and Marko's pmrep updates. The
rest were merged earlier by Nathan but picked up by the push script.
Changes committed to git://git.pcp.io/pcp master
Ken McDonell (16):
src/pmlogrewrite/pmlogrewrite.c: fix small error in temporal index
src/libpcp/src/auxconnect.c: fix diagnostic messages
Revert "src/pmlogrewrite/pmlogrewrite.c: fix small error in temporal
index"
src/pmcd/pmdaproc.sh: change default $signal_delay
src/libpcp/src/pmns.c: change logic in __pmHasPMNSFileChanged()
qa/827: tweak to handle changes in temporal index
src/libpcp/src/pmns.c: correct logic in __pmHasPMNSFileChanged()
src/pmcd/pmdaproc.sh: tweak signal sending logic
qa/964: add some diagnostics to $seq.full
src/libpcp/src/optfetch.c: changes to diagnostics
src/libpcp/src/pdu.c: add assert() for pduread()
qa/759: unbound PMDA no longer forces pmcd restart
qa/041: another case of needing to wait for pmcd _and_ pmlogger
qa/365: add some more checking at the end for pmcd and primary
pmlogger
src/libpcp/src/auxserver.c: be a little more tolerant of EADDRINUSE
src/libpcp/src/context.c: tweak to __pmCloseChannelbyFd() and
__pmCloseChannelbyContext()
Mark Goodwin (2):
collectl2pcp: detect if input data in collectl's "plot" format.
pmdalinux: update units of segsz shm metric to bytes
Dave Brolley (1):
RHBZ 1348349: pmchart(1): Encapsulate Table-Population code.
Liming Wu (1):
pmdalinux: additional shared memory segment metrics
Marko Myllynen (1):
pmrep: add -X "label" flag for instance-per-line output mode
Nathan Scott (1):
pmdalinux: further tweaks to the new shm metrics
qa/041 | 1
qa/1069 | 2
qa/1069.out | 124 +++++++++++++++++++++++++
qa/365 | 28 ++++-
qa/759.out | 3
qa/827 | 9 +
qa/827.out | 13 ++
qa/888 | 7 +
qa/888.out | 38 ++++++-
qa/964 | 5 -
qa/admin/pcp-daily | 2
qa/src/GNUlocaldefs | 9 +
qa/src/qa_shmctl_stat.c | 26 +++++
src/bashrc/pcp_completion.sh | 2
src/collectl2pcp/header.c | 11 ++
src/libpcp/src/auxconnect.c | 8 -
src/libpcp/src/auxserver.c | 22 +++-
src/libpcp/src/check-statics | 1
src/libpcp/src/context.c | 25 ++++-
src/libpcp/src/optfetch.c | 44 +++++---
src/libpcp/src/pdu.c | 7 +
src/libpcp/src/pmns.c | 85 +++++++++--------
src/pmcd/pmdaproc.sh | 19 +++
src/pmchart/chart.cpp | 81 ----------------
src/pmchart/chart.h | 3
src/pmchart/metricdetails.cpp | 83 ++++++++++++++++
src/pmchart/metricdetails.h | 22 ++--
src/pmdas/linux/clusters.h | 1
src/pmdas/linux/help | 6 +
src/pmdas/linux/indom.h | 1
src/pmdas/linux/ipc.c | 196
+++++++++++++++++++++++++++-------------
src/pmdas/linux/ipc.h | 40 ++++++--
src/pmdas/linux/pmda.c | 87 ++++++++++++++++-
src/pmdas/linux/root_linux | 6 +
src/pmlogrewrite/pmlogrewrite.c | 8 -
src/pmrep/TODO | 3
src/pmrep/pmrep.1 | 17 ++-
src/pmrep/pmrep.conf | 1
src/pmrep/pmrep.conf.5 | 9 +
src/pmrep/pmrep.py | 186
+++++++++++++++++++++++++++++++++++++
src/zshrc/_pcp | 1
41 files changed, 983 insertions(+), 259 deletions(-)
Details ...
commit 78a881152382bf37234d6fddd3e5f56677523841
Author: Marko Myllynen <[email protected]>
Date: Wed Nov 9 19:47:51 2016 +1100
pmrep: add -X "label" flag for instance-per-line output mode
This adds the -X flag for instance-per-line mode. e.g.
$ pmrep -s3 -r -X network.interface.in.bytes network.interface.out.bytes
n.i.i.bytes n.i.o.bytes
byte byte
lo 56168 56168
virbr0-n 0 0
virbr0 27350 16350
wlan0 127503211 16790503
tun0 3661355 1188515
lo 56168 56168
virbr0-n 0 0
virbr0 27350 16350
wlan0 127503211 16790503
tun0 3661355 1188515
lo 56168 56168
virbr0-n 0 0
virbr0 27350 16350
wlan0 127503211 16790503
tun0 3661355 1188515
$
Also allows easier grepping (if -i is not feasible) and emulating some
other utilities more closer, e.g., nicstat(1):
https://github.com/superjamie/pcp/commit/c6969807a7dfdf2bce6f3b1dff0ec1084bb947f1
Passes existing QA, new QA case added for this.
Caching results is needed, something like "pmrep -s1 -X proc" took
half a minute without caching, with caching around one second.
Author: Marko Myllynen <[email protected]>
commit 5af2e05a6639362e122cfbef0ffed7c7190bc394
Author: Nathan Scott <[email protected]>
Date: Wed Nov 9 15:15:24 2016 +1100
pmdalinux: further tweaks to the new shm metrics
Some array sizing bounds checking, some metrics should be
instantaneous, small code consistency changes and added a
set of comments for each field in the shm_stat_t struct -
test qa/888 updated too.
commit 18643421adb6d3a21f62ad38a4018bb42b516803
Author: Mark Goodwin <[email protected]>
Date: Wed Nov 9 13:30:23 2016 +1100
pmdalinux: update units of segsz shm metric to bytes
commit f2fa7e2989473ac1ab5c2ab6f7db882c02cc2299
Author: Liming Wu <[email protected]>
Date: Wed Nov 9 13:21:20 2016 +1100
pmdalinux: additional shared memory segment metrics
commit 7579871de03a56df6c4a026a9b299764bad696c9
Author: Dave Brolley <[email protected]>
Date: Fri Nov 4 11:49:32 2016 -0400
RHBZ 1348349: pmchart(1): Encapsulate Table-Population code.
The code which poplulates the table of metrics is now encapsulated
in the MetricDetailsWindow class.
commit 02b81c8d8acccd685b88c5e3426da4afb6e4f633
Author: Mark Goodwin <[email protected]>
Date: Fri Nov 4 12:26:51 2016 +1100
collectl2pcp: detect if input data in collectl's "plot" format.
Only "raw" format is supported. Throw a fatal error if "plot" format is
detected.
commit 33a9819165576b12dffc2560891f4313eae153b5
Author: Ken McDonell <[email protected]>
Date: Fri Nov 4 07:03:26 2016 +1100
src/libpcp/src/context.c: tweak to __pmCloseChannelbyFd() and
__pmCloseChannelbyContext()
These routines get called from multiple places because __pmGetPDU()
is used in lots of places, but it really only makes sense to clobber
to the underlying socket in cases where the caller is a PMAPI client.
So the "close socket" part is NOT done if __pmGetInternalState()
returns PM_STATE_PMCS, which means any socket between pmcd (or dbpmda)
and a PMDA is not closed if a socket read timeout leads us to the
__pmCloseChannelby*() routines. In these cases, the higher level
error handling is sufficient.
Of course, if Frank's un-multiplexing of the client-pmcd socket on
the client side is fully implemented, all of this code is likely to
be culled because the motivation will have been removed.
commit 2261fc7af299a63b93d3df20aeabe5f42bf93fad
Author: Ken McDonell <[email protected]>
Date: Fri Nov 4 06:53:11 2016 +1100
src/libpcp/src/auxserver.c: be a little more tolerant of EADDRINUSE
With recent changes to the daemon restarting logic to speed things up
we seem to have opened a window where we try to bind to a specific
port (e.g. pmcd's client connection port) before the last use has
been fully torn down, resulting in EADDRINUSE errors on restart.
The problem seems particularly bad on my QA vm19 running x86_64
openSUSE 12.2 ... but that may have more to do with this VM than
this distro.
The change here is to add a small delay (of between 1 and 4 x 250msec)
when bind() returns EADDRINUSE ... this seems to be enough to let the
previous socket usage be completely cleaned up and the new daemon is
off to the races. Most calls see no delay. Non-transient problems
now take 4 failed attempts and 1 second longer to report EADDRINUSE.
commit 4d115e9f460117e052bda70aa1df44a7afe8b2b4
Author: Ken McDonell <[email protected]>
Date: Fri Nov 4 06:52:30 2016 +1100
qa/365: add some more checking at the end for pmcd and primary pmlogger
commit a44fd60f0a4d15551ca53022172443cea32ac1d0
Author: Ken McDonell <[email protected]>
Date: Fri Nov 4 06:51:48 2016 +1100
qa/041: another case of needing to wait for pmcd _and_ pmlogger
commit ac28f151eb69ce0ed1f45f091b66f6c315683c16
Author: Ken McDonell <[email protected]>
Date: Thu Nov 3 10:30:55 2016 +1100
qa/759: unbound PMDA no longer forces pmcd restart
Revised 759.out.
commit 4011f44baec30e055aeda72b2f07e05d7c2756bd
Author: Ken McDonell <[email protected]>
Date: Wed Nov 2 10:13:05 2016 +1100
src/libpcp/src/pdu.c: add assert() for pduread()
To help isolate some recent problems (with pmlogger in particular),
we'll abort now if pduread() is called with a file descriptor <0 ...
this should never happen if the world is on good shape.
commit 2ce8f897767469bdcc4a2fec46146ce74a5a47c9
Author: Ken McDonell <[email protected]>
Date: Wed Nov 2 10:10:03 2016 +1100
src/libpcp/src/optfetch.c: changes to diagnostics
Previous diagnostics were guarded by #ifdef DESPERATE which is really
not too useful.
Changed this to run-time guarding requiring -Doptfetch and -Ddesperate.
Also added some diagnostics for __pmOptFetchDel() that apparently missed
the diagnostics boat when it first sailed.
commit 3d612b531436528b4197917265de8ae03b794353
Author: Ken McDonell <[email protected]>
Date: Wed Nov 2 10:09:09 2016 +1100
qa/964: add some diagnostics to $seq.full
Trying to track down a "sometimes on some hosts" failure.
commit 20b38628d639436582a51aff4206cab71ee45d6c
Author: Ken McDonell <[email protected]>
Date: Wed Nov 2 10:02:48 2016 +1100
src/pmcd/pmdaproc.sh: tweak signal sending logic
Nathan's last change made delivery of SIGHUP more robust.
Unfortunately it widened the window between pmcd getting the SIGHUP
and pmcd doing the post-SIGHUP work (this is delayed in pmcd so it does
not run in a signal handler, i.e. interrupt, context). This increased
the chance that the PMDA install script would fail to find the recently
installed PMDA's metrics because pmcd had not yet finished its work.
After the loop checking for the delivery of the SIGHUP, re-instate
a pmsleep $signal_delay to compensate, and this has the side-effect
that it can be over-ridden from the PMDA's Install script if needs be
(which reverts to the former behaviour).
commit ee24830c9834ac02702faf13dd4945ea284114f8
Author: Ken McDonell <[email protected]>
Date: Sat Oct 29 06:44:56 2016 +1100
src/libpcp/src/pmns.c: correct logic in __pmHasPMNSFileChanged()
One of the predicates was rewritten incorrectly in the last change,
also rework the diagnostics so the result of __pmHasPMNSFileChanged()
is also reported under the control of DBG_TRACE_PMNS.
commit ad8f1c13e4c93c9ae5e9af1ba8c7292d24533208
Author: Ken McDonell <[email protected]>
Date: Fri Oct 28 09:47:17 2016 +1100
qa/827: tweak to handle changes in temporal index
After pmlogrewrite, these are now expected.
commit 8b39516e1ec4cbf9bf3a1423094141d0ce24350e
Author: Ken McDonell <[email protected]>
Date: Fri Oct 28 08:56:01 2016 +1100
src/libpcp/src/pmns.c: change logic in __pmHasPMNSFileChanged()
Changes elsewhere have meant that pmcd reconfiguration step are
now significantly quicker (this is good), but this has exposed a latent
design error in __pmHasPMNSFileChanged().
The check for "has the pmns changed" is based on stat()ing the root
PMNS file and checking the mtime ... unfortunately this has only second
resolution on some systems, which was OK when PMDA Install/Remove was
slow ... but not now. Improved the logic to check the file size and
the mtime.
Also needed another static inside the lock guard, so update
check-statics
accordingly.
commit 66d805b8ed9b845559aeb39a2a12a3dc644b4b89
Author: Ken McDonell <[email protected]>
Date: Fri Oct 28 08:53:41 2016 +1100
src/pmcd/pmdaproc.sh: change default $signal_delay
I've found 0.1sec to be unreliable on some QA systems and setting
this to 1sec seems to improve the situation dramatically.
Given the vaguaries of signal delivery, this is not altogether
surprising.
commit 540a5b084b640ff1d57f1808663838355665e865
Author: Ken McDonell <[email protected]>
Date: Tue Oct 25 08:47:23 2016 +1100
Revert "src/pmlogrewrite/pmlogrewrite.c: fix small error in temporal
index"
This reverts commit d35d7efe161fdfe1d9198a02645d44cfa2f49631.
commit 800d6b80562333aa343e0eeb22fbf3d6a1a236b2
Author: Ken McDonell <[email protected]>
Date: Tue Oct 25 08:39:14 2016 +1100
src/libpcp/src/auxconnect.c: fix diagnostic messages
Message contained wrong routine name.
commit d35d7efe161fdfe1d9198a02645d44cfa2f49631
Author: Ken McDonell <[email protected]>
Date: Tue Oct 25 08:37:37 2016 +1100
src/pmlogrewrite/pmlogrewrite.c: fix small error in temporal index
Metadata offset arithmetic was not quite right ... Nathan spotted
this in qa/827.