pmrep: add support for swapping columns and rows in stdout output
"Marko Myllynen" <[email protected]>
| Newsgroups | gmane.comp.sysutils.pcp |
|---|---|
| Organization | Red Hat |
| Message-ID | <[email protected]> |
Hi,
Add an alternative output mode for pmrep:
$ pmrep -s3 -r -X network.interface.in.bytes network.interface.out.bytes
n.i.i.bytes n.i.o.bytes
byte byte
lo 56168 56168
virbr0-n 0 0
virbr0 27350 16350
wlan0 127503211 16790503
tun0 3661355 1188515
lo 56168 56168
virbr0-n 0 0
virbr0 27350 16350
wlan0 127503211 16790503
tun0 3661355 1188515
lo 56168 56168
virbr0-n 0 0
virbr0 27350 16350
wlan0 127503211 16790503
tun0 3661355 1188515
$
Also allows easier grepping (if -i is not feasible) and emulating some
other utilities more closer, e.g., nicstat(1):
https://github.com/superjamie/pcp/commit/c6969807a7dfdf2bce6f3b1dff0ec1084bb947f1
Passes existing QA, new QA case added for this.
Caching results is needed, something like "pmrep -s1 -X proc" took
half a minute without caching, with caching around one second.
---
qa/1069 | 1 +
qa/1069.out | 62 +++++++++++++++
src/bashrc/pcp_completion.sh | 2 +-
src/pmrep/TODO | 3 +-
src/pmrep/pmrep.1 | 16 +++-
src/pmrep/pmrep.conf | 1 +
src/pmrep/pmrep.conf.5 | 6 ++
src/pmrep/pmrep.py | 186 ++++++++++++++++++++++++++++++++++++++++++-
src/zshrc/_pcp | 1 +
9 files changed, 269 insertions(+), 9 deletions(-)
diff --git a/qa/1069 b/qa/1069
index 7d216a9..f73daac 100755
--- a/qa/1069
+++ b/qa/1069
@@ -86,6 +86,7 @@ pmrep -T 7s -t 3 -p $log -c $tmp.config -x sample | _path_filter \
| _archive_filter
echo "---"
pmrep -s 5 -t 2 $log -c $tmp.config -G -H -U sample
+pmrep -s 5 -t 2 $log2 -c $tmp.config -p -l '|' -X disk.dev.read disk.dev.write disk.dev.total network.interface.mtu mem.util.free kernel.all.load mem.util.used
echo "== exercise non-integer options"
pmrep -T 2.5s -t 0.5 -p $log -c $tmp.config -x sample | _path_filter \
diff --git a/qa/1069.out b/qa/1069.out
index 6c3723b..22b97dc 100644
--- a/qa/1069.out
+++ b/qa/1069.out
@@ -78,6 +78,68 @@ HH:MM:SS 1
1.000 1000.033
1.000 1000.001
1.000 1000.016
+ | |d.d.read|d.d.write|d.d.total|n.i.mtu|m.u.free|k.a.load|m.u.used
+ | | count/s| count/s| count/s| byte| Kbyte| | Kbyte
+00:47:01|sda | N/A| N/A| N/A| | | |
+00:47:01|sdb | N/A| N/A| N/A| | | |
+00:47:01|sdc | N/A| N/A| N/A| | | |
+00:47:01|hdc | N/A| N/A| N/A| | | |
+00:47:01|sdd | N/A| N/A| N/A| | | |
+00:47:01|sde | N/A| N/A| N/A| | | |
+00:47:01|lo | | | | 16436| | |
+00:47:01|eth0 | | | | 1500| | |
+00:47:01|None | | | | | 38220| | 6068648
+00:47:01|1 minute| | | | | | 0.820|
+00:47:01|5 minute| | | | | | 0.700|
+00:47:01|15 minut| | | | | | 0.590|
+00:47:03|sda | 0.000| 0.500| 0.500| | | |
+00:47:03|sdb | 30.000| 296.500| 326.500| | | |
+00:47:03|sdc | 0.500| 0.500| 1.000| | | |
+00:47:03|hdc | 0.000| 0.000| 0.000| | | |
+00:47:03|sdd | 0.000| 0.000| 0.000| | | |
+00:47:03|sde | 1.000| 12.000| 12.000| | | |
+00:47:03|lo | | | | 16436| | |
+00:47:03|eth0 | | | | 1500| | |
+00:47:03|None | | | | | 38220| | 6068648
+00:47:03|1 minute| | | | | | 0.820|
+00:47:03|5 minute| | | | | | 0.700|
+00:47:03|15 minut| | | | | | 0.590|
+00:47:05|sda | 0.000| 0.500| 0.500| | | |
+00:47:05|sdb | 30.000| 296.000| 326.000| | | |
+00:47:05|sdc | 0.000| 0.500| 0.500| | | |
+00:47:05|hdc | 0.000| 0.000| 0.000| | | |
+00:47:05|sdd | 0.000| 0.000| 0.000| | | |
+00:47:05|sde | 1.000| 11.500| 12.500| | | |
+00:47:05|lo | | | | 16436| | |
+00:47:05|eth0 | | | | 1500| | |
+00:47:05|None | | | | | 38220| | 6068648
+00:47:05|1 minute| | | | | | 0.820|
+00:47:05|5 minute| | | | | | 0.700|
+00:47:05|15 minut| | | | | | 0.590|
+00:47:07|sda | 0.000| 0.500| 0.500| | | |
+00:47:07|sdb | 30.000| 296.500| 326.500| | | |
+00:47:07|sdc | 0.500| 0.500| 1.000| | | |
+00:47:07|hdc | 0.000| 0.000| 0.000| | | |
+00:47:07|sdd | 0.000| 0.000| 0.000| | | |
+00:47:07|sde | 1.000| 11.500| 12.500| | | |
+00:47:07|lo | | | | 16436| | |
+00:47:07|eth0 | | | | 1500| | |
+00:47:07|None | | | | | 38220| | 6068648
+00:47:07|1 minute| | | | | | 0.820|
+00:47:07|5 minute| | | | | | 0.700|
+00:47:07|15 minut| | | | | | 0.590|
+00:47:09|sda | 0.000| 0.500| 0.500| | | |
+00:47:09|sdb | 30.000| 296.000| 326.000| | | |
+00:47:09|sdc | 0.500| 0.500| 0.500| | | |
+00:47:09|hdc | 0.000| 0.000| 0.000| | | |
+00:47:09|sdd | 0.000| 0.000| 0.000| | | |
+00:47:09|sde | 0.500| 11.500| 12.500| | | |
+00:47:09|lo | | | | 16436| | |
+00:47:09|eth0 | | | | 1500| | |
+00:47:09|None | | | | | 38220| | 6068648
+00:47:09|1 minute| | | | | | 0.820|
+00:47:09|5 minute| | | | | | 0.700|
+00:47:09|15 minut| | | | | | 0.590|
== exercise non-integer options
archive: QAPATH/archives/sample-secs
diff --git a/src/bashrc/pcp_completion.sh b/src/bashrc/pcp_completion.sh
index 9bffae0..ff11b9d 100644
--- a/src/bashrc/pcp_completion.sh
+++ b/src/bashrc/pcp_completion.sh
@@ -42,7 +42,7 @@ _pcp_complete()
;;
pmrep)
- opt_regex="-[AaBbcCdDeEfFGhHiKLloOpPqQrsStTuUVwxyYzZ]"
+ opt_regex="-[AaBbCcDdEeFfGHhiKLlOoPpQqrSsTtUuVwXxYyZz]"
curpos_expand=1
;;
diff --git a/src/pmrep/TODO b/src/pmrep/TODO
index 317b4a7..ecb4c0b 100644
--- a/src/pmrep/TODO
+++ b/src/pmrep/TODO
@@ -1,5 +1,4 @@
- opt to write cols per row with stdout output [-k]
-- opt to swap cols and rows (e.g., compare ps vs sar)
- -Q/-B/-Y to override per-metric settings unit/scale
- shorten unit display count/s -> c/s if needed
- opt to output in short format (see pmdumptext -F / pmNumberStr) [-m/-M]
@@ -27,7 +26,7 @@
-----
In use / reserved cmd line options:
-AaBbcCdDeEfFGhHiKLloOpPqQrsStTuUVwxyYzZ
+AaBbCcDdEeFfGHhiKLlOoPpQqrSsTtUuVwXxYyZz
Tentatively planned:
IkMmR
diff --git a/src/pmrep/pmrep.1 b/src/pmrep/pmrep.1
index 7ccfb08..4755583 100644
--- a/src/pmrep/pmrep.1
+++ b/src/pmrep/pmrep.1
@@ -20,7 +20,7 @@
\f3pmrep\f1 \- performance metrics reporter
.SH SYNOPSIS
\f3pmrep\f1
-[\f3\-CdGHLpruUVxz?\f1]
+[\f3\-CdGHLpruUVxXz?\f1]
[\f3\-a\f1 \f2archive\f1]
[\f3\--archive-folio\f1 \f2folio\f1]
[\f3\-A\f1 \f2align\f1]
@@ -340,9 +340,9 @@ method which is mostly the same as that described in
An empty
.I format
string (i.e., '') will remove the timestamps from the output.
-The default with stdout is
+The default with \fIstdout\fR is
.BR %H:%M:%S .
-The default with csv is
+The default with \fIcsv\fR is
.BR "%Y-%m-%d %H:%M:%S" .
.TP
.B \-F
@@ -632,7 +632,7 @@ Omit unit information from of headers.
Display version number and exit.
.TP
.B \-w
-Set the stdout output column
+Set the \fIstdout\fR output column
.IR width .
Strings will be truncated to this width.
The default
@@ -645,6 +645,14 @@ override possible per-metric specifications.
.B \-x
Print extended header.
.TP
+.B \-X
+Swap columns and rows in \fIstdout\fR output, reporting one instance per
+line. This is convenient to allow easily using
+.BR grep (1)
+to filter results or to more closely emulate other utilities.
+See also
+.BR -i .
+.TP
.B \-y
.I Unit/scale
for time metrics, possible values include
diff --git a/src/pmrep/pmrep.conf b/src/pmrep/pmrep.conf
index fcdd8e5..05142ff 100644
--- a/src/pmrep/pmrep.conf
+++ b/src/pmrep/pmrep.conf
@@ -18,6 +18,7 @@
#delay = no
#type = default
#instances =
+#colxrow = no
#width =
#precision = 3
#delimiter =
diff --git a/src/pmrep/pmrep.conf.5 b/src/pmrep/pmrep.conf.5
index 48550bc..8beb1d3 100644
--- a/src/pmrep/pmrep.conf.5
+++ b/src/pmrep/pmrep.conf.5
@@ -202,6 +202,12 @@ option is \fB-i\fR. Undefined (all current instances are reported)
by default.
.RE
.P
+colxrow (bool)
+.RS 4
+Swaps columns and rows in \fIstdout\fR output. Corresponding command line
+option is \fB-X\fR. Defaults to \fBno\fR.
+.RE
+.P
width (integer)
.RS 4
Indicates the width of stdout output columns. Corresponding command line
diff --git a/src/pmrep/pmrep.py b/src/pmrep/pmrep.py
index 05110c0..0abef4d 100755
--- a/src/pmrep/pmrep.py
+++ b/src/pmrep/pmrep.py
@@ -174,7 +174,7 @@ class PMReporter(object):
'extheader', 'repeat_header', 'timefmt', 'interpol',
'count_scale', 'space_scale', 'time_scale', 'version',
'zabbix_server', 'zabbix_port', 'zabbix_host', 'zabbix_interval',
- 'speclocal', 'instances')
+ 'speclocal', 'instances', 'colxrow')
# Special command line switches
self.arghelp = ('-?', '--help', '-V', '--version')
@@ -206,6 +206,7 @@ class PMReporter(object):
self.delay = 0
self.type = 0
self.instances = []
+ self.colxrow = 0
self.width = 0
self.precision = 3 # .3f
self.delimiter = None
@@ -246,7 +247,7 @@ class PMReporter(object):
opts = pmapi.pmOptions()
opts.pmSetOptionCallback(self.option)
opts.pmSetOverrideCallback(self.option_override)
- opts.pmSetShortOptions("a:h:LK:c:Co:F:e:D:V?HUGpA:S:T:O:s:t:Z:zdri:w:P:l:xE:f:uq:b:y:")
+ opts.pmSetShortOptions("a:h:LK:c:Co:F:e:D:V?HUGpA:S:T:O:s:t:Z:zdri:Xw:P:l:xE:f:uq:b:y:")
opts.pmSetShortUsage("[option...] metricspec [...]")
opts.pmSetLongOptionHeader("General options")
@@ -281,6 +282,7 @@ class PMReporter(object):
opts.pmSetLongOption("delay", 0, "d", "", "delay, pause between updates for archive replay")
opts.pmSetLongOption("raw", 0, "r", "", "output raw counter values (no rate conversion)")
opts.pmSetLongOption("instances", 1, "i", "STR", "instances to report (default: all current)")
+ opts.pmSetLongOption("colxrow", 0, "X", "", "swap columns and rows in stdout output")
opts.pmSetLongOption("width", 1, "w", "N", "default column width")
opts.pmSetLongOption("precision", 1, "P", "N", "N digits after the decimal separator (if width enough)")
opts.pmSetLongOption("delimiter", 1, "l", "STR", "delimiter to separate csv/stdout columns")
@@ -347,6 +349,8 @@ class PMReporter(object):
self.type = 1
elif opt == 'i':
self.instances = self.instances + self.parse_instances(optarg)
+ elif opt == 'X':
+ self.colxrow = 1
elif opt == 'w':
self.width = int(optarg)
elif opt == 'P':
@@ -1034,6 +1038,13 @@ class PMReporter(object):
def prepare_stdout(self):
""" Prepare stdout output """
+ if not self.colxrow:
+ self.prepare_stdout_std()
+ else:
+ self.prepare_stdout_colxrow()
+
+ def prepare_stdout_std(self):
+ """ Prepare standard formatted stdout output """
index = 0
if self.timestamp == 0:
#self.format = "{:}{}"
@@ -1058,6 +1069,41 @@ class PMReporter(object):
l = len(str(index-1)) + 2
self.format = self.format[:-l]
+ def prepare_stdout_colxrow(self):
+ """ Prepare columns and rows swapped stdout output """
+ index = 0
+
+ # Timestamp
+ if self.timestamp == 0:
+ self.format = "{0:}{1}"
+ index += 2
+ else:
+ tstamp = datetime.fromtimestamp(time.time()).strftime(self.timefmt)
+ self.format = "{0:<" + str(len(tstamp)) + "." + str(len(tstamp)) + "}{1}"
+ index += 2
+
+ # Instance name
+ if self.width:
+ self.format += "{2:<" + str(self.width) + "." + str(self.width) + "}{3}"
+ else:
+ ts = datetime.fromtimestamp(time.time()).strftime(self.timefmt)
+ self.format += "{2:<" + str(len(ts)) + "." + str(len(ts)) + "}{3}"
+ index += 2
+
+ # Metrics
+ for i, metric in enumerate(self.metrics):
+ l = str(self.metrics[metric][4])
+ # Value truncated and aligned
+ self.format += "{" + str(index) + ":>" + l + "." + l + "}"
+ index += 1
+ # Dummy
+ self.format += "{" + str(index) + "}"
+ index += 1
+
+ # Drop the last dummy
+ l = len(str(index-1)) + 2
+ self.format = self.format[:-l]
+
def write_ext_header(self):
""" Write extended header """
comm = "#" if self.output == OUTPUT_CSV else ""
@@ -1143,8 +1189,17 @@ class PMReporter(object):
names = ["", self.delimiter] # no timestamp on header line
insts = ["", self.delimiter] # no timestamp on instances line
units = ["", self.delimiter] # no timestamp on units line
+ if self.colxrow:
+ names += ["", self.delimiter] # no instance on header line
+ units += ["", self.delimiter] # no instance on units line
prnti = 0
for i, metric in enumerate(self.metrics):
+ if self.colxrow:
+ names.append(self.metrics[metric][0])
+ names.append(self.delimiter)
+ units.append(self.metrics[metric][2][0])
+ units.append(self.delimiter)
+ continue
prnti = 1 if self.insts[i][0][0] != PM_IN_NULL else prnti
for j in range(len(self.insts[i][0])):
names.append(self.metrics[metric][0])
@@ -1299,6 +1354,13 @@ class PMReporter(object):
def write_stdout(self, timestamp):
""" Write a line to stdout """
+ if not self.colxrow:
+ self.write_stdout_std(timestamp)
+ else:
+ self.write_stdout_colxrow(timestamp)
+
+ def write_stdout_std(self, timestamp):
+ """ Write a line to standard formatted stdout """
if timestamp == None:
# Silent goodbye
return
@@ -1373,6 +1435,126 @@ class PMReporter(object):
nfmt = nfmt[:-l]
self.writer.write(nfmt.format(*tuple(line)) + "\n")
+ def write_stdout_colxrow(self, timestamp):
+ """ Write a line to columns and rows swapped stdout """
+ if timestamp == None:
+ # Silent goodbye
+ return
+
+ # Collect the instances in play
+ insts = []
+ for i in range(len(self.metrics)):
+ for instance in self.insts[i][1]:
+ if instance not in insts:
+ insts.append(instance)
+
+ # Avoid crossing the C/Python boundary more than once per metric
+ res = OrderedDict()
+ for i, metric in enumerate(self.metrics):
+ res[metric] = []
+ try:
+ for inst, name, val in self.metrics[metric][5]():
+ try:
+ res[metric].append([inst, name, val()])
+ except:
+ res[metric].append([inst, name, NO_VAL])
+ if not res[metric]:
+ res[metric].append(['', '', NO_VAL])
+ except:
+ res[metric].append(['', '', NO_VAL])
+
+ # Avoid per-line I/O
+ output = ""
+
+ # Painfully iterate over what we have, the logic below
+ # being that we need to construct each line independently
+ for instance in insts:
+ # Split on dummies
+ fmt = re.split("{\\d+}", self.format)
+
+ # Start a new line
+ k = 0
+ line = []
+
+ # Add timestamp as wanted
+ if self.timestamp == 0:
+ line.append("")
+ else:
+ line.append(timestamp)
+ line.append(self.delimiter)
+ k += 1
+
+ # Add instance
+ line.append(instance)
+ line.append(self.delimiter)
+ k += 1
+
+ # Look for this instance from each metric
+ for metric in self.metrics:
+ l = self.metrics[metric][4]
+
+ found = 0
+ value = NO_VAL
+ for inst in res[metric]:
+ if inst[1] == instance:
+ # This metric has the instance we're
+ # processing, grab it and format below
+ value = inst[2]
+ found = 1
+ break
+
+ if not found:
+ # Not an instance this metric has,
+ # add a placeholder and move on
+ line.append("")
+ line.append(self.delimiter)
+ k += 1
+ continue
+
+ # Make sure the value fits
+ if type(value) is int or type(value) is long:
+ if len(str(value)) > l:
+ value = TRUNC
+ else:
+ fmt[k] = "{X:" + str(l) + "d}"
+
+ if type(value) is float and not math.isinf(value):
+ c = self.precision
+ s = len(str(int(value)))
+ if s > l:
+ c = -1
+ value = TRUNC
+ for f in reversed(range(c+1)):
+ r = "{X:" + str(l) + "." + str(c) + "f}"
+ t = "{0:" + str(l) + "." + str(c) + "f}"
+ if len(t.format(value)) > l:
+ c -= 1
+ else:
+ fmt[k] = r
+ break
+
+ # Finally add the value
+ line.append(value)
+ line.append(self.delimiter)
+ k += 1
+
+ # Print the line in a Python 2.6 compatible manner
+ del line[-1]
+ index = 0
+ nfmt = ""
+ for f in fmt:
+ if type(line[index]) is float and math.isinf(line[index]):
+ line[index] = "inf"
+ nfmt += f.replace("{X:", "{" + str(index) + ":")
+ index += 1
+ nfmt += "{" + str(index) + "}"
+ index += 1
+ l = len(str(index-1)) + 2
+ nfmt = nfmt[:-l]
+ output += nfmt.format(*tuple(line)) + "\n"
+
+ self.writer.write(output)
+
def write_zabbix(self, timestamp):
""" Write (send) metrics to a Zabbix server """
if timestamp == None:
diff --git a/src/zshrc/_pcp b/src/zshrc/_pcp
index b625164..6f81073 100644
--- a/src/zshrc/_pcp
+++ b/src/zshrc/_pcp
@@ -440,6 +440,7 @@ _pcp () {
"(-d --delay --container -h --host -L --local-PMDA -K --spec-local -u --no-interpol $exargs)"{-d,--delay}'[delay between updates in archive mode]' \
"(-r --raw $exargs)"{-r,--raw}'[report raw values]' \
"($exargs)"\*{-i+,--instances}'[specify instances to report]:instances:->instances' \
+ "(-X --colxrow $exargs)"{-X,--colxrow}'[swap columns and rows in stdout output]' \
"(-w --width $exargs)"{-w+,--width}'[set default column width]:width:' \
"(-P --precision $exargs)"{-P+,--precision}'[set floating point precision]:precision:' \
"(-l --delimiter $exargs)"{-l+,--delimiter}'[set column delimiter]:delimiter:' \
Thanks,
--
Marko Myllynen
-=-=-=-=-=-=-=-=-=-=-=-
pcp mailing list
[email protected]
https://groups.io/g/pcp/messages
-=-=-
Groups.io Links:
You receive all messages sent to this group.
View/Reply Online (#14638): https://groups.io/g/pcp/message/14638
View All Messages In Topic (1): https://groups.io/g/pcp/topic/3037310
Mute This Topic: https://groups.io/mt/3037310?uid=174580
New Topic: https://groups.io/g/pcp/post
Change Your Subscription: https://groups.io/g/pcp/editsub?uid=174580
Group Home: https://groups.io/g/pcp
Contact Group Owner: [email protected]
Terms of Service: https://groups.io/static/tos
Unsubscribe: https://groups.io/g/pcp/leave/354243/563757577/xyzzy
-=-=-=-=-=-=-=-=-=-=-=-