pcp updates: merge, merge, merge
"Nathan Scott" <[email protected]> Mon, 13 Feb 2017 00:25:07 -0500 (EST)
| Newsgroups | gmane.comp.sysutils.pcp |
|---|---|
| Message-ID | <[email protected]> |
Changes committed to git://git.pcp.io/pcp.git master
Paul Cuzner (15):
1st code commit
pmdalio.python: Updated domain ID to avoid upstream conflicts
pcp-lio.te: Add example SELINUX TE file
README: clarified use of the TE example
Tests: 1st commit of test/validation code
Sample configfs directory added for testing
tests: json conf file added
tests: 1st commit for automated tests of the pmda agent
pmdalio: syntax fix
pmdalio: Catch RTSLIBNotInCFS error on storage object
tests: enable the drop configuraton method
README: document current issues with repeated test runs
pmdalio: ensure pmda sees config changes, and correct LUN size calculation
tests: add unittest based automated testing
spec: spec file added
Ken McDonell (12):
qa/964: make sure proper PMDA cleanup happens at the end of the test
qa/1059: work around possible lack of lsof(1)
qa/918 & qa/1083: (new) exercise pmcd rc-init script with systemd
src/pcp/pmcd/rc-proc.sh: add mkdir_and_chown()
qa/new: tweak script so _cleanup() provided and used in trap
qa/934: (new) exercise mkdir_and_chown() from rc-proc.sh
src/pmcd/rc-proc.sh & qa/934: add mode parameter to mkdir_and_chown
src/pmlogger/pmlogger_check.sh & qa/920(new): use mkdir_and_chown()
qa/920: report mode for created directories
src/pmlogger/pmlogger_daily.sh & qa/925(new): created output directory hierarchy
src/pmie/pmie_check.sh & qa/1060(new): make mkdir_and_chown() changes
qa: assorted minor fallout from mkdir_and_chown() changes
Lukas Berk (9):
pmdalio: rework file layout slightly and add build bits
pmdalio: Move test to qa directory
qa: add wrapper test for the pmdalio tests
pmdalio: further qa work
pmdalio: add postun call to pmda removal scriptlet
Update .gitignore
pmdalio: QA Install and Remove pmdalio in qa/1060
Add pmdalio to pmns
pmdalio: add metric help text and minor corrections
pcuzner (4):
Initial commit
added README documentation
README: include rtslib and SELINUX info
Updated to reflect python3 rtslib tests
Nathan Scott (3):
docs: add indom help text for recent pmdalinux indom addition
pmdalio: drop redundant/generated files, install the README
qa: add top level "tape" PMNS entry for filtering
Mark Goodwin (2):
Fix containers/pcp-base build error
pmdalinux: add support for scsi tape statistics
.gitignore | 95 ++++++++
LICENSE | 339 +++++++++++++++++++++++++++++
README.md | 73 ++++++
build/containers/pcp-base/GNUmakefile | 12 -
build/rpm/pcp.spec.in | 36 ++-
dev/null |binary
lio/Install | 28 ++
lio/Remove | 25 ++
lio/pmdalio.python | 356 +++++++++++++++++++++++++++++--
pcp-lio.te | 29 ++
pcp-pmda-lio.spec | 65 +++++
qa/1059 | 37 +--
qa/1060 | 146 ++++++++++++
qa/1060.out | 38 ++-
qa/1083 | 61 +++++
qa/1083.out | 7
qa/115.out | 1
qa/781 | 2
qa/918 | 76 ++++++
qa/918.out | 8
qa/920 | 76 ++++++
qa/920.out | 41 ++-
qa/925 | 90 +++++++
qa/925.out | 24 ++
qa/934 | 145 +++++++++++-
qa/934.out | 196 ++++++++++++++---
qa/964 | 6
qa/964.out | 14 +
qa/GNUmakefile | 2
qa/GNUmakefile.install | 2
qa/common.filter | 1
qa/group | 10
qa/lio/.gitignore | 5
qa/lio/GNUmakefile | 22 +
qa/lio/GNUmakefile.install | 23 ++
qa/lio/README | 39 +++
qa/lio/iscsi_conf.json | 22 +
qa/lio/iscsi_target.py | 202 ++++++++---------
qa/lio/iscsi_target.python | 101 ++++++++
qa/lio/iscsi_target_configfs_dump.tar.gz |binary
qa/lio/lio_test.py | 212 +++++++++---------
qa/lio/lio_test.python | 106 +++++++++
qa/new | 8
src/pmcd/rc-proc.sh | 69 +++++-
src/pmdas/GNUmakefile | 2
src/pmdas/linux/GNUmakefile | 4
src/pmdas/linux/help | 23 ++
src/pmdas/linux/linux.h | 4
src/pmdas/linux/pmda.c | 77 ++++++
src/pmdas/linux/root_linux | 19 +
src/pmdas/linux/sysfs_tapestats.c | 122 ++++++++++
src/pmdas/linux/sysfs_tapestats.h | 52 ++++
src/pmdas/lio/GNUmakefile | 47 ++++
src/pmdas/lio/Install | 32 ++
src/pmdas/lio/LICENSE | 339 -----------------------------
src/pmdas/lio/Remove | 29 ++
src/pmdas/lio/help | 84 +++----
src/pmdas/lio/lio/Install | 28 --
src/pmdas/lio/lio/Remove | 25 --
src/pmdas/lio/lio/pmdalio.python | 320 ---------------------------
src/pmdas/lio/pcp-lio.te | 29 --
src/pmdas/lio/pcp-pmda-lio.spec | 65 -----
src/pmdas/lio/pmdalio.python | 328 ++++++++++++++++++++++++++++
src/pmdas/lio/tests/README | 39 ---
src/pmdas/lio/tests/iscsi_conf.json | 22 -
src/pmdas/lio/tests/iscsi_target.py | 101 --------
src/pmdas/lio/tests/lio_test.py | 106 ---------
src/pmie/pmie_check.sh | 4
src/pmlogger/pmlogger_check.sh | 11
src/pmlogger/pmlogger_daily.sh | 16 +
src/pmns/stdpmid.pcp | 1
tests/README | 91 +++++--
tests/iscsi_conf.json | 22 +
tests/iscsi_target.py | 119 +++++++++-
tests/iscsi_target_configfs_dump.tar.gz |binary
tests/lio_test.py | 136 ++++++++++-
76 files changed, 3646 insertions(+), 1501 deletions(-)
commit 99ee5d0deb7a0aa4a7fb32bcfdd8e16ca97387bd
Author: Nathan Scott <[email protected]>
Date: Mon Feb 13 16:18:08 2017 +1100
qa: add top level "tape" PMNS entry for filtering
commit 8b72912c170cb5b293ded8ff04a051067f4633a9
Author: Nathan Scott <[email protected]>
Date: Mon Feb 13 16:01:17 2017 +1100
pmdalio: drop redundant/generated files, install the README
commit 649de91b60bbc0af2c49bfce3206f752b40bd17a
Author: Nathan Scott <[email protected]>
Date: Mon Feb 13 15:25:45 2017 +1100
docs: add indom help text for recent pmdalinux indom addition
commit b3660e3be239cc77516f8bbf7cc31242b406adff
Author: Mark Goodwin <[email protected]>
Date: Mon Feb 13 14:22:35 2017 +1100
pmdalinux: add support for scsi tape statistics
Linux tape stats are now fairly widely available. The new Liux PMDA metrics
are named after the statistics exported below /sys/class/scsi_tape/*/stats/*
as follows :
tape.dev.in_flight [number of I/Os currently outstanding to this tape device]
tape.dev.io_ns [cummulative amount of time spent waiting for all I/O to complete to tape device]
tape.dev.other_cnt [number of I/Os issued to the tape drive other than read or write commands]
tape.dev.read_byte_cnt [number of bytes read from the tape drive]
tape.dev.read_cnt [number of read requests issued to the tape drive]
tape.dev.read_ns [cummulative amount of time spent waiting for read requests to complete]
tape.dev.resid_cnt [count of read or write residual data, per tape device]
tape.dev.write_byte_cnt [number of bytes written to the tape drive]
tape.dev.write_cnt [number of write requests issued to the tape drive]
tape.dev.write_ns [cummulative amount of time spent waiting for write requests to complete]
All of the above span the new TAPEDEV instance domain. We support tape devices being
dynamically added or removed. This can be tested with scsi_debug on supported kernels,
e.g. to create 4 virtual scsi tape devices :
# modprobe scsi_debug ptype=1 add_host=4 dev_size_mb=100
To remove one of them (e.g. st2) :
# echo 1 > /sys/class/scsi_tape/st2/device/delete
To remove all of them :
# rmmod scsi_debug
The above scsi_debug interface will be used to create fake sysfs files for QA purposes,
which will be in a followup commit. There are also a couple of QA updates for existing
tests.
We also support the singular metric 'hinv.ntape' as the current number of scsi tape devices.
Note also that the various derivative devices, such as /dev/nst1 (and so forth) actually
share the same statistics exports as the regular devices and so the tape device indom is
not populated with these devices.
modified: src/pmdas/linux/GNUmakefile
modified: src/pmdas/linux/help
modified: src/pmdas/linux/linux.h
modified: src/pmdas/linux/pmda.c
modified: src/pmdas/linux/root_linux
new file: src/pmdas/linux/sysfs_tapestats.c
new file: src/pmdas/linux/sysfs_tapestats.h
commit 63fa293e82578bdb435bf633aa2123f6825e706a
Author: Ken McDonell <[email protected]>
Date: Mon Feb 13 13:32:28 2017 +1100
qa: assorted minor fallout from mkdir_and_chown() changes
commit 7d5d7d0f27642a7546fdf14c631957d3d87b362d
Author: Ken McDonell <[email protected]>
Date: Mon Feb 13 13:31:10 2017 +1100
src/pmie/pmie_check.sh & qa/1060(new): make mkdir_and_chown() changes
Same as for pmlogger_check ... same issue existed with a directory
hierarchy for the output directory.
commit eaeeffd045546a3b7e4554e1f2ac7120f87dc715
Author: Ken McDonell <[email protected]>
Date: Mon Feb 13 07:16:29 2017 +1100
src/pmlogger/pmlogger_daily.sh & qa/925(new): created output directory hierarchy
Replicate logic from pmlogger_check to create the full directory
hierarchy for PCP archives (if it does not already exist) with the
correct modes and ownership.
This addresses the issue of pmlogger_daily possibly running before
pmlogger_check as reported by Martins Innus <[email protected]>.
commit 2bf9fb89bb447e9e60ad1eec1da7f22f6fdb4f5f
Author: Ken McDonell <[email protected]>
Date: Mon Feb 13 07:12:18 2017 +1100
qa/920: report mode for created directories
commit 4cffba28eba781148fde2b703fb789f1165f0afa
Author: Ken McDonell <[email protected]>
Date: Mon Feb 13 06:46:14 2017 +1100
src/pmlogger/pmlogger_check.sh & qa/920(new): use mkdir_and_chown()
Ensure all directories created by pmlogger_check for the PCP archives
are writeable by $PCP_USER:$PCP_GROUP.
This addresses the issue with multi-level directory structures from
the control file, as reported by Martins Innus <[email protected]>.
commit 9ba8435dc7d748f8bcfe75e3b8d7990ed26a9f94
Author: Ken McDonell <[email protected]>
Date: Mon Feb 13 06:32:38 2017 +1100
src/pmcd/rc-proc.sh & qa/934: add mode parameter to mkdir_and_chown
For our scripts using mkdir_and_chown() we need to control the mode
of any created directories.
commit 9d08c13468262f7f1ed925c37cce8cef4c855d85
Author: Ken McDonell <[email protected]>
Date: Sun Feb 12 17:05:58 2017 +1100
qa/934: (new) exercise mkdir_and_chown() from rc-proc.sh
commit c7512ca3ac1cd4cac228274ca55e14f2fd208b19
Author: Ken McDonell <[email protected]>
Date: Sun Feb 12 17:05:21 2017 +1100
qa/new: tweak script so _cleanup() provided and used in trap
commit 0960182b7321eab37684676ed4290d34587469b3
Author: Ken McDonell <[email protected]>
Date: Sun Feb 12 17:00:53 2017 +1100
src/pcp/pmcd/rc-proc.sh: add mkdir_and_chown()
This helper function provides the moral equivalent of mkdir -p,
except that for any directories created there is an option to also
chown each of these.
Intended for the {pmlogger,pmie}_{check,daily} scripts where we
are doing
mkdir -p $path
chown $PCP_USER:$PCP_GROUP $path
which is not quite right of $path contains multiple directory
levels that do not currently exist.
This starts to address one of the problems reported by Martins Innus
<[email protected]> on 31 Jan 2017.
commit bf9a5e4e1dcb0e4ec54257c3300026ca1e724fc1
Author: Lukas Berk <[email protected]>
Date: Sat Feb 11 15:14:43 2017 -0500
pmdalio: add metric help text and minor corrections
Install/Remove - correct date / pmda name
help - add help text for metrics
pmdalio.python - small typo correction
pcp-pmda-lio.spec - remove
pcp-lio.te - remove, now within pcp's upstream policy package
commit 678dd3e9ce8c74163e2028bf12c6bbbedbdfb106
Author: Ken McDonell <[email protected]>
Date: Sun Feb 12 11:49:19 2017 +1100
qa/918 & qa/1083: (new) exercise pmcd rc-init script with systemd
Explicitly check restart, stop, start and autorestart using systemd
when it is available.
commit 2aff9a22c9c334857eb95f5480d768d0a10520d2
Author: Ken McDonell <[email protected]>
Date: Sun Feb 12 07:31:45 2017 +1100
qa/1059: work around possible lack of lsof(1)
Workaround is a bit generous, just fake one part of the test if there
is no lsof.
commit 125d18c626aee915bd2ef7d2b34f1ac71ec0e251
Author: Ken McDonell <[email protected]>
Date: Sun Feb 12 07:30:35 2017 +1100
qa/964: make sure proper PMDA cleanup happens at the end of the test
Was leaving the PMNS trashed with names from the dm PMDA, but the
dm PMDA was not necessarily installed.
commit 7204e68b0de8f0bb8be8c4a8ad93acf4e11c2856
Author: Mark Goodwin <[email protected]>
Date: Sat Feb 11 16:13:33 2017 +1100
Fix containers/pcp-base build error
The RPMS directory needs to be created before we can populate it
commit 7e8fb15cf944a02769470bd769bd1d3c583c2780
Author: Lukas Berk <[email protected]>
Date: Fri Feb 10 16:17:15 2017 -0500
Add pmdalio to pmns
Added as 142
commit c5734bbdb2bf48322d9d2c6e53b29cc829b1f60e
Author: Lukas Berk <[email protected]>
Date: Tue Feb 7 10:42:14 2017 -0500
pmdalio: QA Install and Remove pmdalio in qa/1060
commit e73b0e2e5af39f35147a1d8db2f0587c784d5dd0
Author: Lukas Berk <[email protected]>
Date: Fri Feb 3 22:13:24 2017 -0500
Update .gitignore
commit de6a283e51b513646a807ec8b3048846237fd9f1
Author: Lukas Berk <[email protected]>
Date: Fri Feb 3 21:32:41 2017 -0500
pmdalio: add postun call to pmda removal scriptlet
commit 358463d045df8e21df35db7940491e03f78a09c2
Author: Lukas Berk <[email protected]>
Date: Fri Feb 3 21:32:23 2017 -0500
pmdalio: further qa work
commit 0370eac68461d43597f73f35c53ddc94c8cd975c
Author: Lukas Berk <[email protected]>
Date: Fri Feb 3 12:04:13 2017 -0500
qa: add wrapper test for the pmdalio tests
commit 0a5f45012e6c0cdee001982acaf55f78d3ad46e1
Author: Lukas Berk <[email protected]>
Date: Thu Feb 2 21:29:59 2017 -0500
pmdalio: Move test to qa directory
Tests are typically kept within the pcp/qa directory, just moving things across for
a typical qa test to be written to include the pre-written test
commit c0853f13e617d5e3ae553b499fa8f36a93115c8d
Author: Lukas Berk <[email protected]>
Date: Thu Feb 2 17:48:49 2017 -0500
pmdalio: rework file layout slightly and add build bits
Drop the sub lio directory and move Install/Remove/pmdalio.python
files to the main pmdas/lio dir
Add GNUmakefile, include pmda in pcp.spec.in
commit e2e2df067ac41cd652b943494375f3f707812663
Author: Paul Cuzner <[email protected]>
Date: Thu Jan 5 17:28:21 2017 +1300
spec: spec file added
Allow rpm to be built from the repo
commit ccf7f3032b877cd0b4893fc8c0b71cb88b9abf02
Author: Paul Cuzner <[email protected]>
Date: Wed Jan 4 11:20:19 2017 +1300
tests: add unittest based automated testing
the lio_test script has been added to build a test environment and
validate the PMDA values dynamically. The test iscsi environment is
determined by iscsi_conf.json and so can be changed to meet
different levels of complexity.
See README for further details
commit 4a6f5930e0d1c3fa25836314d2d762e76cbbf5d3
Author: Paul Cuzner <[email protected]>
Date: Wed Jan 4 11:12:58 2017 +1300
pmdalio: ensure pmda sees config changes, and correct LUN size calculation
By default the tcm module holds a cache dict of storage object so it only
walks configfs once for device discovery. This is fine, if adds/removes are
done through the same system - but in the case of the pmda the cache is
a problem since it won't reflect changes to configfs correctly. This fix drops
the cache (bs_cache) at each refresh interval.
In addition a minor change was made to the calculation of LUN size to round
up (4.8GB = 5GB)
commit 0368886f8b0c073b75fae1a611882aee31d21546
Author: Paul Cuzner <[email protected]>
Date: Fri Dec 23 18:10:01 2016 +1300
README: document current issues with repeated test runs
Issues section added to the README
commit e8f6364a74ea48425399bd0e064db7780ee99b0d
Author: Paul Cuzner <[email protected]>
Date: Fri Dec 23 18:00:49 2016 +1300
tests: enable the drop configuraton method
In the teardown class method, the call to drop the iscsi
definition needed to be enabled (was commented out!)
commit 360d9f328af5686c4d6f4232ca3f333f359fa5ef
Author: Paul Cuzner <[email protected]>
Date: Fri Dec 23 17:44:52 2016 +1300
pmdalio: Catch RTSLIBNotInCFS error on storage object
When rebuilding the lun dict (refresh call), new luns added to the environment
can trigger an RTSLIBNotInCFS exception, likely due to timing. By catching the exception
we allow the lun dict to be updated properly, bypassing this issue.
commit a5069220a3e9739ca898543db45c8d93eb5fe24f
Author: Paul Cuzner <[email protected]>
Date: Fri Dec 23 16:06:53 2016 +1300
pmdalio: syntax fix
call public property not internal method on tpg object
commit e7416ce3872645a78c6ad75b193a4eaa48f0ceee
Author: Paul Cuzner <[email protected]>
Date: Fri Dec 23 15:48:23 2016 +1300
tests: 1st commit for automated tests of the pmda agent
the lio_test script creates a standalone iscsi target instance on the
local machine using the iscsi_target module, which is then compared
to the values in the testcases
commit 61f99b0957148d8ef0e2cb17d0f32ef7e93c73b2
Author: Paul Cuzner <[email protected]>
Date: Fri Dec 23 15:46:29 2016 +1300
tests: json conf file added
the json file is used by the lio_test.py code to dynamically create a
local iscsi configuration on the host.
commit 152b40c9780a54feaa3cb00d0d440289af2bc284
Author: Paul Cuzner <[email protected]>
Date: Fri Dec 23 12:04:43 2016 +1300
Sample configfs directory added for testing
commit b4a57906880a36a6927fde597bf43e2cda560308
Author: Paul Cuzner <[email protected]>
Date: Wed Dec 21 12:49:55 2016 +1300
Tests: 1st commit of test/validation code
commit 9a0f3242ccf36dba61229f997ed504a67e4c1d1f
Author: pcuzner <[email protected]>
Date: Wed Dec 21 09:14:02 2016 +1300
Updated to reflect python3 rtslib tests
commit 67351bfde15fb2695fdca0ca4c101b0f85a60cbf
Author: Paul Cuzner <[email protected]>
Date: Tue Dec 20 10:09:42 2016 +1300
README: clarified use of the TE example
commit 13c8cc36964478371411322ac4280cc036c711ed
Author: Paul Cuzner <[email protected]>
Date: Tue Dec 20 10:01:14 2016 +1300
pcp-lio.te: Add example SELINUX TE file
This is an example TE file used during testing to allow pmcd to start the lio pmda at
boot time.
commit 68188c24a414cdc985b553f51a0f5e2b0537de99
Author: pcuzner <[email protected]>
Date: Tue Dec 20 09:53:57 2016 +1300
README: include rtslib and SELINUX info
Updated to explain the dependency on python-rtslib for interaction with the LIO metadata held in configfs. Also included information relating to running the pmda in SELINUX environments.
commit b3e6ecd0398c63acfc71fe01bc014daff8167815
Author: Paul Cuzner <[email protected]>
Date: Tue Dec 13 17:10:49 2016 +1300
pmdalio.python: Updated domain ID to avoid upstream conflicts
Initial domain id was 140, but informed by Lucas Berk @ Red Hat that this conflicts
with upstream domains.
commit 14d0737fc42b745072e19e25e743aaf2d75e876f
Author: Paul Cuzner <[email protected]>
Date: Tue Dec 13 16:17:53 2016 +1300
1st code commit
commit ae901e270390e19aa01f2a63f9811cab33d93359
Author: pcuzner <[email protected]>
Date: Tue Dec 13 15:54:47 2016 +1300
added README documentation
commit c57c02efe5b2422b8561f25d346b7a67460f90a8
Author: pcuzner <[email protected]>
Date: Tue Dec 13 15:24:06 2016 +1300
Initial commit
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links:
You receive all messages sent to this group.
View/Reply Online (#15126): https://groups.io/g/pcp/message/15126
View All Messages In Topic (1): https://groups.io/g/pcp/topic/4362976
Mute This Topic: https://groups.io/mt/4362976?uid=174580
New Topic: https://groups.io/g/pcp/post
-=-=-
pcp mailing list
[email protected]
https://groups.io/g/pcp/messages
-=-=-
Change Your Subscription: https://groups.io/g/pcp/editsub?uid=174580
Group Home: https://groups.io/g/pcp
Contact Group Owner: [email protected]
Terms of Service: https://groups.io/static/tos
Unsubscribe: https://groups.io/g/pcp/leave/354243/563757577/xyzzy
-=-=-=-=-=-=-=-=-=-=-=-