GSOC 2017 ideas

"Mark Goodwin" <[email protected]> Mon, 30 Jan 2017 11:49:57 +1100
Newsgroups gmane.comp.sysutils.pcp
Message-ID <[email protected]>
Here's some ideas for PCP GSOC'17 students. Some of these are a bit
"out-there",
especially the last one. Thoughts/comments welcome :)

*htop brain transplant*
  - replace htop backend with PCP API calls
  - add remote access, archive replay and key bindings for time controls

*Blocked Process Analysis tool*
  - extend pcp-pidstat or write a new tool to report blocked processes
    in a top-like display - when, why and by whom they're blocked /
unblocked
  - add archive replay and key bindings for time control

*BPF tracing export for PCP MMV*
  - Netflix developed a byte-code interpreter based on Berkeley Packet
Filter to
    generate light-weight, configurable trace events of arbitrary kernel
functions
    and existing [fd]trace probe points. This has been in-tree in mainline
Linux
    since 3.11 and might be in RHEL7 kernels one day. BPF is a sandboxed VM
    interpreter, so it's relatively safe and implicitly sanctioned for use
in production
    environments.
  - develop MMV extensions for monitoring and recording BPF tracing events,
    e.g. for blktrace style i/o latency, etc.

*Vector Customization and Extensibility*
  - JS code generator for creating new Vector charts for arbitrary PCP
metrics

*pmcollectl*
  - fix it up and finish it off to be fully comparable to collectl

*Shell interpreter extensions for pmrep*
  - #! /usr/bin/pmrep
    config defs, #include, etc ...
    reporting options

*Rust language PMAPI bindings*
  - Rust is gaining popularity - strongly typed marriage of C and python,
sort of

*Graphical analysis tools for pmlogsummary*
  - develop python class libs to read in entire PCP archives with
multidimensional
    indexing for data-science style analysis and perf data mining.
  - also feed pmlogsummary statistical data into python's extensive
data-science
    class libs (numpy et al) for analysis and graphical visualization

*PMAPI Transparent in-place archive decompression*
  - replace archive open/read/lseek/close in libpcp with zlib (gzip) calls
  - figure out how to do this with stdio calls too (or deprecate stdio
usage)
  - extend and abstract with struct file_operations for xz decompression

*Augment existing PCP Container Services*
  - existing PCP container images on bintray.com are based on fedora:latest
  - extend and generalize this to RHEL, Deb, and other Linux based distros.
  - integrate with Openshift, Kubernetes and other PaaS/orchestration
platforms

*Remote Archive PMAPI extensions*
  - investigate how to extend PMAPI for remote archive access, - or -
    develop NFS infrastructure for mounting remote pmlogger directories, -
or -
    create simple FUSE based "remote pmlogger filesystem" extensions with
IPC
  - extend PMAPI for host named contexts in archive mode (unify
live/archive replay)