Pcp

- Client tools and utilities:
pmchart: fix chart auto-scaling under fetch error conditions
pmseries: fix label reporting state bug with multiple series
pcp-atop: fix write mode (-w) handling of proc vs hotproc metrics
pcp-atop: resolve intermittent qa/1080 valgrind failure
- PMDA additions, enhancements and bug fixes:
pmdalinux: fix hugepage metric value calculations
pmdalinux: fetch more network metrics in --container namespace
pmdaproc: fix memory leak in pidlist refresh
pmdaproc: remove legacy /proc/ksyms and System.map readers
- Server-side utilities and log management scripts:
pmproxy: complete REST API pmStore support
pmproxy: complete the secure TLS connection support
pmproxy: add indom profile support to the async API
pmproxy: /pmapi/children REST API for pmGetChildrenStatus(3)
pmproxy: fix a double-free memory corruption
pmproxy: do not report indom for metric if null
pmproxy: indom help text API support, other small compat tweaks
pmlogextract: add -x option for more resilient merging
logutil: fix broken pipe errors from utilproc.sh _is_archive()
logutil: fix pmlogger and pmlogger_daily systemd services
pmiectl: reflect changes to pmlogger start process in that of pmie
- libpcp, libpcp_pmda, libpcp_mmv, libpcp_web and language bindings:
libpcp_pmda: close help files after mmap'ing contents
libpcp_web: fix memory corruption on fetch code path
libpcp_web: fix memory corruption during metric scrape
libpcp_web: add support for metric help text extraction
libpcp_web: fix discovery segfaults deleting monitored paths
libpcp_web: fix valgrind bad memory accesses for aggregate metrics
libpcp_web: make the dictDelete function available
- Misc build, infrastructure and packaging updates:
configure.ac: changes for OpenSSL on OpenSUSE
build: rework libpcp_web exports file exposed-symbols
build: fix warnings from Qwt code with recent compilers
build: ensure Qt qIsNaN macro available where its needed
build: assorted strncpy() changes for gcc 8 warnings
build: pylint work for missing parts of the pcp modules
- Documentation and QA infrastructure:
docs: add a deprecation note for pcp-collectl in its man page
docs: add a deprecation note for pmwebd in its man page
BCC QA: Detect BPF compilation failures early
azure CI: fail on first error when building PCP
CI: change cppcheck --std=posix (deprecated) to --library=posix
CI: cppcheck: temporary suppress objectIndex error in siphash.c
build: fix some small make check failures on f30 and rawhide
docs: fix new NFS client per-op error count help text typo
docs: correct pattern match parameter name in series REST API
qa/README: add notes for ssh access to pcpqa login with selinux
README.github: document the github workflow

- Client tools and utilities:
pcp-dstat: prevent comma-only lines from being added to csv output
pcp-dstat: refactor time printing code for csv output
pcp-dstat: fix misaligned headers in CSV output
pcp-dstat: ensure duplicate columns collapse to just one
pcp-dstat: fix handling of --bits command line option
pcp-dstat: fix typo in printtype detection
pcp-free: Adding support for -r/ --terabytes flag.
pmchart: fix handling of -z timezone option with no source
pmrep: fix wait.formula for collectl-dm-sD and collectl-sD
pmproxy: consolidate the pmseries/grafana REST API
pmproxy: removed Grafana code to prepare for new API
pmseries: add a value retrieval API by series identifier
pmproxy: add new, async PMWEBAPI(3) REST API implementation
pmproxy: small improvements to messages logged at startup
pmproxy: initial secure socket connection code for pmproxy
pmproxy: fix memory leaks picked up by coverity scanning
shell completions: add support for new -m option to pmlogcheck
- PMDA additions, enhancements and bug fixes:
pmdabcc: fix tracepoints module for bcc 0.10.0 and later
pmdanfsclient: add per-op, per-mount rpc errors metrics
pmdalmsensors: improvements in name parsing and error handling
pmdalinux: add active and total slabs to slabinfo v2 parsing
pmdalinux: fix several unix socket and icmp6 metrics
pmdabcc: fix install (due to README rename)
nfsclient PMDA: fix some 32 -> U32 metadata types
pmdalinux: code re-factoring to reduce coverity scan noise
- Server-side utilities and log management scripts:
pmie_check: add systemctl pmie service dependency
logutil: add -m option to pmlogcheck to skip log volume checking
logutil: change pmlogger_daily_report to be an opt-in service
pmlogrewrite: fix compiler warnings, other small changes
- Security Enhanced Linux:
selinux: Fedora 29 changes for libvirt PMDA and pmmgr
selinux: remove old rules that have been superceded
selinux: conditionally re-introduce files_mmap_all_files() in policy
selinux: more unconfined_service_t (conditional) changes
src/selinux: more tweaking for unconfined_service_t (conditional)
configure.ac: more selinux changes
selinux/build: replace files_mmap_all_files() with explicit policy rule
configure.ac: fix bpf conditional test for selinux
selinux: Add pcp_domain level read access to maps
src/selinux/pcpupstream.te.in: changes from QA on vm26 (Fedora 27)
src/selinux/README: explain FOO meta-syntax elements
selinux: allow a pmlogger_check.log file unlink/rename
- libpcp, libpcp_pmda, libpcp_mmv, libpcp_web and language bindings:
libpcp_web: resolve memory leak in map handling, from coverity scan
libpcp: prepare for the new async pmwebapi(3) implementation
python: fix pmcc time window handling
libpcp_web: add some pmDebug diags to discovery
python api: add explicit Py_None test on callback results
libpcp_web: update a couple of snprintf calls to pmsprintf
libpcp_web: update Redis client code with TLS/SSL support
libpcp: rework labels code to correct handle maps
libpcp_pmda: change semantics for empty indom cache file
- Misc build, infrastructure and packaging updates:
build: update openssl package checks in configure.ac
build: pmproxy.conf should be config(noreplace)
debian build: couple of packaging changes
Debian builds and libssl et al
build: update gitignore file to skip over a header symlink
build: add libuv-devel dep for pcp-testsuite building sources
configure.ac: need some more care with cleanup in selinux land
build: add provisioning/release.sh script for vagrantfiles
build: drop -y for bison
build: bump version numbers for next planned release
build: ensure new systemd timer configs are packaged
build: correct install locations of pmproxy configs
build: update fedora rpm changelog with another bz
build: remove selinux-policy-updates.patch from fedora.spec
build/qa: add new deps to dockerfiles,
build: rename libopenssl-dev to libssl-dev in debian/control.main
- Documentation and QA infrastructure:
qa, pmproxy: fixes to derived metrics registration
qa: add minimal valgrind 3.15.0 suppressions (f30+)
qa: improve filtering of new service discovery names
docs: use same preamble for all api man pages
docs: Mention pcp archive support in sysstat 12.0.5 in sar2pcp manpage
qa: start simple test to trigger test setup in build phase
qa: include test durations
qa/admin/check-vm et al: changes for OpenSUSE Leap 15.0
qa: improve test report generation
qa: use travis only for coverity scan
qa: increase min kernel version for ucalls test 1173
qa: update slabinfo verification test to check PMDA changes
qa: replace travis with Azure Pipelines
qa: update tests for correct unix socket and icmp6 values
qa: additional filtering for latest valgrind versions
qa: fix incorrect (always null) diagnostic in derived.c
qa: many: purge explicit $seq.notrun uses and replace by _notrun()
qa/common.check: add $here prefix to $seq.pre-avc for rm in _notrun()
qa/admin/pcp-daily: conditionally run freespace
docs: all OSX instructions are in INSTALL.md now
docs: Tweak copyright years

- Client tools and utilities:
pcp-dstat: fix invisible gray coloring ansi escape code
pcp-dstat: GH624 handle the end-of-archive condition correctly
pcp-pidstat: drop unused -h/--host option
pcp-atopsar: RHBZ1673996 fix -A for replying from archive
pcp-atopsar: improve handling of mark records and log end
pcp2spark: minor output tweak
pcp2zabbix: send first metrics without waiting
pcp2template: aid to creating new pcp2xxx tools
pmlogrewite: diagnostic changes to use new flags
pmrep: tweak output in case of no values available with -X
pmrep: introduce fixed header (-7, --fixed-header option)
pmrep/pcp2xxx: more helpful error message when pmcd not running
pmsnap: fix leak of Xvfb processes
pmgadgets pmgsys.py: make this work with Python2 and Python3
pmgadgets: fixes so it builds again with Qt5
pmview: set correct default time if no -t on command line
- PMDA additions, enhancements and bug fixes:
pmdalinux: fixes to parsing of /proc/zoneinfo
pmdalinux: fix two metrics with no values on some kernel versions
pmdalinux: updates to SYSV IPC metrics
pmdaproc: fix Linux cgroup *service_bytes.* metric metadata
pmdabcc: uprobe_hits: fix use of wrong variable
pmdabcc: make cachestat module currently working with ext4 only
pmdabcc: add kprobe hit count BPF/Python module
pmdabcc: keep uprobes_hits logging consistent with other hit modules
several pmdas: remove PM_ERR_AGAIN uses, correct PMDA_FETCH_* macros
txmon: update from PMDA_INTERFACE_2 to PMDA_INTERFACE_7
pmdaperfevent: deprecate pmdapapi, upgrade to perfevent
pmdanvidia: fix up indom help text macro name typo
pmdazimbra: major overhaul
- Server-side utilities and log management scripts:
systemd: fix Environment parameter usage
pmdaroot: ensure zombie PMDA processes are reaped
pmproxy: initial pmseries and grafana datasource REST APIs
pmproxy, libpcp_web: support for pmproxy configuration file
pmcd: RHBZ1641161 add support for PMDA suspend and resume (fencing)
pmlogger_daily.sh: try harder in the presence of errors
pmwebd: fix a broken diagnostic causing crashes
- Security Enhanced Linux:
RHBZ1693332 update BPF permissions for pmdabcc
fix some fallout from the systemd timer transition
allow pcp_pmcd_t module_request access from class system
allow the nvidia pmda to dlopen libnvidia-ml
- libpcp, libpcp_pmda, libpcp_mmv, libpcp_web and language bindings:
perl PMDA.pm: expose PMDA_FETCH_* macros
pmapi.h: add appl3, appl4 and appl5 debug flags
libpcp_web: refactoring and support for Redis cluster protocol
- Misc build, infrastructure and packaging updates:
builddefs: rework qmake recipe
replace libdbd-pg-perl Debian dep with python3-psycopg2
add python-devel (v2) explicitly for el7 vagrants
update generated rpm spec to drop 'Group' keyword
vagrant: various fixes, improvements to provisioning scripts
configure: set pcp_rc_dir correctly for FreeBSD
configure: allow --with-qt=DIR to specify QT installation dir
- Documentation and QA infrastructure:
qa: adjust for Python 3 only systems
qa: massive change to remove init script chatter
qa/check et al: rework callback workflow
qa/common.check: change to using systemctl if available
qa/check.callback captures the before+after state of pmcd and pmlogger
qa/check.callback captures any new selinux AVC denials from each test
INSTALL.md: update and clarify a few things

- Client tools and utilities:
pcp-dstat: improve --time option handling with no other arguments
pcp-dstat: handle complete instance removal with --full mode
pcp-atop: update to v2.4.0 atoptool.nl sources
. reporting pressure stall information
. reporting infiniband statistics
. reporting perf_event statistics
. reporting nvidia GPU stats
pmseries: error handling improvements when parsing fails
pmseries: ensure commas separate labels in metadata reporting
pmseries: improve user friendliness of option handling
pmseries: shell completions added
pmseries: install a pmseries.conf
pmseries: globbing support on global name lookups too
pmrep: add support for sorting -X output by values
pmrep: fix a dynamic header corner case with missing values
pmlogger: add systemd timers as cron alternative
pmie: add systemd timers as cron alternative
pmie: re-implement 'instancename' indom lookup
pmie: fix for segv and bad sum_inst() use
pmview: fix a missing timeout slot connection
pmview: add in the step() signal/slot connections
- PMDA additions, enhancements and bug fixes:
linux: add support for pressure-stall information metrics
openbsd: remove remaining /dev/kmem reading
openbsd: fixes for 32 bit kernels
openbsd: add kernel.all.uptime
openbsd: fix cpu time metrics - SMP and CP_SPIN support
netbsd: fixes for 32 bit kernels
netbsd: add kernel.all.uptime
bcc: handle gone processes in ustat module
bcc: adjust biolatency module tracepoints
dm: new VDO device metrics
dm: fix mishandling of missing VDO kernel stats
nvidia: add support for per-process nvidia metrics
pmproxy: extend MMV to support new pmproxy metrics
- Server-side utilities and log management scripts:
pmcd: allow use of an optional pmcd.conf.access file
rc_pmcd: optional PMDA ./Upgrade to resolve upgrade issues
pmproxy: fix shutdown issues in --timeseries mode
pmlogger_check: make pmlogconf check optional
pmlogconf: add per-CPU interrupts metrics to the zeroconf set
pmlogconf: additional /proc/vmstat metrics to default set
pmwebd: tweak y-axis labels in png output
pmwebd: fix an open file descriptor leak
- libpcp, libpcp_pmda, libpcp_mmv, libpcp_web and language bindings:
libpcp: fix compilation for --with-threads=no configure
libpcp: exclude indom instances level labels in result from pmLookupLabels
libpcp: the big libpcp lock is no longer recursive
libpcp: refactor registered.mutex use
libpcp_web: add .ini configuration file parsing
python pmapi: add __str__ handler for class pmLabelSet
python api: Rename pmconfig.py helper function - get_sorted_results()
- Misc build, infrastructure and packaging updates:
python: fix Python installations flags for DIST_ROOT=
Revert "pmlogger_daily: use whole day granularity for PCP_COMPRESSAFTER"
Wider use of automated builds through Vagrant for QA and releases
build: add -I paths for cppcheck, fix travis-ci checking
build: add conditional BuildRequires libuv-devel in fedora.spec`
travis-ci: tweak service timeouts
pmns: don't package .NeedRebuild for SUSE distros
pmns/Rebuild: add warning (with -v) for skipped root_foo file
Qt: correctly insert compiler flags, release vs debug build fixups
configure, Makepkgs, et al: move Qt version detection into configure
- Security Enhanced Linux:
Update shm,sem permissions for userdomains
Reduce size of policy package (now that we have overarching rules)
- Documentation and QA infrastructure:
qa/admin/check-vm: refactoring and customization for all platforms
qa/common.check: improvements for pmlogger and pmie tests
build: add fedora/rhel testsuite dep on redhat-rpm-config
docs: update pmLookupLabels(3) and pmseries(1)
qa: pervasive NetBSD, OpenBSD and OpenIndiana changes

- Client tools and utilities:
- Add pmseries(1) for fast, scalable time series querying (experimental)
- pmrep, pcp2xxx: add support for the --names-change option
- pmrep, pcp2xxx: support for the --ignore-unknown metrics option
- Implement the dstat --full option to expand instances
- Add an 'entropy' dstat plugin configuration file
- pcp2elasticsearch: support for es_search_type config flag
- pcp2elasticsearch: reduce logging in server down scenarios
- pcp-atop: fix an incorrect use of varargs vfprintf interface
- pmlogextract: numerous scalability improvements
- pmchart: fix sigsegv on non-existant archive via command line
- PMDA additions, enhancements and bug fixes:
- FreeBSD: add kernel.all.uptime, mem.freemem metrics
- FreeBSD: fix 32 vs 64 bit types on memory and CPU metrics
- FreeBSD: fix units on some disk (bytes) metrics
- FreeBSD: fix CPU count logic error affecting per-CPU metrics
- perfevent: add hv_24x7 perf events
- perfevent: add support for raw perf events
- pmdapostgresql: add optional 'osuser' to configuration
- Log rotation, management scripts, compression and server-side utilities:
- zeroconf: change logging frequency for proc metrics to 30 seconds
- rc scripts: improve systemd interaction for local pmie
- pmlogger_daily: whole day granularity for PCP_COMPRESSAFTER config
- pmlogger_check: resolve a 'cat: broken pipe' warning message
- pmlogrewrite config files for FreeBSD metadata changes
- pmproxy: auto-loading of historical timeseries from discovery
- pmproxy: add --timeseries option to switch libpcp/uv versions
- pmwebd: tolerate live archives with multi-hour-old volumes
- Misc build, infrastructure and packaging updates:
- Fixed RHEL-5 build packaging for nfsclient and lmsensors PMDAs
- Enable dstat -> pcp-dstat binary symlink on more platforms now
- Drop the elasticsearch python[23]-urllib3 spec file dependency
- Changes for Mac OS X HomeBrew build and packaging
- Support for Mac OS X Mojave latest release
- libpcp, libpcp_pmda, libpcp_mmv, libpcp_web and language bindings:
- libpcp: instance filtering for derived metrics
- libpcp: add scalar() for derived metrics
- libpcp: fix process existence checks on FreeBSD
- libpcp: be more defensive in pmMergeLabelSets
- libpcp: add an integrity check in __pmMergeLabelSets
- python pmapi: fix a crash in labels wrapper functions
- python pmapi: fix a crash in fetchgroup clearing function
- python pmapi: rework labels lookup, merge and free functions
- python pmconfig: fix logic error in get_sorted_results function
- python logimport: fix wrong variable usage in pmiPutLabel
- Security Enhanced Linux:
- Generic policy rules to allow pcp_pmcd process to read system objects
- Fix the build on older selinux versions (RHEL6 era)
- Documentation and QA infrastructure:
- logimport(3): new pmiPutLabel(3) API function for adding labels
- docs: improve formatting for Vector containers screenshot
- docs: tweaks for pcp-dstat.1 to use same conventions as pmrep.1
- docs: correct typo in INSTALL notes for MacOSX
- docs: fix formatting in mmv(5) labels section, add more detail
- docs: add python synopsis and examples in pmMergeLabels(3)
- docs: add python synopsis in pmLookupLabels(3)
- qa/README.vm: instructions for setting up VMs for QA
- qa: add kernel group for base OS PMDA testing
- qa: add sample.controller.mirage to sample PMDA
- qa: numerous Vagrantfile updates and improvements
- qa: add helper tool for reporting internal/external sha IDs
- qa/admin/check-vm: Mac OS X changes
- qa/admin/check-vm et al: refactor for Vagrant

- Client tools and utilities:
- pcp-dstat: fix cursor offsetting in unusual terminal states
- pcp-dstat: additional exception handling for I/O error on exit
- pcp-iostat: remove variable named 'await', now a keyword
- pmchart: ensure only one system activities menu item is presented
- pmchart: fix incorrect use of std::vector::resize() for single stepping
- pmie: fix small error in expression tree binding
- pmie: fix instance info propogation when metrics become available
- pmie: fix rare corner case logic errors
- pmrep, pcp2{json,xml}: properly handle broken pipe
- PMDA additions, enhancements and bug fixes:
- pmdapodman (new): export podman container and pod state and statistics
- pmdaroot, pmdaproc: add support for 'podman' containers
- pmdapostgresql: fix regression in config parser
- pmdapostgresql: add port number to config options
- pmdapostgresql: tone down the missing table and column warnings
- pmdabcc: add ucalls/ustat Tcl support
- pmdalmsensors: re-implement in python, update
- pmdadocker: fix a missed lock release issue coverity reported
- pmdaelasticsearch: Change perl implementation to python
- pmdaelasticsearch: enable user selection from conf file
- pmdanetbsd: add hinv.machine
- pmdaopenbsd: fixups for OpenBSD 6.4
- pmdasample: add sample.ghosts.* metrics
- Log rotation, management scripts, compression and server-side utilities:
- pmlogger_daily: change workflow to cull early and reliably compress
- pmcd_wait: default to using unix domain sockets if supported
- pmlogrewrite: Bug: Too many labelsets deleted.
- pmlogrewrite: Allow instance names to be used to specify instances labels.
- pmlogrewrite: Remove support for "ALL" as a text record type.
- pmlogrewrite: escape characters in the content string for text records.
- pmlogrewrite: Help Text: Support filtering by content.
- pmlogrewrite: Help Text: Additional text type allowed.
- pmlogrewrite: fix error handling when duplicate output files found
- libpcp, libpcp_pmda, libpcp_mmv, libpcp_web and language bindings:
- pmdaproc.sh: change a few warnings to use $prog instead of Install
- libpcp: fix buffer length error in pmAtomStr_r(3)
- python api: propagate pmFetch(3) result for fetchgroup users
- Security Enhanced Linux:
- selinux: update container policy for pmdaroot/podman
- selinux: fix bpf related build issue
- selinux: RHBZ1626487 unix_read on gpsd_t:shm
- selinux: RHBZ1633211 bpf class & inheritence issues
- selinux: RHBZ1634205 add pmlogger/cache_home context dir:search
- selinux: RHBZ1635394 fix configure glue to trigger existing rule inclusion
- selinux: RHBZ1535522 add pmcd/cache_home context dir:search
- Misc build, infrastructure and packaging updates:
- build: Fix FreeBSD Qt builds
- build: fix a number of Debian packaging issues
- build: pcp-gui debian packaging made to be conditional
- build: rpm spec and manifest updates for pmdadm dmstats
- build: deprecate pcp-collector and pcp-monitor RPM packages
- build: add option for python bcc packages from iovisor repos
- build: add --check option to Makepkgs script for static checks
- build: corrections to dstat package replacement in rpm spec
- Documentation and QA infrastructure:
- docs: pcp2spark doc and example code tweaks
- docs: update pmdadm(1) man page
- docs: update LOGIMPORT(3) man page to mention python wrapper
- docs: dbpmda(1) - correct default for -q ... it is 3 not 5
- docs: updates the containers tutorial for 'podman'