Benchexec

This release fixes two security issues, all users are encouraged to update:

- Since BenchExec 2.1, the setup of the container for the tool-info module
(which was added in BenchExec 1.20) could silently fail, for example
if user namespaces are disabled on the system. In this case the tool-info
module would be executed outside of the container.
Run execution was not affected.
- The kernel offers a keyring feature for storage of keys related to features
like Kerberos and ecryptfs. Before Linux 5.2, there existed one keyring per
user, and BenchExec did not prevent access from the tool inside the container
to the kernel keyring of the user who started BenchExec.
Now such accesses are forbidden (on all kernel versions) using
[seccomp](http://man7.org/linux/man-pages/man2/seccomp.2.html)
if [libseccomp2](https://github.com/seccomp/libseccomp) is installed,
which should the case on any standard distribution.
Note that seccomp filters do have a slight performance impact
and could prevent some binaries on exotic architectures from working.
In such a case please file a [bug report](https://github.com/sosy-lab/benchexec/issues/new).

`benchexec` can now partition the Level 3 cache of the CPU for parallel runs
and measure cache usage and memory bandwidth,
at least on some Intel CPUs and if the [pqos](https://github.com/intel/intel-cmt-cat/tree/master/pqos)
and [pqos_wrapper](https://gitlab.com/sosy-lab/software/pqos-wrapper) are installed.
More information is in the [documentation](https://gitlab.com/sosy-lab/software/pqos-wrapper/wikis/home).

Furthermore, some error messages for systems without container support were improved.

This release does not add new features compared to BenchExec 1.22,
but removes several deprecated features and brings several other backwards-incompatible changes
to make BenchExec more consistent and user-friendly:

- Support for Python 3.2 and 3.3 is removed, the minimal Python version is now 3.4.
Additionally, `runexec`/`RunExecutor` continue to support Python 2.7
[until end of 2019](https://github.com/sosy-lab/benchexec/issues/438).
- Support for running benchmarks as a different user with `sudo` is removed
(parameters `--user`/`--users`).
Use container mode as better method for isolating runs.
- [Container mode](doc/container.md) is enabled by default.
It can be disabled with `--no-container`,
but this decreases reliability of benchmarking.
- If the `cpuacct` cgroup is not available,
CPU-time measurements and limits are not supported.
- Either container mode or the `freezer` cgroup are required
to ensure protection against fork bombs.
- [Niceness](http://man7.org/linux/man-pages/man2/nice.2.html)
of benchmarked process is not changed, previously it was increased by 5.
- Changes to input of `benchexec`:
- The memory limit given to `benchexec` requires an explicitly specified unit.
- Support for `<test>` tags, `<sourcefiles>` tags,
and variables named `${sourcefile_*}` removed from benchmark definitions.
Use `<rundefinition>`, `<tasks>`, and `${inputfile_*}` instead.
- Variables named `${taskdef_*}` are defined only if task-definition files are used,
and variables named `${inputfile_*}` only otherwise.
- Changes to `table-generator`:
- A column named `memUsage` is automatically renamed to `memory`.
- A column named `memory` is automatically converted to Megabytes.
Both conversions are only applied if no `<column>` tags are used.
- Changes to [run-result data](doc/run-results.md):
- In case of aborted or failed runs, no dummy results (e.g., `cputime` of 0s)
are present.
- The memory results of `benchexec` are named `memory`, not `memUsage`.
- Memory results have the unit `B` explicitly specified.
Furthermore, units are present in all attributes of the result XML files
where they were still missing.
- Result item `exitcode` is removed, only `RunExecutor.execute_run()` still returns it,
but as an object instance instead of an `int`.
Use `returnvalue` and `exitsignal` instead.
- Module `benchexec.test_tool_wrapper` is removed, use `benchexec.test_tool_info` instead.
- BenchExec (both `benchexec` and `runexec`) terminates itself cleanly after aborting all runs
if it receives one of the signals `SIGTERM`, `SIGINT` (Ctrl+C), or `SIGQUIT`.

Additionally, this release adds a fix for the container
that is used since BenchExec 1.20 for the tool-info module.
In this container, the environment variable `HOME` did not point to `/home/benchexec`
as expected but to the user's real home directory.
This broke tools like Ultimate if the `/home` was configured to be hidden or read-only.

Furthermore, we declare the following features deprecated
and plan on removing them for [BenchExec 3.0](https://github.com/sosy-lab/benchexec/milestone/8),
which is expected to be released in January 2020:

- Support for Python 2.7 and 3.4 (cf. [438](https://github.com/sosy-lab/benchexec/issues/438))
- Support for checking correctness of run results and computing scores
if task-definition files are *not* used (cf. [439](https://github.com/sosy-lab/benchexec/issues/439))

Please respond in the respective issue if one of these deprecations
is a problem for you.

- More robust handling of Ctrl+C in `benchexec`.
For example, output files are now always fully written, whereas previously
pressing Ctrl+C at the wrong time could result in truncated files.
A side effect of this is that if you call
`benchexec.benchexec.BenchExec().start()` in own Python code,
you must now add a signal handler for `SIGINT`.
The same was already true for users of `RunExecutor`, this is now documented.
- Fix Ctrl+C for `benchexec` in container mode.
In BenchExec 1.21, one would need to press Ctrl+C twice to stop `benchexec`.
- Fix unreliable container mode on Python 3.7.
- Some robustness improvements and fixes of rare deadlocks.
- Decreased overhead of `benchexec` while runs are executing.

This release contains only a few bug fixes:

- Forwarding signals to the benchmarked process (and thus, stopping runs via Ctrl+C),
was broken on Python 2.
- If the freezer cgroup was available but mounted in a separate hierarchy,
it was not used reliably as protection against fork bombs when killing processes.
- Since BenchExec 1.19, an exception would occur if a non-existing command
was started in container mode.
- Since BenchExec 1.19, copying output files from a container would occur
while subprocesses are still running and would be counted towards the
walltime limit. This is fixed, although subprocesses will still be running
if the freezer cgroup is not available (cf. 433).

- If `benchexec --container` is used, all code that is part of the tool-info
module (as well as all processes started by it) are now run in a separate
container with the same layout and restrictions as the run container.
Note, however, that it is not the same container, so any modifications
made by the tool-info module to files on disk are *not* visible in the runs!
The `test_tool_info` utility also has gained a parameter `--container`
for testing how a tool-info module behaves in a container.
- Nested containers are now supported.
Due to a change to the internal implementation of the container mode,
commands like the following succeed now:
`containerexec -- containerexec --hidden-dir /sys -- /bin/bash`.
(Some parts of `/sys` need to be excluded because of kernel limitations.)
Note that nesting `runexec` or `benchexec` is still not supported,
because nested cgroups are not implemented,
so any cgroup-related features (resource limitations and measurements)
are missing. But nesting `containerexec` and `runexec --container`
(or vice-versa) now works.
- `/etc/hostname` in container now also shows the container's host name
that exists since BenchExec 1.19.
- Change how CPUs with several NUMA nodes per CPU are handled:
BenchExec will now treat each NUMA node like a separate CPU package
and avoid creating runs that span several NUMA nodes.
Thanks [alohamora](https://github.com/alohamora)!