Benchexec

Latest version: v3.29

Safety actively analyzes 714973 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 12

3.29

- The name of the wheel and tar archives in BenchExec releases
changes from `BenchExec-$VERSION*` to `benchexec-$VERSION*`
in accordance with Python standard PEP 625 and PyPI requirements.
- Some new and improved tool-info modules.

3.28

- Some new and improved tool-info modules.

3.27

- Support for [version 2.1 of the task-definition format](https://gitlab.com/sosy-lab/benchmarking/task-definition-format/-/tags/2.1).
- Updated tool-info modules for SV-COMP'25 and Test-Comp'25.

3.26

This release brings several important fixes for cgroups v2
and all users on systems with cgroups v2 are strongly recommended to upgrade.

- Fix regression from BenchExec 3.25 for detecting memory-limit violations of a run.
The changes related to the LXCFS integration in the last version caused the
problem that a run that hit the memory limit was no longer marked as "out of memory"
but as "failed" and did not have any measurement results
(only for cgroups v2 and with container mode enabled).
Note that the memory limit was still correctly working
and also measurements produced for non-failed runs were correct.
- Fix AssertionError when creating a systemd scope for cgroups v2 usage.
This was caused by a race condition and did occur only with a certain probability
depending on the system load.
- Fix warning about left-over cgroup during run cleanup for cgroups v2.
- More robust cleanup of cgroups created by the benchmarked process.
- Fix installation of our Python package on Windows.
- Several new and improved tool-info modules.

3.25

Not secure
- Support for fuse-overlayfs as alternative to kernel-based overlayfs
BenchExec uses overlayfs for providing a virtualized file-system
in its containers (unless configured otherwise by the user).
Unfortunately, the overlayfs implementation in the Linux kernel
does not support all use cases such as overlayfs for the root directory
or triple-nested containers (cf. 776 and 1067).
Now BenchExec makes use of the alternative implementation
[fuse-overlayfs](https://github.com/containers/fuse-overlayfs/)
if installed in version 1.10 or newer as an automatic fallback when necessary.
This makes BenchExec work again in its default configuration
without requiring parameters like `--read-only-dir /`.
We are glad about this long-awaited feature
being contributed by [GSoC participant](https://summerofcode.withgoogle.com/programs/2024/projects/UzhlnEel)
[younghojan](https://github.com/younghojan)! Thanks!
- Improve LXCFS integration
If installed, BenchExec uses [LXCFS](https://github.com/lxc/lxcfs)
to provide a better virtualization of the environment visible inside the container,
for example by virtualizing the uptime.
Now we use LXCFS also to virtualize CPU information in `/proc/cpuinfo`
and `/sys/devices/system/cpu`.
This allows the benchmarked process to see more easily
how many CPU cores they are allowed to use.
- New tool-info module for `super_prove`.

3.24

Not secure
- Duplicate tasks in benchmark definitions ignored.
Inside a single `<tasks>` tag in a benchmark definition it can happen
that a specific task is included more than once.
These copies of the task would be exactly identical and thus redundant,
but they also create problems for benchmarking because they violate
the assumption that tasks have unique identifiers.
So now `benchexec` simply ignores such duplicates of tasks.
This makes it easier to write benchmark definitions with broad wildcards,
for example if you want to include both `foo*.yml` and `*foo.yml`
but some tasks match both wildcards.
- Two new tool-info modules for KeY CLI and Pono.

The HTML tables produced by `table-generator` also contain a major
technical improvement thanks to our GSoC participant [EshaanAgg](https://github.com/EshaanAgg),
but no user-visible changes.

Page 1 of 12

© 2025 Safety CLI Cybersecurity Inc. All Rights Reserved.