Reframe-hpc

ReFrame v3.6.1 Release Notes

New features and enhancements

- Add a new build system backend for building test code with Spack (1871)
- Add new logging handler for sending test data in JSON to an HTTP server (1964)
- Support test retries in the JUnit report (1965)

Bug fixes

- Generate run reports per test with `--ci-generate` (1972)
- Mark jobs properly as completed in PBS scheduler (1944)
- Fix error checking of module commands in the class Tmod backends (1971)
- Fixes `find_modules()` behaviour with flat module scheme when an environment is a substring of another one (1961)

Other

- Use PNG logo for compatibility with all browsers (1976)

ReFrame v3.6.0 Release Notes

New features and enhancements

- Add command-line option to generate a JUnit XML report (1925)
- Add a `--list-tags` command-line action to list all the available test tags (1945)
- Add a configuration option to disable automatic module conflict resolution (1941)
- Deprecate the use of the `parameterized_test` decorator (1934)
- Issue a warning if a module operation is attempted on the `nomod` modules system (1933)
- Abbreviate node lists in `FAILURE INFO` reports (1912)
- Re-implement pipeline hook mechanism and add a post-init hook (1865)
- Improve error message when parameters are undefined (1888)
- Add test API methods for skipping tests (1875)
- Show a preview of the standard error in cases of build failures (1873)
- Improve access to the namespaces of both parameter and variable built-ins (1819)
- Add a `time_limit` partition configuration parameter and a `build_time_limit` variable for regression tests (1860)
- Allow setting the test description and executable name directly in the class body (1862)
- Minimize the list of doc requirements (1847)

Bug fixes

- Fix regression in setting the graylog server address (1956)
- Warn when a check search path does not exist (1949)
- Properly check exit status of `ensurepip` in `bootstrap.sh` (1953)
- Fix crash when `OSError` occurs during dumping of test cases (1926)
- Fix crash when `sourcesdir` is set to `None` in class body (1910)
- Fix missing path to build stdout when collecting EasyBuild installed modules (1891)
- Fix squeue backend scheduler crash (1907)
- Fix rescheduling condition for asynchronous execution policy (1904)
- Color `bootstrap.sh` only if stdout is a terminal (1898)
- Fix bootstrap script for `pip`-less Python installations (1900)
- Set stdin to null when launching processes from ReFrame, so that processes do no think they are on a tty (1879)
- Fix the call order on the `__set_name__` hook for built-in variables (1881)
- Do not reset `sourcesdir` to `None` if it is set to a URL (1874)
- Fix typo in variable name (1866)

Other

- Pull base images for CI instead of building them from scratch (1909)
- Update documentation of built-ins and update tutorial examples removing the use of `__init__` (1940)
- Make basic configuration example more intuitive w.r.t. to modules systems (1932)
- Change favicon to the ReFrame logo (1924)
- Add URL to Twitter badge (1922)
- Add `-pthread` to the multithreaded tutorial test (1870)
- Add tutorial about reusable tests (1895)
- Update tutorial examples to use the parameter builtin (1857)
- Various documentation fixes (1859)

ReFrame v3.5.3 Release Notes

New features and enhancements

- Deprecate the use of the `parameterized_test` decorator (1934)
- The decorator is set to deprecate with ReFrame 3.6.0
- Issue a warning if a module operation is attempted on the `nomod` modules system (1933)
- Abbreviate node lists in `FAILURE INFO` reports (1912)

Bug fixes

- Fix crash when `OSError` occurs during dumping of test cases (1926)

Other

- Pull base images for CI instead of building them from scratch (1909)
- Make basic configuration example more intuitive w.r.t. to modules systems (1932)
- Change favicon to the ReFrame logo (1924)
- Add URL to Twitter badge (1922)

ReFrame v3.5.2 Release Notes

New features and enhancements

- Re-implement pipeline hook mechanism and add a post-init hook (1865)
- Improve error message when parameters are undefined (1888)

Bug fixes

- Fix crash when `sourcesdir` is set to `None` in class body (1910)
- Fix missing path to build stdout when collecting EasyBuild installed modules (1891)
- Fix squeue backend scheduler crash (1907)
- Fix rescheduling condition for asynchronous execution policy (1904)
- Color `bootstrap.sh` only if stdout is a terminal (1898)
- Fix bootstrap script for `pip`-less Python installations (1900)
- Do not skip test if `sanity_patterns` is not set (1897)
- Set stdin to null when launching processes from ReFrame, so that processes do no think they are on a tty (1879)
- Fix the call order on the `__set_name__` hook for built-in variables (1881)

Other

- Add `-pthread` to the multithreaded tutorial test (1870)
- Add tutorial about reusable tests (1895)

ReFrame v3.5.1 Release Notes

New features and enhancements

- Add test API methods for skipping tests (1875)
- Skip test if `sanity_patterns` is undefined (1876)
- Show a preview of the standard error in cases of build failures (1873)
- Improve access to the namespaces of both parameter and variable built-ins (1819)
- Add a `time_limit` partition configuration parameter and a `build_time_limit` variable for regression tests (1860)
- Allow setting the test description and executable name directly in the class body (1862)
- Minimize the list of doc requirements (1847)

Bug fixes

- Do not reset `sourcesdir` to `None` if it is set to a URL (1874)
- Fix typo in variable name (1866)

Other

- Update tutorial examples to use the parameter builtin (1857)
- Various documentation fixes (1859)

ReFrame v3.5.0 Release Notes

New features and enhancements

- Add processor and device information to the system partitions and make them accessible from the tests (1773)
- Add support for building tests using EasyBuild (1719)
- Add a new `prepare_cmds` partition configuration parameter which allows to emit commands before loading any environment (1803)
- Improve the container platform API exposed to tests (1775)
- Respect the entrypoint and command defined inside the container image.
- Deprecate workdir and commands fields.
- Introduce the command field.
- Allow to skip pulling of the container image from a container registry.
- Expand container platform tutorial example.
- Log reference values of all performance variables in a single line (1816)
- Allow deprecation of fields in a future version and issue deprecation warnings also in prereleases (1804)
- Support parameter and variable access directly from the class body (1806)
- Add built-ins to manage test variables and parameters (1699)
- Allow logging arbitrary job attributes (1788)
- Comply with the semantic versioning specification (1761)
- Add new action for generating dynamic Gitlab pipelines (1641)
- Add auto completion support to ReFrame (1685)
- Add a new family of sanity functions that looks for patterns inside text instead of files (1705)
- Support multiple job submissions when slurm's job submit limit is reached (1704)
- Add `--maxfail` option to stop execution after a certain number of failures (1676)
- Set executable bit in generated job script files (1716)

Bug fixes

- Fix passing of stagedir to container platforms (1842)
- Properly pass `num_tasks_per_socket` to the scheduler backends (1837)
- Remove VERSION variable from docs Makefile (1831)
- Fix status printout in case tests are retried (1814)
- Deepcopy inherited variables from the parent classes (1810)
- Make `attrs()` utility function pass when an `AttributeError` is raised (1802)
- Print a FAILURE result if not all test cases have completed (1801)
- More documentation fixes (1798)
- Document `osext.cray_cdt_version()` more precisely (1796)
- Fix `AttributeError` crash when handling OS errors while writing a test case file (1785)
- Move shell completion scripts under `share/completions/` (1787)
- Add Python version check to bootstrap script (1762)
- Various documentation fixes (1771)
- Fix bug and parameterize cuda samples tests (1758)
- Add missing required packages in 'setup.py' (1767)
- Fix validation of the generated Gitlab pipeline files (1764)
- Fix skipping of test cases in case of unresolved dependencies (1751)
- Fix "New in version" annotation in documentation for certain new features (1754)
- Fix parameter value specification in the `parameter()` directive (1722)
- Fix handling of single character scheduler backend options passed through `-J` (1718)