Reframe-hpc

ReFrame v2.21 Release Notes

New features and enhancements

- Print repository's revision hash when ReFrame's version is requested (1107)
- Make job completion time available as a logging variable (1090)
- Emit job options to `aprun` command line (1067)
- Add nodelist in failure report (1077)
- Improve error message when no action is specified (1073)
- Add support for TMod version 3.1 (935)
- Allow passing arbitrary options to a container platform (1064)
- Add support for TACC's ibrun parallel job launcher (1066)
- Add a `print()` sanity function (1062)
- Change command line option `--flex-alloc-tasks` to `--flex-alloc-nodes` (1058)
- Rename `make_deferrable()` to `defer()` and make it publicly available along with `evaluate()` (1050)
- Support for running dependent tests with the asynchronous execution policy (1022)
- Add support for test dependencies in serial execution policy (1015)

Bug fixes

- Fix crash of local scheduler with flexible allocation (1088)
- Fix unit test failure for scheduler backends when run alone (1076)
- Ignore invalid Slurm nodes in flexible task allocation (1054)

Regression tests

- Update maintainers list (1096)
- Add check for Slurm error message when no constraint is given (1104)
- Adapt HaloExchangeTest/Alltoallv for Kesch (1105)
- Fix netCDF tests for Kesch (1103)
- Fix sanity error in jacobi_perftools check (1102)
- Update AllocSpeed check for Kesch (1098)
- Remove slurm switch option from dom (1086)
- Fix CCE flags in HelloWorld tests for Kesch (1085)
- Add NumPy test (1021)
- Add strided access bandwidth benchmark (959)
- Add check for TensorFlow2+Horovod (1072)
- Fix reference of the tensorflow+horovod test (1071)
- Update automatic Fortran arrays check (1045)
- Update compiler flag for Gperftools check (1040)
- Update compiler version for Mpip check (1041)
- Update tool version for Intel tools check (1039)
- Add Tiger as valid system (1035)
- Add JupyterHub submission test (1033)
- Fix tags in Gridtools checks (1034)
- Port Perftools check to CLE7 (1025)
- Update alloc speed test (1024)
- Update HPX "hello world" program name to v1.3.0 (1020)

Other

- Use `--flex-alloc-nodes` option to run user tests in CI (1100)
- Fix quotes in CI script (1036)
- Document the test dependency feature (1084)
- Stress out that hostnames are Python regexes (1097)
- Add link to the contribution guide in the README page (1080)
- Add slides from SC19 (1079)

New features and enhancements

- Write the number of tasks in performance report (976)
- Enable running containers inside the test pipeline (853)
- Lazy evaluation of job-related attributes and environments (961)
- Add SSH job launcher (952)
- Add syntax for running functions after dependency resolution (951)
- Enable easy testing of non-default Cray programming environments (949)
- Replace `reframe_module` configuration option with command line option (946)
- Improve and simplify the topological sorting of the test cases (934)
- Add new syntax for attaching arbitrary regression test methods to pipeline stages (920)
- Print compiler commands and flags when showing the environment configuration (918)

Bug fixes

- Support inheritance in pipeline hooks (994)
- Do not use login shell when submitting jobs (998)
- Fix unit tests leaving stale files behind (948)
- Use login shell when building executables (911)

Regression tests

- Fix buffer decode error in MpiP check (1013)
- Fix nvidia and libsci/mkl resolve tests (1011)
- Bump PE version in module names (1007)
- Fix Scorep_mpi_omp check (1006)
- Minor fixes for CLE7 (1003)
- Update Intel_VTuneAmplifier (1004)
- Add PrgEnv-pgi to SPEC-ACCEL test (999)
- Port GpuBandwidthCheck to Tiger (1000)
- Remove unnecessary scheduler options from the DGEMM benchmark (990)
- Port GPUDIRECTACC check to Tiger (971)
- Port OSU check to Tiger (981)
- Add mem-per-cpu resources to ALLOC_SPEED check for Tiger (977)
- Remove cce compiler from SCORE-P check (958)
- Port DGEMM check to Tiger (978)
- Port OPENACCCUDACPP to Tiger (973)
- Port GPUDIRECTCUDA check to Tiger (972)
- Port CPUTARGET check to Tiger (984)
- Port SHMEM to Tiger (983)
- Port FFTW check to Tiger (982)
- Port KERNEL_LATENCY check to Tiger (980)
- Port GPUBURN check to Tiger (979)
- Port collectives halo exchange check to Tiger (970)
- Port automatic arrays check to Tiger (968)
- Port NETCDF check to Tiger (967)
- Remove workaround for NetCDF C++ test with PGI (987)
- Tag checks that use external resources (964)
- Add specific tag to checks that test Cray PE (969)
- Update NVML with source from cudatoolkit (965)
- Port GPUDirect check to Tiger (966)
- Update Paraview test (962)
- Update IPM check with new Cray path to PAPI (`PAT_BUILD_PAPI_LIBDIR`) (957)
- Remove tags from nsimd checks (956)
- Update reference for AutomaticArrays test on Dom (955)
- Update Gromacs reference value (950)
- Fix cuda checks on Kesch (943)
- Adapt module versions to PE19.09 (945)
- Enable IOR test on Daint (937)
- Update reference values of G2G [1] check (939)
- Make the MpiP check generic (938)
- Add performance report to the Intel Advisor test (916)
- Enable performance reporting in the flexible alltoall OSU benchmark (908)
- Add CPU memory latency test (874)
- Fixing OpenACCFortran checks on Kesch (928)
- Update GitHub repository in collectives MCH check (929)
- Fixing I/O checks on Kesch (925)
- Adapt MCH checks to CrayPE 17.06 on Kesch (RH7.5) (913)
- Add performance report to Empirical Roofline Tool test (917)
- Increase iterations for async kernel latency test (910)
- Add libVC simd test (899)
- Add nsimd simd test (900)

Other

- Add config file for additional system (960)
- Fix tags of modified regression tests to run in ci (914)
- Run benchmark tests in CI with predefined number of tasks (912)
- Fix literal include lines in performance logging documentation (1009)
- Fix syntax of container tutorial example (1008)
- Document the container platforms feature (985)
- Update documentation links in README (989)

ReFrame 2.19 Release Notes

New features and enhancements

- Print debug info when preload environment fails to load (906)
- Enable modifications to ReFrame's environment per system basis (887)
- Add automatically a default `*` reference for performance checks (866)
- Remove support for the very old test syntax using `_get_checks()` (876)
- Fall back to a generic system configuration if system auto-detection fails (867)
- Allow self loops in module mappings (855)
- Implement topological sort of the test cases (815)
- Limit concurrent build jobs to `1` by default in build systems (810)

Bug fixes

- Fix wrong error message emitted when a user module is loaded with the `-m` option (886)
- Do not ignore `sourcepath` if build system is explicitly set (875)
- Change to working directory before merging job array files (869)
- Make Slurm WLM backends aware of job arrays (840)
- Fix how Slurm states are mapped internally by the Slurm backend (845)
- Fix ReFrame hanging when Slurm job gets OOM killed (835)
- Fix timing in the allocation speed test (824)
- Fix bug in performance report output (818)

Regression tests

- Update tensorflow/horovod tests for latest modules (904)
- Update reference for CP2K cpu check small (902)
- Update Tensorflow checks for PE19.06 (896)
- Update IPM tests for PE19.06 (894)
- Update Intel VTune test (891)
- Fix CPU sum output label for the MCH OpenACC CUDA MPI CPPSTD test (889)
- inspector (890)
- Add workaround for netcdf test with PrgEnv-pgi on Dom for CLE7 (870)
- Remove RRTMGP from production (877)
- Update STREAM benchmark test (850)
- Log maximum GPU temperature in the GPU burn test (872)
- Update OpenCL test for PrgEnv-pgi (865)
- Update JacobiNoTool perf numbers (864)
- Update Boost test for PE19.06 (863)
- Remove stale comment in IOR check (857)
- Adapt Haswell FMA check for CCE 9.0.x (851)
- Adapt OpenMP flags for CCE 9.0.x in Jacobi tools tests (849)
- Adapt OpenMP flags for CCE 9.0.x in HelloWorld tests (847)
- Update IOR check (838)
- Use `ldflags` to link cublas in `CudaSimpleMPICheck` (846)
- Enable dynamic linking tests for Trilinos (842)
- Add GridTools mini app benchmark (760)
- Fix DefaultRequestGPUSetsGRES for updated slurm output (837)
- Fix MemoryOverconsumptionCheck message check (836)
- Set `num_cpus_per_task` to the TensorFlow Horovod test (831)
- Make MCH halo cell exchange generic (826)
- Make HPCG basic test generic (830)
- Make DGEMM test generic (827)
- Make GPU shared memory bandwidth test generic (829)
- Make GPU burn test generic (828)
- Update Intel tools version on Dom (825)
- Add API calls in Intel Advisor Roofline check (812)
- Make the Jacobi check generic (819)
- Make `HaloCellExchangeTest` generic (817)
- Add Intel Vtune roofline check (813)
- Add `dom:mc` as valid system to `IntelRooflineAdvisorTest` (814)

Other

- Add setuptools as requirement to reframe (878)
- Update system config for Kesch (861)
- Re-enable PGI on GPU/MC partitions on Dom (862)
- Ignore irrelevant `` comments when triggering the CI (859)
- Run unit tests for Daint in the CI partition (858)
- Define machines to run the CI with Jenkins params (852)
- Reenable PGI on Dom (841)
- Add missing slides to the documentation page (879)

ReFrame 2.18 Release Notes

New features and enhancements

- Print performance variable name in case of performance errors (791)
- Print task allocation type (standard or flexible) for tests in detailed listing (786)
- Do not use absolute paths in generated batch scripts (779)
- Add option to print performance report (733)
- Remove deprecated `--logdir` command-line option and the corresponding attribute in the logging configuration (741)
- Enable environment variable substitution in shell code emitted by build systems (744)

Bug fixes

- Fix crash when using syslog logging handler with the `-v` option (808)
- Add version checks for TMod and TMod4 (794)
- Fix exception message formatting in reservation node extraction (793)
- Fix filename order in build error message (790)
- Do not crash when invalid regular expressions are passed to the test selection options (757)
- Force units in gperftools report (759)
- Fix gdb4hpc failure with C++ (747)
- Fix failure in Intel Advisor Roofline check (745)
- Fix duplicate module commands emitted while executing test cases (740)
- Fix typo in Likwid bandwidth benchmark (735)
- Fix crash in flexible task allocation due to refactoring of `SystemPartition` (729)
- Remove unnecessary module from the "Hello, World" tests (726)
- Fix string error in header file of CUDA memtest (718)
- Fix deprecation warning in check filter functions unit tests (723)

Regression tests

- Revise MAGMA library benchmark (783)
- Add simple FFTW benchmark (776)
- Make OSU tests check generic (801)
- Make the STREAM test generic (799)
- Make alloc_speed check generic (800)
- Make CUDA kernel latency tests generic (796)
- Hardcode version in Intel Vtune test to prevent failure when default changes (785)
- Add Roofline check of the GPP code using NVIDIA nvprof (778)
- Add ert roofline check (gpu) (775)
- Add ert roofline check (773)
- Add sde roofline check (774)
- Fix ddt+cuda/10.0 check (772)
- Add CUDA shared memory bandwidth benchmark (770)
- Adapt TensorFlow+Horovod test to CrayGNU-19.03 (771)
- Update Tensorflow check to r1.12.0 (768)
- Add flexible HPCG GPU benchmark test (762)
- Update Score-P check for PE/19.03 (767)
- Update IPM check for PE/19.03 (766)
- Adapt Paraview GPU check to CLE7 (758)
- Update magma tests (754)
- Update Jacobi test for CLE7 (727)
- Fix ldflags for `nvprof`, `cuda_gdb` tests (751)
- Update boost python test to use the latest module (748)
- Update libsci_acc_symlink tests for PE19.03 (734)
- Update Nvidia35Resolve test for the new PE (732)
- Update perftools check for CLE7 (731)
- Rename `for` loop variable to fix warnings in kernel latency test (750)
- Add test for gdb4hpc (603)
- Enable small scale application tests also on Daint (738)
- Sort cores list in affinity test before comparing (739)
- Add Intel Advisor Roofline test (721)
- Add mpiP parallel test (720)
- Add gperftools parallel test (719)

Other

- Require full tag names in genrelnotes.py script (807)
- Expand Ault configuration (782)
- Add Ault TDS to the config file (777)
- Update deployment script (755)
- Add PyPI badge to README (752)
- Disable running modified tests on Daint (749)
- Prepare ReFrame for deployment to PyPI (743)
- Document the `--performance-report` command line option (761)
- Fix reference to `exclusive_access` in tutorial (742)
- Ongoing work for supporting containers (787)
- Ongoing work for supporting test dependencies (769, 797)