Libensemble

-------------

:Date: August 19, 2019

* Code has been restructured to meet xSDK package policies for interoperable ECP software (version 0.5.0). 208
* The use of MPI.COMM_WORLD has been removed. Uses a duplicate of COMM_WORLD if no communicator passed (any process not in communicator returns with an exit code of 3). 108
* All output from libEnsemble goes via logger. MANAGER_WARNING level added. This level and above are echoed to stderr by default. API option to change echo level.
* Simulation directories are created only during sim_f calls are suffixed by _worker. 146
* New user function libE.check_inputs() can be used to check valid configuration of inputs. Can be called in serial or under MPI (see libE API). 65
* Installation option has been added to install dependencies used in tests ``pip install libensemble[extras]``.
* A profiling option has been added to sim_specs. 170
* Results comparison scripts have been included for convenience.

:Note:

* Tested platforms include Linux, MacOS (**New**), Theta (Cray XC40/Cobalt), Summit (IBM Power9/LSF), and Bebop (Cray CS400/Slurm).
* Tested Python versions: (Cpython) 3.5, 3.6, 3.7
* **Note** Support has been removed for Python 3.4 since it is officially retired. Also NumPy has removed support.

:Known issues:

* These are unchanged from v0.5.0.

-------------

:Date: July 11, 2019

* Fixed LSF resource detection for large jobs on LSF systems (e.g., Summit). 184
* Added support for macOS. 182
* Improved the documentation (including addition of beginner's tutorial and FAQ).

:Note:

* Tested platforms include Local Linux, Theta (Cray XC40/Cobalt), Summit (IBM Power9/LSF), and Bebop (Cray CS400/Slurm).
* Tested Python versions: (Cpython) 3.4, 3.5, 3.6, 3.7.

:Known issues:

* These are unchanged from v0.5.0.

-------------

:Date: May 22, 2019

* Added local (multiprocessing) and TCP options for manager/worker communications, in addition to mpi4py. (42).

* Example: libEnsemble can be run on MOM/launch nodes (e.g., those of ALCF/Theta & OLCF/Summit) and can remotely detect compute resources.
* Example: libEnsemble can be run on a system without MPI.
* Example: libEnsemble can be run with a local manager and remote TCP workers.

* Added support for Summit/LSF scheduler in job controller.
* MPI job controller detects and retries launches on failure; adding resilience. (143)
* Job controller supports option to extract/print job times in libE_stats.txt. (136)
* Default logging level changed to INFO. (164)
* Logging interface added, which allows user to change logging level and file. (110)
* All worker logging and calculation stats are routed through manager.
* libEnsemble can be run without a gen_func, for example, when using a previously computed random sample. (122)
* Aborts dump persis_info with the history.

:Note:

* **This version no longer supports Python 2.**
* Tested platforms include Local Linux, Theta (Cray XC40/Cobalt), Summit (IBM Power9/LSF), and Bebop (Cray CS400/Slurm).

:Known issues:

* Open-MPI does not work with direct MPI job launches in mpi4py comms mode, since it does not support nested MPI launches
(Either use local mode or Balsam job controller).
* Local comms mode (multiprocessing) may fail if MPI is initialized before forking processors. This is thought to be responsible for issues combining with PETSc.
* Remote detection of logical cores via LSB_HOSTS (e.g., Summit) returns number of physical cores since SMT info not available.
* TCP mode does not support (1) more than one libEnsemble call in a given script or (2) the auto-resources option to the job controller.

-------------

:Date: February 20, 2019

* Logging no longer uses root logger (also added option to change libEnsemble log level). (105)
* Added wait_on_run option for job controller launch to block until jobs have started. (111)
* persis_info can be passed to sim as well as gen functions. (112)
* Postprocessing scripts added to create performance/utilization graphs. (102)
* New scaling test added (not part of current CI test suite). (114)

-------------

:Date: November 7, 2018

* Separated job controller classes into different modules including a base class (API change).
* Added central_mode run option to distributed type (MPI) job_controllers (API addition). (93)
* Made poll and kill job methods (API change).
* In job_controller, set_kill_mode is removed and replaced by a wait argument for a hard kill (API change).
* Removed register module - incorporated into job_controller (API change).
* APOSMM has improved asynchronicity when batch mode is false (with new example). (96)
* Manager errors (instead of hangs) when alloc_f or gen_f don't return work when all workers are idle. (95)

:Known issues:

* Open-MPI is not supported with direct MPI launches since nested MPI launches are not supported.

-------------

:Date: September 7, 2018

* Issues with killing jobs have been fixed. (21)
* Fixed job_controller manager_poll to work with multiple jobs. (62)
* API change: persis_info now included as an argument to libE and is returned from libE instead of gen_info
* Gen funcs: aposmm_logic module renamed to aposmm.
* New example gen and allocation functions.
* Updated Balsam launch script (with new Balsam workflow).
* History is dumped to file on manager or worker exception and MPI aborted (with exit code 1). (46)
* Default logging level changed to DEBUG and redirected to file ensemble.log.
* Added directory of standalone tests (comms, job kills, and nested MPI launches).
* Improved and sped up unit tests. (68)
* Considerable documentation enhancements.

:Known issues:

* Open-MPI is not supported with direct MPI launches since nested MPI launches are not supported.