Mle-hyperopt

"end": 0.5,
"prior": "log-uniform"}},
integer={"batch_size": {"begin": 32,
"end": 128,
"prior": "uniform"}},
categorical={"arch": ["mlp", "cnn"]})

Simple ask - eval - tell API
configs = strategy.ask(5)
values = [train_network(**c) for c in configs]
strategy.tell(configs, values)


- Different search strategies: Grid, random, SMBO, base nevergrad

Fixed
- Fix `MutableMapping` to also work with python 3.10.
- Fix `pickle5` install requirement to only apply for python < 3.8

Fixed
- Minor fixes for broken `scikit-optimize` dependency.

Added
- Log reloading helper for post-processing.

Fixed
- Bug fix in `mle-search` with imports of dependencies. Needed to append path.
- Bug fix with cleaning nested dictionaries. Have to make sure not to delete entire sub-dictionary.

Added

- Adds a command line interface for running a sequential search given a python script `<script>.py` containing a function `main(config)`, a default configuration file `<base>.yaml` & a search configuration `<search>.yaml`. The `main` function should return a single scalar performance score. You can then start the search via:


mle-search <script>.py --base_config <base>.yaml --search_config <search>.yaml --num_iters <search_iters>

Or short via:

mle-search <script>.py -base <base>.yaml -search <search>.yaml -iters <search_iters>

- Adds doc-strings to all functionalities.
Changed

- Make it possible to optimize parameters in nested dictionaries. Added helpers `flatten_config` and `unflatten_config`. For shaping `'sub1/sub2/vname' <-> {sub1: {sub2: {vname: v}}}`
- Make start-up message also print fixed parameter settings.
- Cleaned up decorator with the help of `Strategies` wrapper.

Added

- Adds possibility to store and reload entire strategies as pkl file (as asked for in issue 2).
- Adds `improvement` method indicating if score is better than best stored one
- Adds save option for best plot
- Adds `args, kwargs` into decorator
- Adds synchronous Successive Halving (`SuccessiveHalvingSearch` - issue 3)
- Adds synchronous HyperBand (`HyperbandSearch` - issue 3)
- Adds synchronous PBT (`PBTSearch` - issue 4)
- Adds option to save log in `tell` method
- Adds small torch mlp example for SH/Hyperband/PBT w. logging/scheduler
- Adds print welcome/update message for strategy specific info

Changed
- Major internal restructuring:
- `clean_data`: Get rid of extra data provided in configuration file
- `tell_search`: Update model of search strategy (e.g. SMBO/Nevergrad)
- `log_search`: Add search specific log data to evaluation log
- `update_search`: Refine search space/change active strategy etc.
- Also allow to store checkpoint of trained models in `tell` method.
- Fix logging message when log is stored
- Make json serializer more robust for numpy data types
- Robust type checking with `isinstance(self.log[0]["objective"], (float, int, np.integer, np.float))`
- Update NB to include `mle-scheduler` example
- Make PBT explore robust for integer/categorical valued hyperparams
- Calculate total batches & their sizes for hyperband