Optuna

What's Changed

Add button to upgrade RDB schema by c-bata in https://github.com/optuna/optuna-dashboard/pull/207

When the RDB schema is incompatible between the runtime Optuna version and the given RDB storage, optuna-dashboard redirects to an error page that shows error message. There is a button to run alembic migration, so that users can upgrade the schema without executing `optuna storage upgrade` on a Terminal.

https://user-images.githubusercontent.com/5564044/164342862-8f7ab2a3-3b97-426d-ab1f-aaf6c51b85c5.mp4

**Full Changelog**: https://github.com/optuna/optuna-dashboard/compare/v0.6.2...v0.6.3

What's Changed

In this release, optuna-dashboard supports Optuna v3.0.0b0.

* Set include_best_trial=True to get study summaries for v3 release by c-bata in https://github.com/optuna/optuna-dashboard/pull/196
* Co-authored by Ellon-M in https://github.com/optuna/optuna-dashboard/pull/200
* Bump minimist from 1.2.5 to 1.2.6 by dependabot in https://github.com/optuna/optuna-dashboard/pull/203

Misc

* Publish to PyPI when GitHub release is created by c-bata in https://github.com/optuna/optuna-dashboard/pull/194
* Rename DEVELOPMENT.md to CONTRIBUTING.md by c-bata in https://github.com/optuna/optuna-dashboard/pull/197
* Run tests with Python 3.10 on CI by c-bata in https://github.com/optuna/optuna-dashboard/pull/195
* Bump the version up to v0.6.2 by c-bata in https://github.com/optuna/optuna-dashboard/pull/204

New Contributors
* Ellon-M made their first contribution in https://github.com/optuna/optuna-dashboard/pull/200

**Full Changelog**: https://github.com/optuna/optuna-dashboard/compare/v0.6.1...v0.6.2

Highlights

Dark mode support

| Light mode | Dark mode |
| --- | --- |
| <img width="966" alt="Light mode" src="https://user-images.githubusercontent.com/5564044/159600417-7c077a15-1723-49a6-aca9-9a4970d01825.png"> | <img width="963" alt="Dark mode" src="https://user-images.githubusercontent.com/5564044/159600408-fc02ba9e-effc-4d88-acd6-dfc90a226ba8.png"> |

Gunicorn support

optuna-dashboard uses [wsgiref](https://docs.python.org/3/library/wsgiref.html) module which is NOT suitable for the production use since it has not been reviewed for security issues. From v0.6.1 release, you can use "--server gunicorn" option to run more secure and faster. If you want to know more about Gunicorn, please check [the documentation](https://gunicorn.org/).


$ pip install gunicorn
$ optuna-dashboard sqlite:///db.sqlite3 --server gunicorn


A component to take a memo

You can freely write anything in a Note area. The text is stored in study's system_attrs with version control, so it is prohibited to accidentally overwrite existing text.

https://user-images.githubusercontent.com/5564044/159110279-1a784286-583c-436e-a205-b550cd12613b.mp4

An official Docker image

You can use [an official Docker image](https://github.com/optuna/optuna-dashboard/pkgs/container/optuna-dashboard) instead of setting up your Python environment. The Docker image only supports SQLite3, MySQL(PyMySQL), and PostgreSQL(Psycopg2).


SQLite3
$ docker run -it --rm -p 8080:8080 -v `PWD`:/app -w /app ghcr.io/optuna/optuna-dashboard sqlite:///db.sqlite3



MySQL (PyMySQL)
$ docker run -it --rm -p 8080:8080 ghcr.io/optuna/optuna-dashboard mysql+pymysql://username:passwordhostname:3306/dbname



PostgreSQL (Psycopg2)
$ docker run -it --rm -p 8080:8080 ghcr.io/optuna/optuna-dashboard postgresql+psycopg2://username:passwordhostname:5432/dbname


Stable Python interface

optuna-dashboard now provides stable Python interfaces.

**`run_server(storage: Union[str, BaseStorage], host: str = 'localhost', port: int = 8080) -> None`**

Start running optuna-dashboard and blocks until the server terminates.
This function uses wsgiref module which is not intended for the production use.

**`wsgi(storage: Union[str, BaseStorage]) -> WSGIApplication`**

This function exposes WSGI interface for people who want to run on the
production-class WSGI servers like Gunicorn or uWSGI.

Changes

New features

* Provide stable Python API by c-bata in https://github.com/optuna/optuna-dashboard/pull/167
* Add --server option for Gunicorn support by c-bata in https://github.com/optuna/optuna-dashboard/pull/168
* Dark Mode Support by c-bata in https://github.com/optuna/optuna-dashboard/pull/172
* Find study by keywords by c-bata in https://github.com/optuna/optuna-dashboard/pull/178
* Hide intermediate values chart if no intermediate values by c-bata in https://github.com/optuna/optuna-dashboard/pull/179
* Note component to set study's description by c-bata in https://github.com/optuna/optuna-dashboard/pull/180


Bug fixes

* Fix bug when given inf objective value by c-bata in https://github.com/optuna/optuna-dashboard/pull/176

Other improvements

* Publish an official Docker image by c-bata in https://github.com/optuna/optuna-dashboard/pull/146
* Avoid using Python 3.10 to run integration tests. by c-bata in https://github.com/optuna/optuna-dashboard/pull/148
* FIX 142 Responsive width large screens by ezioishan in https://github.com/optuna/optuna-dashboard/pull/149
* Bump tmpl from 1.0.4 to 1.0.5 by dependabot in https://github.com/optuna/optuna-dashboard/pull/152
* Bump ansi-regex from 5.0.0 to 5.0.1 by dependabot in https://github.com/optuna/optuna-dashboard/pull/151
* Bump axios from 0.21.1 to 0.21.2 by dependabot in https://github.com/optuna/optuna-dashboard/pull/150
* Fix Dockerfile by c-bata in https://github.com/optuna/optuna-dashboard/pull/158
* Publish a Docker image to GitHub Container Registry. by c-bata in https://github.com/optuna/optuna-dashboard/pull/157
* Update the usage of docker image by c-bata in https://github.com/optuna/optuna-dashboard/pull/159
* Upgrade node version to v16 by c-bata in https://github.com/optuna/optuna-dashboard/pull/160
* Add FloatDistribution and IntDitribution for Optuna v3 release. by c-bata in https://github.com/optuna/optuna-dashboard/pull/162
* Bump follow-redirects from 1.14.6 to 1.14.8 by dependabot in https://github.com/optuna/optuna-dashboard/pull/165
* Make all functions private except for the stable Python interface by c-bata in https://github.com/optuna/optuna-dashboard/pull/169
* Fix type hints and improve README by c-bata in https://github.com/optuna/optuna-dashboard/pull/170
* Migrate to MUI v5 by c-bata in https://github.com/optuna/optuna-dashboard/pull/171
* Update plotly.js and recoil.js by c-bata in https://github.com/optuna/optuna-dashboard/pull/173
* Add title attr to buttons by c-bata in https://github.com/optuna/optuna-dashboard/pull/174
* Improve build performance by c-bata in https://github.com/optuna/optuna-dashboard/pull/175
* Add some UI improvements. by c-bata in https://github.com/optuna/optuna-dashboard/pull/177
* Show animated SVG while loading bundle.js by c-bata in https://github.com/optuna/optuna-dashboard/pull/181
* Implement in-memory cache for hyperparameter importance by c-bata in https://github.com/optuna/optuna-dashboard/pull/182
* Set favicon.ico by c-bata in https://github.com/optuna/optuna-dashboard/pull/183
* Refactor project structure by c-bata in https://github.com/optuna/optuna-dashboard/pull/184
* Use webpack CompressionPlugin (4MB -> 1.2MB) by c-bata in https://github.com/optuna/optuna-dashboard/pull/185
* Disable gzip when DEBUG mode is enabled by c-bata in https://github.com/optuna/optuna-dashboard/pull/186
* Reduce Cumulative Layout Shift for better user experience. by c-bata in https://github.com/optuna/optuna-dashboard/pull/187
* Debounce input to create new study by c-bata in https://github.com/optuna/optuna-dashboard/pull/188
* Refactor graph layouts by c-bata in https://github.com/optuna/optuna-dashboard/pull/190
* Use plotly.js-dist-min to reduce bundle.js. by c-bata in https://github.com/optuna/optuna-dashboard/pull/191
* Add some minor improvements by c-bata in https://github.com/optuna/optuna-dashboard/pull/192
* Bump the version up to v0.6.0 by c-bata in https://github.com/optuna/optuna-dashboard/pull/189

New Contributors
* ezioishan made their first contribution in https://github.com/optuna/optuna-dashboard/pull/149

**Full Changelog**: https://github.com/optuna/optuna-dashboard/compare/v0.5.0...v0.6.1

This is the release note of v0.6.0. See [here](https://github.com/pfnet/optuna/milestone/2?closed=1) for the complete list of solved issues and merged PRs.

New Features
--------------
- Support cross validation in lightgbm.py (260, thanks shikiponn!)
- Implement Keras integration. (256, thanks higumachan!)
- Add a pruner based on Successive Halving. (236)

Examples
----------
- Add an example and FAQ about object functions that take additional arguments. (277)

Documents
------------
- Add instruction sentences for RDB URL setup in the tutorial. (295)
- Add an FAQ page about unit tests of objective functions. (291)

Enhancements
---------------
- Address integrity errors caused by timing issues when setting attributes in RDBStorage. (290)
- Optimize query of count trials. (282)
- Apply autopep8. (274)
- Fix dashboard access problem from remote host. (254, thanks higumachan!)

seed = 1

rng = np.random.RandomState(seed=seed)
weights = rng.randint(10, 101, size=(n_knapsacks, n_items))
profits = rng.randint(10, 101, size=(n_knapsacks, n_items))
constraints = (np.sum(weights, axis=1) * feasible_rate).astype(np.int)

def objective(trial):
xs = np.array([trial.suggest_categorical(f"x_{i}", (0, 1)) for i in range(weights.shape[1])])
total_weights = np.sum(weights * xs, axis=1)
total_profits = np.sum(profits * xs, axis=1)

Constraints which are considered feasible if less than or equal to zero.
constraints_violation = total_weights - constraints
trial.set_user_attr("constraint", constraints_violation.tolist())

return total_profits.tolist()

def constraints_func(trial):
return trial.user_attrs["constraint"]

sampler = optuna.samplers.NSGAIISampler(population_size=10, constraints_func=constraints_func)

study = optuna.create_study(directions=["maximize"] * n_knapsacks, sampler=sampler)
study.optimize(objective, n_trials=200)



New Features

- Ask-and-Tell API (`Study.ask`, `Study.tell`) (2158)
- Add `constraints_func` argument to NSGA-II (2175)
- Add heartbeat functionality using `threading` (2190)
- Add `Study.add_trials` to simplify creating customized study (2261)

Enhancements

- Support log scale in parallel coordinate (2164, thanks tohmae!)
- Warn if constraints are missing in constrained NSGA-II (2205)
- Immediately persist suggested parameters with `_CachedStorage` (2214)
- Include the upper bound of uniform/loguniform distributions (2223)

Bug Fixes

- Call base sampler's `after_trial` in `PartialFixedSampler` (2209)
- Fix `trials_dataframe` for multi-objective optimization with fail or pruned trials (2265)
- Fix `calculate_weights_below` method of `MOTPESampler` (2274, thanks y0z!)

Installation

- Remove version constraint for AllenNLP and run `allennlp_*.py` on GitHub Actions (2226)
- Pin `mypy==0.790` (2259)
- Temporarily avoid AllenNLP v2 (2276)

Documentation

- Add `callback` & `(add|enqueue)_trial` recipe (2125)
- Make `create_trial`'s documentation and tests richer (2126)
- Move import lines of the callback recipe (A follow-up of 2125) (2221)
- Fix `optuna/samplers/_base.py` typo (2239)
- Introduce optuna-dashboard on README (2224)

Examples

- Refactor `examples/multi_objective/pytorch_simple.py` (2230)
- Move BoTorch example to `examples/multi_objective` directory (2244)
- Refactor `examples/multi_objective/botorch_simple.py` (2245, thanks nzw0301!)
- Fix typo in `examples/mlflow` (2258, thanks nzw0301!)

Tests

- Make `create_trial`'s documentation and tests richer (2126)
- Fix unit test of median pruner (2171)
- `SkoptSampler` acquisition function in test to more likely converge (2194)
- Diet `tests/test_study.py` (2218)
- Diet `tests/test_trial.py` (2219)
- Shorten the names of trial tests (2228)
- Move `STORAGE_MODES` to `testing/storage.py` (2231)
- Remove duplicate test on `study_tests/test_optimize.py` (2232)
- Add init files in test directories (2257)

Code Fixes

- Code quality improvements (2009, thanks srijan-deepsource!)
- Refactor CMA-ES sampler with search space transform (2193)
- `BoTorchSampler` minor code fix reducing dictionary lookup and clearer type behavior (2195)
- Fix bad warning message in `BoTorchSampler` (2197)
- Use `study.get_trials` instead of `study._storage.get_all_trials` (2208)
- Ensure uniform and loguniform distributions less than high boundary (2243)

Continuous Integration

- Add `RDBStorage` tests in github actions (2200)
- Publish distributions to TestPyPI on each day (2220)
- Rename GitHub Actions jobs for (Test)PyPI uploads (2254)

Other

- Fix the syntax of `pypi-publish.yml` (2187)
- Fix `mypy` local fails in `tests/test_deprecated.py` and `tests/test_experimental.py` (2191)
- Add an explanation of "no period in the PR title" to `CONTRIBUTING.md` (2192)
- Bump up version number to 2.5.0.dev (2238)
- Fix mypy version to 0.790 (Follow-up of 2259) (2260)
- Bump up version number to v2.5.0 (2282)

Thanks to All the Contributors!

This release was made possible by authors, and everyone who participated in reviews and discussions.

Crissman, HideakiImamura, c-bata, crcrpar, g-votte, himkt, hvy, keisuke-umezawa, not522, nzw0301, sile, srijan-deepsource, tohmae, toshihikoyanase, y0z, ytsmiling

CHANGES

New features

* Add multi-objective support for ParameterImportance, Edf and ParallelCoordinate plot (102)
* Add visualization preference panel (119)
* Add x log scale for slice plot (120, thanks keisuke-umezawa)

Bug fixes

* Improve label overlaps (131, thanks kenrota)

Enhancements

* Update npm dependencies (103)
* Set margin top on each plots (105)
* Add unit test for TrialTable (110)
* Use YAML based issue template (114)
* Add incremental sync for the study summary API (115)
* Rename goptuna to optuna (118, thanks keisuke-umezawa)
* Run frontend tests on GitHub Actions (123)
* Rename tests to python_tests (124)
* Update "Release the new version" section on DEVELOPMENT.md (127)
* Rename log scale to log y scale (130, thanks keisuke-umezawa)
* Fix typo in graph title of Parallel coordinate (132, thanks kenrota)
* Fix typo in WSGI environment variable (133, thanks kenrota)
* Add tests for filter trial state (134)
* Add tests for DataGrid (135)