Optuna

This is the release note of [v4.1.0](https://github.com/optuna/optuna/milestone/64?closed=1). Highlights of this release include:
- 🤖 AutoSampler: Automatic Selection of Optimization Algorithms
- 🚀 More scalable RDB Storage Backend
- 🧑‍💻 Five New Algorithms in OptunaHub (MO-CMA-ES, MOEA/D, etc.)
- 🐍 Support Python 3.13

The updated list of tested and supported Python releases is as follows:
- [Optuna 4.1](https://github.com/optuna/optuna/releases/tag/v4.1.0): supported by Python 3.8 - 3.13
- [Optuna Integration 4.1](https://github.com/optuna/optuna-integration/releases/tag/v4.1.0): supported by Python 3.8 - 3.12
- [Optuna Dashboard 0.17.0](https://github.com/optuna/optuna-dashboard/releases/tag/v0.17.0): supported by Python 3.8 - 3.13

Highlights

AutoSampler: Automatic Selection of Optimization Algorithms

<img width="750" alt="Blog-1" src="https://github.com/user-attachments/assets/f3ecd366-7e7b-49d6-ae38-f35301fa7d11">

[AutoSampler](https://hub.optuna.org/samplers/auto_sampler/) automatically selects a sampler from those implemented in Optuna, depending on the situation. Using AutoSampler, as in the code example below, users can achieve optimization performance equal to or better than Optuna's default without being aware of which optimization algorithm to use.


$ pip install optunahub cmaes torch scipy


python
import optuna
import optunahub

auto_sampler_module = optunahub.load_module("samplers/auto_sampler")
study = optuna.create_study(sampler=auto_sampler_module.AutoSampler())


See the [Medium blog post](https://medium.com/optuna/autosampler-automatic-selection-of-optimization-algorithms-in-optuna-1443875fd8f9) for details.


Enhanced RDB Storage Backend

This release incorporates comprehensive performance tuning on Optuna’s [RDBStorage](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.storages.RDBStorage.html), leading to significant performance improvements. The table below shows the comparison results of execution times between versions 4.0 and 4.1.

| trials | v4.0.0 | v4.1.0 | Diff |
| --- | --- | --- | --- |

Here is the release note of [v4.0.0](https://github.com/optuna/optuna/milestone/63?closed=1). Please also check out the [release blog post](https://medium.com/optuna/announcing-optuna-4-0-3325a8420d10).

If you want to update the Optuna version of your existing projects to v4.0, please see the [migration guide](https://github.com/optuna/optuna/discussions/5573).

We have also published blog posts about the development items. Please check them out!
- [OptunaHub, a Feature-Sharing Platform for Optuna, Now Available in Official Release!](https://medium.com/optuna/optunahub-a-feature-sharing-platform-for-optuna-now-available-in-official-release-4b99efe9934d)
- [File Management during LLM (Large Language Model) Trainings by Optuna v4.0.0 Artifact Store](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7)
- [Significant Speed Up of Multi-Objective TPESampler in Optuna v4.0.0](https://medium.com/optuna/significant-speed-up-of-multi-objective-tpesampler-in-optuna-v4-0-0-2bacdcd1d99b)

Highlights

Official Release of Feature-Sharing Platform OptunaHub
We officially released [OptunaHub](https://hub.optuna.org/), a feature-sharing platform for Optuna. A large number of optimization and visualization algorithms are available in OptunaHub. Contributors can easily register their methods and deliver them to Optuna users around the world.

Please also read the [OptunaHub release blog post](https://medium.com/optuna/optunahub-a-feature-sharing-platform-for-optuna-now-available-in-official-release-4b99efe9934d).



Enhanced Experiment Management Feature: Official Support of Artifact Store
Artifact Store is a file management feature for files generated during optimization, dubbed artifacts. In Optuna v4.0, we stabilized the existing file upload API and further enhanced the usability of Artifact Store by adding some APIs such as the artifact download API. We also added features to show JSONL and CSV files on [Optuna Dashboard](https://github.com/optuna/optuna-dashboard) in addition to the existing support for images, audio, and video. With this official support, the API backward compatibility will be guaranteed.

For more details, please check the [blog post](https://medium.com/optuna/file-management-during-llm-large-language-model-trainings-by-optuna-v4-0-0-artifact-store-5bdd5112f3c7).





`JournalStorage`: Official Support of Distributed Optimization via Network File System
`JournalStorage` is a new Optuna storage experimentally introduced in Optuna v3.1 (see the [blog post](https://medium.com/optuna/distributed-optimization-via-nfs-using-optunas-new-operation-based-logging-storage-9815f9c3f932) for details). Optuna has `JournalFileBackend`, a storage backend for various file systems. It can be used on NFS, allowing Optuna to scale to multiple nodes.


In Optuna v4.0, the API for `JournalStorage` has been reorganized, and `JournalStorage` is officially supported. This official support guarantees its backward compatibility from v4.0. For details on the API changes, please refer to the [Optuna v4.0 Migration Guide](https://github.com/optuna/optuna/discussions/5573).

python
import optuna
from optuna.storages import JournalStorage
from optuna.storages.journal import JournalFileBackend

def objective(trial: optuna.Trial) -> float:
...

storage = JournalStorage(JournalFileBackend("./optuna_journal_storage.log"))
study = optuna.create_study(storage=storage)
study.optimize(objective)



Significant Speedup of Multi-Objective `TPESampler`
Before v4.0, the multi-objective `TPESampler` sometimes limits the number of trials during optimization due to the sampler bottleneck after a few hundred trials. Optuna v4.0 drastically improves the sampling speed, e.g., 300 times faster for three objectives with 200 trials, and enables users to handle much more trials. Please check the [blog post](https://medium.com/optuna/significant-speed-up-of-multi-objective-tpesampler-in-optuna-v4-0-0-2bacdcd1d99b) for details.

Introduction of a New `Terminator` Algorithm
Optuna `Terminator` was originally introduced for hyperparameter optimization of machine learning algorithms using cross-validation. To accept broader use cases, Optuna v4.0 introduced the Expected Minimum Model Regret (EMMR) algorithm. Please refer to the [`EMMREvaluator` document](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.terminator.EMMREvaluator.html) for details.

Enhancements of Constrained Optimization
We have gradually expanded the support for constrained optimization. In v4.0, [`study.best_trial`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trial) and [`study.best_trials`](https://optuna.readthedocs.io/en/v4.0.0/reference/generated/optuna.study.Study.html#optuna.study.Study.best_trials) start to support constraint optimization. They are guaranteed to satisfy the constraints, which was not the case previously.



Breaking Changes

Optuna removes deprecated features in major releases. To prevent users' code from suddenly breaking, we take a long interval between when a feature is deprecated and when it is removed. By default, features are removed when the major version has increased by two since the feature was deprecated. For this reason, the main target features for removal in v4.0 were deprecated at v2.x. Please refer to the [migration guide](https://github.com/optuna/optuna/discussions/5573) for the removed features list.

- Delete deprecated three integrations, `skopt`, `catalyst`, and `fastaiv1` (https://github.com/optuna/optuna-integration/pull/114)
- Remove deprecated `CmaEsSampler` from integration (https://github.com/optuna/optuna-integration/pull/116)
- Remove verbosity of `LightGBMTuner` (https://github.com/optuna/optuna-integration/pull/136)
- Move positional args of `LightGBMTuner` (https://github.com/optuna/optuna-integration/pull/138)
- Remove `multi_objective` (5390)
- Delete deprecated `_ask` and `_tell` (5398)
- Delete deprecated `--direction(s)` arguments in the `ask` command (5405)
- Delete deprecated three integrations, `skopt`, `catalyst`, and `fastaiv1` (5407)
- Remove the default normalization of importance in f-ANOVA (5411)
- Remove `samplers.intersection` (5414)
- Drop implicit create-study in `ask` command (5415)
- Remove deprecated `study optimize` CLI command (5416)
- Remove deprecated `CmaEsSampler` from integration (5417)
- Support constrained optimization in `best_trial` (5426)
- Drop `--study` in `cli.py` (5430)
- Deprecate `constraints_func` in `plot_pareto_front` function (5455)
- Rename some class names related to `JournalStorage` (5539)
- Remove `optuna.samplers.MOTPESampler` (5640)

New Features

- Add Comet ML integration (https://github.com/optuna/optuna-integration/pull/63, thanks caleb-kaiser!)
- Add Knowledge Gradient candidates functions (https://github.com/optuna/optuna-integration/pull/125, thanks alxhslm!)
- Add `is_exhausted()` function in the `GridSampler` class (5306, thanks aaravm!)
- Remove experimental from plot (5413)
- Implement `download_artifact` (5448)
- Add a function to list linked artifact information (5467)
- Stabilize artifact APIs (5567)
- Stabilize `JournalStorage` (5568)
- Add `EMMREvaluator` and `MedianErrorEvaluator` (5602)

Enhancements

- Pass two arguments to the forward of `ConstrainedMCObjective` to support `botorch=0.10.0` (https://github.com/optuna/optuna-integration/pull/106)
- Speed up non-dominated sort (5302)
- Make 2d hypervolume computation twice faster (5303)
- Reduce the time complexity of HSSP 2d from `O(NK^2 log K)` to `O((N - K)K)` (5346)
- Introduce lazy hypervolume calculations in HSSP for speedup (5355)
- Make `plot_contour` faster (5369)
- Speed up `to_internal_repr` in `CategoricalDistribution` (5400)
- Allow users to modify categorical distance more easily (5404)
- Speed up `WFG` by NumPy vectorization (5424)
- Check whether the study is multi-objective in `sample_independent` of `GPSampler` (5428)
- Suppress warnings from `numpy` in hypervolume computation (5432)
- Make an option to assume Pareto optimality in WFG (5433)
- Adapt multi objective to NumPy v2.0.0 (5493)
- Enhance the error message for integration installation (5498)
- Reduce journal size of `JournalStorage` (5526)
- Simplify a SQL query for getting the `trial_id` of `best_trial` (5537)
- Add import check for artifact store objects (5565)
- Refactor `_is_categorical()` in `optuna/optuna/visualization` (5587, thanks kAIto47802!)
- Speed up WFG by using a fact that hypervolume calculation does not need (second or later) duplicated Pareto solutions (5591)
- Enhance the error message of `multi_objective` deletion (5641)

Bug Fixes

- Log `None` objective trials correctly (https://github.com/optuna/optuna-integration/pull/119, thanks neel04!)
- Pass two arguments to the forward of `ConstrainedMCObjective` in `qnei_candidates_func` (https://github.com/optuna/optuna-integration/pull/124, thanks alxhslm!)
- Allow single split cv in `OptunaSearchCV` (https://github.com/optuna/optuna-integration/pull/128, thanks sgerloff!)
- Update BoTorch samplers to support new constraints interface (https://github.com/optuna/optuna-integration/pull/132, thanks alxhslm!)
- Fix `WilcoxonPruner` bug when `best_trial` has no intermediate value (5354)
- Debug an error caused by convergence in `GPSampler` (5359)
- Fix `average_is_best` implementation in `WilcoxonPruner` (5366)
- Create an unique renaming filename for each release operation in lock systems of `JournalStorage` (5389)
- Fix `_normalize_value` for incomplete trials (5422)
- Fix heartbeat for race condition (5431)
- Guarantee `weights_below` to be finite in MOTPE (5435)
- Fix `_log_complete_trial` for constrained optimization (5462)
- Convert `step` to `int` in `report` (5488)
- Use a fixed seed value when `seed=None` in `GridSampler` (5490)
- Acquire session lock only for write in heartbeat (5496)
- Fix a bug in `_create_new_trial` and refactor it (5497)
- Add rdb create new trial test (5525)
- Fix a bug in `sample_normalized_param` for `GPSampler` (5543)
- Fix the error caused in `plot_contour()` with an impossible pair of variables (5630, thanks kAIto47802!)
- Fix inappropriate behavior in `plot_rank()` when `None` values exist in trial (5634, thanks kAIto47802!)

Installation

- Add an option to install integration dependencies via pip (https://github.com/optuna/optuna-integration/pull/130)
- Add version constraint to numpy (https://github.com/optuna/optuna-integration/pull/131)

Documentation

- Enhance `README.md` (https://github.com/optuna/optuna-integration/pull/126)
- Add document page and fix docstring and comments (https://github.com/optuna/optuna-integration/pull/134)
- Update the docstring of `PyCmaSampler` (https://github.com/optuna/optuna-integration/pull/145)
- Add Dashboard and OptunaHub links to the document header (https://github.com/optuna/optuna-integration/pull/155)
- Remove duplicated license definition (https://github.com/optuna/optuna-integration/pull/156)
- Use sphinx rst syntax (5345)
- Fix typo (5351)
- Update docs about `show_progress_bar` (5393)
- Fix artifact tutorial (5451)
- Revise docs to specify `ArtifactStore` methods as non-public (5474)
- Improve the docstring of `JournalFileStorage` (5475)
- Improve `make clean` in `docs` (5487)
- Improve the example of chemical structures in `optuna artifact tutorial` (5491)
- Add FAQ for artifact store remove API (5501)
- Add a documentation for `ArtifactMeta` (5511)
- Change docs version from latest to stable (5518)
- Add a list of supported storage for artifact store (5532)
- Rename filenames for `JournalFileStorage` in documents and a tutorial (5535)
- Add explanation for `lock_obj` to `JournalFileStorage` docstring (5540)
- Fix storages document sections (5553)
- Add `Returns` to the docstring of `load_study` (5554, thanks kAIto47802!)
- Fix paths related to `storages.journal` (5560)
- Rename filename for `JournalFileBackend` in examples (5562)
- Add thumbnail (5575)
- Add link to `optunahub-registry` (5586)
- Fix minor artifacts document issues (5592)
- Add the OptunaHub link to the document header (5595)
- Improve a docstring of `CmaEsSampler` (5603)
- Fix the expired links caused by the directory name change in `optuna-examples` (5623, thanks kAIto47802!)
- Remove the use of `evaluator.evaluate` function in the example of `PedAnovaImportanceEvaluator` (5632)
- Add the news section to `README.md` (5636)
- Add SNS URLs to `README.md` (5637)
- Add TPE blog post link to doc (5638)
- Add artifact store post link (5639)
- Add installation guide for visualization tutorial (5644, thanks kAIto47802!)
- Remove a broken link (5648)
- Update the `News` section of `README.md` (5649)

Examples

- Support tensorflow 2.16.1 and separate CI run for tensorflow estimator (https://github.com/optuna/optuna-examples/pull/248)
- Replace deprecated jax function (https://github.com/optuna/optuna-examples/pull/250)
- Drop Python 3.7 support for wandb (https://github.com/optuna/optuna-examples/pull/252)
- Support python 3.12 in the CIs (https://github.com/optuna/optuna-examples/pull/253)
- Remove `dask` version constraint (https://github.com/optuna/optuna-examples/pull/254)
- Update GitHub actions versions to `actions/checkoutv4` and `actions/setup-pythonv5` (https://github.com/optuna/optuna-examples/pull/255)
- Delete an example using deprecated `fastaiv1` (https://github.com/optuna/optuna-examples/pull/256)
- Rename `fastaiv2` to `fastai` (https://github.com/optuna/optuna-examples/pull/257)
- Fix CI (https://github.com/optuna/optuna-examples/pull/258)
- Fix path in pruners workflow (https://github.com/optuna/optuna-examples/pull/259)
- Install `tensorflow-cpu` in CIs (https://github.com/optuna/optuna-examples/pull/260)
- Add python 3.12 and run terminator search cv (https://github.com/optuna/optuna-examples/pull/264)
- Enhance `README.md` (https://github.com/optuna/optuna-examples/pull/265)
- Organize the directory structure (https://github.com/optuna/optuna-examples/pull/266)
- Fix CI with version constraints of packages (https://github.com/optuna/optuna-examples/pull/267)
- Install `numpy<2.0.0` for `catboost` example (https://github.com/optuna/optuna-examples/pull/268)
- Add Python 3.12 to `aim` test python versions (https://github.com/optuna/optuna-examples/pull/270)
- Add Python 3.12 to `fastai` test python versions (https://github.com/optuna/optuna-examples/pull/271)
- Use auc score to unify the intermediate and objective values (https://github.com/optuna/optuna-examples/pull/272)
- Add python 3.12 for `mlflow` CI python versions (https://github.com/optuna/optuna-examples/pull/274)
- Add python 3.12 to CI python versions for `keras` and `tensorboard` examples (https://github.com/optuna/optuna-examples/pull/275)
- Add an example of artifact store (https://github.com/optuna/optuna-examples/pull/276)
- Separate basic and faq directories (https://github.com/optuna/optuna-examples/pull/277, thanks kAIto47802!)
- Remove chainer CI (https://github.com/optuna/optuna-examples/pull/278)

Tests

- Suppress `ExperimentalWarning`s (https://github.com/optuna/optuna-integration/pull/108)
- Add a unit test for convergence of acquisition function in `GPSampler` (5365)
- Remove unused block (5368)
- Implement backward compatibility tests for `JournalStorage` (5486)
- Add some tests for visualization functions (5599)

Code Fixes

- Align run name strategy between `MLflowCallback` and `track_in_mlflow` method (https://github.com/optuna/optuna-integration/pull/111, thanks TTRh!)
- Use `TYPE_CHECKING` for `ObjectiveFuncType` (https://github.com/optuna/optuna-integration/pull/113)
- Ignore `ExperimentalWarning` for `track_in_wandb` (https://github.com/optuna/optuna-integration/pull/122)
- Remove unused module (https://github.com/optuna/optuna-integration/pull/127)
- Apply formatter and follow the Optuna conventions (https://github.com/optuna/optuna-integration/pull/133)
- Refactor an internal process of `BoTorchSampler` (https://github.com/optuna/optuna-integration/pull/147)
- Add missing experimental decorators to the comet integration (https://github.com/optuna/optuna-integration/pull/148)
- Use `__future__.annotations` (https://github.com/optuna/optuna-integration/pull/150)
- Fix E721 errors (https://github.com/optuna/optuna-integration/pull/151)
- Fix future annotations in `percentile.py` (5322, thanks aaravm!)
- Delete examples directory (5329)
- Simplify annotations in `optuna/pruners/_hyperband.py` (5338, thanks keita-sa!)
- Simplify annotations in `optuna/pruners/_successive_halving.py` and `optuna/pruners/_threshold.py` (5343, thanks keita-sa!)
- Reduce test warnings (5344)
- Simplify type annotations for `storages/_base.py` (5352, thanks Obliquedbishop!)
- Use `TYPE_CHECKING` for `Study` in `samplers` (5391)
- Refactor `_normalize_objective_values` in NSGA-III to supress `RuntimeWarning` (5399)
- Make `ObjectiveFuncType` available externally (5401)
- Replace `numpy` with `np` (5412)
- Bundle experimental feature warning (5434)
- Simplify annotations in `_brute_force.py` (5441)
- Add `__future__.annotations` to base sampler (5442)
- Introduce `__future__.annotations` to TPE-related modules (5443)
- Adapt to `__future__.annotations` in `optuna/storages/_rdb/models.py` (5452, thanks aisha-partha!)
- Debug an unintended assertion error in `GPSampler` (5484)
- Make `WFG` a function (5504)
- Enhance the comments in `create_new_trial` (5510)
- Replace single trailing underscore with double trailing underscore in `.rst` files (5514, thanks 47aamir!)
- Fix hyperlinks to use double trailing underscores (5515, thanks virendrapatil24!)
- Replace `_` with `__` in the link in Sphinx (5517)
- Refactor `plot_parallel_coordinate()` (5527, thanks karthikkurella!)
- Remove an obsolete TODO comment (5528)
- Unite the argument order of artifact APIs (5533)
- Make `assume_unique_lexsorted` in `is_pareto_front` required (5534)
- Simplify annotations in `terminator` module (5536)
- Simplify the type annotations in `samplers/_qmc.py` (5538, thanks kAIto47802!)
- Rename `solution_set` to `loss_vals` in hypervolume computation (5541)
- Expand the type of `callbacks` in optimize to `Iterable` (5542, thanks kz4killua!)
- Organize the directory structure of `JournalStorage` (5544)
- Add followup of journal storage module structure organization PR (5546)
- Move `RetryFailedTrialCallback` to `optuna.storages._callbacks` (5551)
- Change `removed_version` of deprecated `JournalStorage` classes (5552)
- Reorganize a journal storage module structure (5555)
- Remove unused private deprecated class `BaseJournalLogSnapshot` (5561)
- Replace relative import to absolute path (5569, thanks RektPunk!)
- Simplify annotation in `importance` (5578, thanks RektPunk!)
- Simplify annotation in `trial` (5579, thanks RektPunk!)
- Fix mypy warnings in `matplotlib/_contour.py` with Python3.11 (5580)
- Remove redundant loop in `_get_rank_subplot_info` (5581, thanks RektPunk!)
- Simplify annotations in `_partial_fixed.py` (5583)
- Simplify annotation in `_cmaes.py` (5584, thanks RektPunk!)
- Simplify annotations related to `GridSampler` (5588)
- Reorganize names related to `search_space` in `terminator/improvement/evaluator.py` (5594)
- Refactor `_bisect` in `_truncnorm.py` (5598)
- Simplify annotation storages (5608, thanks RektPunk!)
- Add a note about `np.unique` (5615)
- Fix ruff errors except for E731 (5627)

Continuous Integration

- Split ci (https://github.com/optuna/optuna-integration/pull/98)
- Install `pipdeptree` v2.16.2 to hotfix parse error (https://github.com/optuna/optuna-integration/pull/109)
- Remove dask version constraint (https://github.com/optuna/optuna-integration/pull/112)
- Rename CI name (https://github.com/optuna/optuna-integration/pull/120)
- Add `deprecated` arg for the comet CI job (https://github.com/optuna/optuna-integration/pull/135)
- Hotfix the CI in LightGBM test (https://github.com/optuna/optuna-integration/pull/142)
- Hotfix the mypy error of `test_pytorch_lightning.py` (https://github.com/optuna/optuna-integration/pull/144)
- Rename CI jobs (5353, thanks Obliquedbishop!)
- Update GitHub actions versions of `actions/setup-python` and `actions/checkout` (5367)
- Add `type: ignore` for CI hotfix (5419)
- Use CPU-only PyTorch wheels on GitHub Actions (5465)
- Add a version constraint to NumPy (5492)
- Add the note of the Hotfix in the workflow file (5494)
- Remove unnecessary version constraint from workflow (5495)
- Remove skipped tests for BoTorch with Python 3.12 (5519)
- Fix the version conflicts in the workflow for the minimum version constraints (5523)
- Replace `sleep` function with spin waiting to stabilize the frequent Mac test failure (5549)

Other

- Bump up version number to 4.0.0dev (https://github.com/optuna/optuna-integration/pull/102)
- Update PyPI classifiers for Python 3.11 support (https://github.com/optuna/optuna-integration/pull/129)
- Bump up version number to 4.0.0b0 (https://github.com/optuna/optuna-integration/pull/139)
- Bump up version number to 4.0.0 (https://github.com/optuna/optuna-integration/pull/158)
- Bump up to version number v4.0.0.dev (5319)
- Bump up to version number 4.0.0b0 (5572)
- Split `LICENSE` file (5597)
- Add OptunaHub section in `README.md` (5601)
- Remove duplicated definition of license in `pyproject.toml` (5645)
- Bump up to version number 4.0.0 (5653)
- Bump up to version number 4.1.0dev (5654)

Thanks to All the Contributors and Sponsors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

47aamir, Alnusjaponica, HideakiImamura, Obliquedbishop, RektPunk, TTRh, aaravm, aisha-partha, alxhslm, c-bata, caleb-kaiser, contramundum53, eukaryo, gen740, kAIto47802, karthikkurella, keisuke-umezawa, keita-sa, kz4killua, nabenabe0928, neel04, not522, nzw0301, porink0424, sgerloff, toshihikoyanase, virendrapatil24, y0z

Optuna is sponsored by the following sponsors on [GitHub](https://github.com/sponsors/optuna).

AlphaImpact, dec1costello, dubovikmaster, shu65, raquelhortab

This is the release note of [v4.0.0-b0](https://github.com/optuna/optuna/milestone/61?closed=1).

If you want to update your existing projects from Optuna v3.x to Optuna v4, please see the [migration guide](https://github.com/optuna/optuna/discussions/5573) and try out Optuna v4.

Highlights

OptunaHub Beta Release

The Optuna team released the beta version of OptunaHub, the feature-sharing platform for Optuna. Registered features can be easily implemented on users’ code and contributors can register the features they implement. The beta version of OptunaHub is now ready to accept contributions from all over the world. Visit [hub.optuna.org](https://hub.optuna.org)!

The following code shows an example to use a sampler registered on OptunaHub.

bash
% pip install optunahub


python
import optunahub
import optuna

def objective(trial):
x = trial.suggest_float("x", 0, 1)
return x

mod = optunahub.load_module("samplers/simulated_annealing")

sampler = mod.SimulatedAnnealingSampler()
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=20)



Stabilization of Artifact

The stable version of the artifact module is available, now equipped with several new APIs. This module introduces capabilities for managing the relatively large-sized data such as model snapshots in hyperparameter tuning, training/validation datasets, and etc. Compared to third-party libraries for experiment tracking, the advantage of using Optuna’s artifact module is a tight integration of [Optuna Dashboard](https://github.com/optuna/optuna-dashboard). This allows users to see artifacts (files) associated with the Optuna trial or study.

Here is a list of new APIs:
- [`download_artifact`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/artifacts.html#optuna.artifacts.download_artifact): Download an artifact from the artifact store.
- [`get_all_artifact_meta`](https://optuna.readthedocs.io/en/v4.0.0-b0/reference/artifacts.html#optuna.artifacts.get_all_artifact_meta): List the associated artifact information of the provided trial or study.


Stabilization of `JournalStorage`

The stable version of `JournalStorage` is available. This implies we have decided to maintain backward compatibility of the log format in the future releases.

Please note that this release introduces the following API changes to improve the clarity of class names and the module structure.

| Deprecated APIs | Corresponding active APIs |
-|-
| `optuna.storages.JournalFileStorage` | `optuna.storages.journal.JournalFileBackend` |
| `optuna.storages.JournalFileSymlinkLock` | `optuna.storages.journal.JournalFileSymlinkLock` |
| `optuna.storages.JournalFileOpenLock` | `optuna.storages.journal.JournalFileOpenLock` |
| `optuna.storages.JournalRedisStorage` | `optuna.storages.journal.JournalRedisBackend` |


Breaking Changes

- Delete deprecated three integrations, `skopt`, `catalyst`, and `fastaiv1` (https://github.com/optuna/optuna-integration/pull/114)
- Remove deprecated `CmaEsSampler` from integration (https://github.com/optuna/optuna-integration/pull/116)
- Remove verbosity of `LightGBMTuner` (https://github.com/optuna/optuna-integration/pull/136)
- Move positional args of LightGBM tuner (https://github.com/optuna/optuna-integration/pull/138)
- Remove `multi_objective` (5390)
- Delete deprecated `_ask` and `_tell` (5398)
- Delete deprecated `--direction(s)` arguments in the `ask` command (5405)
- Delete deprecated three integrations, `skopt`, `catalyst`, and `fastaiv1` (5407)
- Remove the default normalization of importance in f-ANOVA (5411)
- Remove `samplers.intersection` (5414)
- Drop implicit create-study in `ask` command (5415)
- Remove deprecated `study optimize` CLI command (5416)
- Remove deprecated `CmaEsSampler` from integration (5417)
- Support constrained optimization in `best_trial` (5426)
- Drop `--study` in `cli.py` (5430)
- Deprecate `constraints_func` in `plot_pareto_front` function (5455)
- Rename some classnames related to `JournalStorage` (5539)

New Features

- Add Comet ML integration (https://github.com/optuna/optuna-integration/pull/63, thanks caleb-kaiser!)
- Add Knowledge Gradient candidates functions (https://github.com/optuna/optuna-integration/pull/125, thanks alxhslm!)
- Add `is_exhausted()` function in the `GridSampler` class (5306, thanks aaravm!)
- Remove experimental from plot (5413)
- Implement `download_artifact` (5448)
- Add a function to list linked artifact information (5467)
- Stabilize artifact APIs (5567)
- Stabilize `JournalStorage` (5568)

Enhancements

- Pass two arguments to the forward of `ConstrainedMCObjective` to support `botorch=0.10.0` (https://github.com/optuna/optuna-integration/pull/106)
- Speed up non-dominated sort (5302)
- Make 2d hypervolume computation twice faster (5303)
- Reduce the time complexity of HSSP 2d from `O(NK^2 log K)` to `O((N - K)K)` (5346)
- Introduce lazy hypervolume calculations in HSSP for speedup (5355)
- Make `plot_contour` faster (5369)
- Speed up `to_internal_repr` in `CategoricalDistribution` (5400)
- Allow users to modify categorical distance more easily (5404)
- Speed up `WFG` by NumPy vectorization (5424)
- Check whether the study is multi-objective in `sample_independent` of `GPSampler` (5428)
- Suppress warnings from `numpy` in hypervolume computation (5432)
- Make an option to assume Pareto optimality in WFG (5433)
- Adapt multi objective to NumPy v2.0.0 (5493)
- Enhance the error message for integration installation (5498)
- Reduce journal size of `JournalStorage` (5526)
- Simplify a SQL query for getting the `trial_id` of `best_trial` (5537)

Bug Fixes

- Log `None` objective trials correctly (https://github.com/optuna/optuna-integration/pull/119, thanks neel04!)
- Pass two arguments to the forward of `ConstrainedMCObjective` in `qnei_candidates_func` (https://github.com/optuna/optuna-integration/pull/124, thanks alxhslm!)
- Allow single split cv in `OptunaSearchCV` (https://github.com/optuna/optuna-integration/pull/128, thanks sgerloff!)
- Fix `WilcoxonPruner` bug when `best_trial` has no intermediate value (5354)
- Debug an error caused by convergence in `GPSampler` (5359)
- Fix `average_is_best` implementation in `WilcoxonPruner` (5366)
- Create an unique renaming filename for each release operation in lock systems of `JournalStorage` (5389)
- Fix `_normalize_value` for incomplete trials (5422)
- Fix heartbeat for race condition (5431)
- Guarantee `weights_below` to be finite in MOTPE (5435)
- Fix `_log_complete_trial` for constrained optimization (5462)
- Convert `step` to `int` in `report` (5488)
- Use a fixed seed value when `seed=None` in `GridSampler` (5490)
- Acquire session lock only for write in heartbeat (5496)
- Fix a bug in `_create_new_trial` and refactor it (5497)
- Add rdb create new trial test (5525)
- Fix a bug in `sample_normalized_param` for `GPSampler` (5543)

Installation

- Add an option to install integration dependencies via pip (https://github.com/optuna/optuna-integration/pull/130)
- Add version constraint to numpy (https://github.com/optuna/optuna-integration/pull/131)

Documentation

- Enhance `README.md` (https://github.com/optuna/optuna-integration/pull/126)
- Add document page and fix docstring and comments (https://github.com/optuna/optuna-integration/pull/134)
- Use sphinx rst syntax (5345)
- Fix typo (5351)
- Update docs about `show_progress_bar` (5393)
- Fix artifact tutorial (5451)
- Revise docs to specify `ArtifactStore` methods as non-public (5474)
- Improve the docstring of `JournalFileStorage` (5475)
- Improve `make clean` in `docs` (5487)
- Improve the example of chemical structures in `optuna artifact tutorial` (5491)
- Add FAQ for artifact store remove API (5501)
- Add a documentation for `ArtifactMeta` (5511)
- Change docs version from latest to stable (5518)
- Add a list of supported storage for artifact store (5532)
- Rename filenames for `JournalFileStorage` in documents and a tutorial (5535)
- Add explanation for `lock_obj` to `JournalFileStorage` docstring (5540)
- Fix storages document sections (5553)
- Add `Returns` to the docstring of `load_study` (5554, thanks kAIto47802!)
- Fixing paths related to `storages.journal` (5560)
- Rename filename for `JournalFileBackend` in examples (5562)

Examples

- Support tensorflow 2.16.1 and separate CI run for tensorflow estimator (https://github.com/optuna/optuna-examples/pull/248)
- Replace deprecated jax function (https://github.com/optuna/optuna-examples/pull/250)
- Drop Python 3.7 support for wandb (https://github.com/optuna/optuna-examples/pull/252)
- Support python 3.12 in the CIs (https://github.com/optuna/optuna-examples/pull/253)
- Remove dask version constraint (https://github.com/optuna/optuna-examples/pull/254)
- Update GitHub actions versions to `actions/checkoutv4` and `actions/setup-pythonv5` (https://github.com/optuna/optuna-examples/pull/255)
- Delete an example using deprecated `fastaiv1` (https://github.com/optuna/optuna-examples/pull/256)
- Rename fastai v2 to fastai (https://github.com/optuna/optuna-examples/pull/257)
- Fix CI (https://github.com/optuna/optuna-examples/pull/258)
- Fix path in pruners workflow (https://github.com/optuna/optuna-examples/pull/259)
- Install `tensorflow-cpu` in CIs (https://github.com/optuna/optuna-examples/pull/260)
- Add python 3.12 and run terminator search cv (https://github.com/optuna/optuna-examples/pull/264)
- Organize the directory structure (https://github.com/optuna/optuna-examples/pull/266)
- Fix CI with version constraints of packages (https://github.com/optuna/optuna-examples/pull/267)
- Install `numpy<2.0.0` for catboot example (https://github.com/optuna/optuna-examples/pull/268)

Tests

- Suppress `ExperimentalWarning`s (https://github.com/optuna/optuna-integration/pull/108)
- Add a unit test for convergence of acquisition function in `GPSampler` (5365)
- Remove unused block (5368)
- Implement backward compatibility tests for `JournalStorage` (5486)

Code Fixes

- Align run name strategy between MLflowCallback and track_in_mlflow method (https://github.com/optuna/optuna-integration/pull/111, thanks TTRh!)
- Use `TYPE_CHECKING` for `ObjectiveFuncType` (https://github.com/optuna/optuna-integration/pull/113)
- Ignore `ExperimentalWarning` for `track_in_wandb` (https://github.com/optuna/optuna-integration/pull/122)
- Remove unused module (https://github.com/optuna/optuna-integration/pull/127)
- Apply formatter and follow the Optuna conventions (https://github.com/optuna/optuna-integration/pull/133)
- Fix future annotations in `percentile.py` (5322, thanks aaravm!)
- Delete examples directory (5329)
- Simplify annotations in `optuna/pruners/_hyperband.py` (5338, thanks keita-sa!)
- Simplify annotations in `optuna/pruners/_successive_halving.py` and `optuna/pruners/_threshold.py` (5343, thanks keita-sa!)
- Reduce test warnings (5344)
- Simplify type annotations for `storages/_base.py` (5352, thanks Obliquedbishop!)
- Use `TYPE_CHECKING` for `Study` in `samplers` (5391)
- Refactor `_normalize_objective_values` in NSGA-III to supress `RuntimeWarning` (5399)
- Make `ObjectiveFuncType` available externally (5401)
- Replace `numpy` with `np` (5412)
- Bundle experimental feature warning (5434)
- Simplify annotations in `_brute_force.py` (5441)
- Add `__future__.annotations` to base sampler (5442)
- Introduce `__future__.annotations` to TPE-related modules (5443)
- Adapt to `__future__.annotations` in `optuna/storages/_rdb/models.py` (5452, thanks aisha-partha!)
- Debug an unintended assertion error in `GPSampler` (5484)
- Make `WFG` a function (5504)
- Enhance the comments in `create_new_trial` (5510)
- Replace single trailing underscore with double trailing underscore in `.rst` files (5514, thanks 47aamir!)
- Fix hyperlinks to use double trailing underscores (5515, thanks virendrapatil24!)
- Replace `_` with `__` in the link in Sphinx (5517)
- Refactor `plot_parallel_coordinate()` (5527, thanks karthikkurella!)
- Remove an obsolete TODO comment (5528)
- Unite the argument order of artifact APIs (5533)
- Make `assume_unique_lexsorted` in `is_pareto_front` required (5534)
- Simplify annotations in `terminator` module (5536)
- Simplify the type annotations in `samplers/_qmc.py` (5538, thanks kAIto47802!)
- Rename `solution_set` to `loss_vals` in hypervolume computation (5541)
- Expand the type of `callbacks` in optimize to `Iterable` (5542, thanks kz4killua!)
- Organize the directory structure of `JournalStorage` (5544)
- Add followup of journal storage module structure organization PR (5546)
- Move `RetryFailedTrialCallback` to `optuna.storages._callbacks` (5551)
- Change `removed_version` of deprecated `JournalStorage` classes (5552)
- Reorganize a journal storage module structure (5555)
- Remove unused private deprecated class `BaseJournalLogSnapshot` (5561)

Continuous Integration

- Split ci (https://github.com/optuna/optuna-integration/pull/98)
- Install `pipdeptree` v2.16.2 to hotfix parse error (https://github.com/optuna/optuna-integration/pull/109)
- Remove dask version constraint (https://github.com/optuna/optuna-integration/pull/112)
- Rename CI name (https://github.com/optuna/optuna-integration/pull/120)
- Add `deprecated` arg for the comet CI job (https://github.com/optuna/optuna-integration/pull/135)
- Rename CI jobs (5353, thanks Obliquedbishop!)
- Update GitHub actions versions of `actions/setup-python` and `actions/checkout` (5367)
- Add `type: ignore` for CI hotfix (5419)
- Use CPU-only PyTorch wheels on GitHub Actions (5465)
- Add a version constraint to NumPy (5492)
- Add the note of the Hotfix in the workflow file (5494)
- Remove unnecessary version constraint from workflow (5495)
- Remove skipped tests for BoTorch with Python 3.12 (5519)
- Fix the version conflicts in the workflow for the minimum version constraints (5523)
- Replace `sleep` function with spin waiting to stabilize the frequent Mac test failure (5549)

Other

- Bump up version number to 4.0.0dev (https://github.com/optuna/optuna-integration/pull/102)
- Update PyPI classifiers for Python 3.11 support (https://github.com/optuna/optuna-integration/pull/129)
- Bump up version number to 4.0.0b0 (https://github.com/optuna/optuna-integration/pull/139)
- Bump up to version number v4.0.0.dev (5319)
- Bump up to version number 4.0.0b0 (5572)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

47aamir, Alnusjaponica, HideakiImamura, Obliquedbishop, TTRh, aaravm, aisha-partha, alxhslm, c-bata, caleb-kaiser, contramundum53, eukaryo, gen740, kAIto47802, karthikkurella, keisuke-umezawa, keita-sa, kz4killua, nabenabe0928, neel04, not522, nzw0301, porink0424, sgerloff, toshihikoyanase, virendrapatil24, y0z

This is the release note of v3.6.2.

Bug Fixes

- [Backport] Fix the default sampler of `load_study` function.

Other

- Bump up version number to v3.6.2

This is the release note of [v3.6.1](https://github.com/optuna/optuna/milestone/62?closed=1).

Bug Fixes

- [Backport] Fix Wilcoxon pruner bug when best_trial has no intermediate value 5370
- [Backport] Address issue5358 (5371)
- [Backport] Fix `average_is_best` implementation in `WilcoxonPruner` (5373)

Other

- Bump up version number to v3.6.1 (5372)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

HideakiImamura, eukaryo, nabenabe0928

This is the release note of [v3.6.0](https://github.com/optuna/optuna/milestone/60?closed=1).

Highlights

Optuna 3.6 newly supports the following new features. See [our release blog](https://medium.com/optuna/announcing-optuna-3-6-f5d7efeb5620) for more detailed information.
- Wilcoxon Pruner: New Pruner Based on Wilcoxon Signed-Rank Test
- Lightweight Gaussian Process (GP)-Based Sampler
- Speeding up Importance Evaluation with PED-ANOVA
- Stricter Verification Logic for FrozenTrial
- Refactoring the Optuna Dashboard
- Migration to Optuna Integration

Breaking Changes

- Implement `optuna.terminator` using `optuna._gp` (5241)

These migration-related PRs do not break the backward compatibility as long as optuna-integration v3.6.0 or later is installed in your environment.

- Move TensorBoard Integration (https://github.com/optuna/optuna-integration/pull/56, thanks dheemantha-bhat!)
- Delete TensorBoard integration for migration to `optuna-integration` (5161, thanks dheemantha-bhat!)
- Remove CatBoost integration for isolation (5198)
- Remove PyTorch integration (5213)
- Remove Dask integration (5222)
- Migrate the `sklearn` integration (5225)
- Remove BoTorch integration (5230)
- Remove `SkoptSampler` (5234)
- Remove the `cma` integration (5236)
- Remove the `wandb` integration (5237)
- Remove XGBoost Integration (5239)
- Remove MLflow integration (5246)
- Migrate LightGBM integration (5249)
- Add CatBoost integration (https://github.com/optuna/optuna-integration/pull/61)
- Add PyTorch integration (https://github.com/optuna/optuna-integration/pull/62)
- Add XGBoost integration (https://github.com/optuna/optuna-integration/pull/65, thanks buruzaemon!)
- Add `sklearn` integration (https://github.com/optuna/optuna-integration/pull/66)
- Move Dask integration (https://github.com/optuna/optuna-integration/pull/67)
- Migrate BoTorch integration (https://github.com/optuna/optuna-integration/pull/72)
- Move `SkoptSampler` (https://github.com/optuna/optuna-integration/pull/74)
- Migrate `pycma` integration (https://github.com/optuna/optuna-integration/pull/77)
- Migrate the Weights & Biases integration (https://github.com/optuna/optuna-integration/pull/79)
- Add LightGBM integration (https://github.com/optuna/optuna-integration/pull/81, thanks DanielAvdar!)
- Migrate `MLflow` integration (https://github.com/optuna/optuna-integration/pull/84)

New Features

- Backport the change of the timeline plot in Optuna Dashboard (5168)
- Wilcoxon pruner (5181)
- Add `GPSampler` (5185)
- Add a super quick f-ANOVA algorithm named PED-ANOVA (5212)

Enhancements

- Add `formats.sh` based on `optuna/master` (https://github.com/optuna/optuna-integration/pull/75)
- Use vectorization for categorical distance (5147)
- Unify implementation of fast non-dominated sort (5160)
- Raise `TypeError` if `params` is not a `dict` in `enqueue_trial` (5164, thanks adjeiv!)
- Upgrade `FrozenTrial._validate()` (5211)
- Import SQLAlchemy lazily (5215)
- Add UCB for `optuna._gp` (5224)
- Enhance performance of `GPSampler` (5274)
- Fix inconsistencies between terminator and its visualization (5276, thanks SimonPop!)
- Enhance `GPSampler` performance other than introducing local search (5279)

Bug Fixes

- Fix import path (https://github.com/optuna/optuna-integration/pull/83)
- Fix `README.md` (https://github.com/optuna/optuna-integration/pull/88)
- Fix `LightGBMTuner` test (https://github.com/optuna/optuna-integration/pull/89)
- Fix `JSONDecodeError` in `JournalStorage` (5195)
- Fix trial validation (5229)
- Make `gp.fit_kernel_params` more robust (5247)
- Fix checking value in `study.tell` (5269, thanks ryota717!)
- Fix `_split_trials` of `TPESampler` for constrained optimization with constant liar (5298)
- Make each importance evaluator compatible with doc (5311)

Documentation

- Remove `study optimize` from CLI tutorial page (5152)
- Clarify the `GridSampler` with ask-and-tell interface (5153)
- Clean-up `faq.rst` (5170)
- Make Methods section hidden from Artifact Docs (5188)
- Enhance README (5189)
- Add a new section explaing how to customize figures (5194)
- Replace legacy `plotly.graph_objs` with `plotly.graph_objects` (5223)
- Add a note section to explain that reseed affects reproducibility (5233)
- Update links to papers (5235)
- adding link for module's example to documetation for the `optuna.terminator` module (5243, thanks HarshitNagpal29!)
- Replace the old example directory (5244)
- Add Optuna Dashboard section to docs (5250, thanks porink0424!)
- Add a safety guard to Wilcoxon pruner, and modify the docstring (5256)
- Replace LightGBM with PyTorch-based example to remove `lightgbm` dependency in visualization tutorial (5257)
- Remove unnecessary comment in `Specify Hyperparameters Manually` tutorial page (5258)
- Add a tutorial of Wilcoxon pruner (5266)
- Clarify that pruners module does not support multi-objective optimization (5270)
- Minor fixes (5275)
- Add a guide to PED-ANOVA for `n_trials>10000` (5310)
- Minor fixes of docs and code comments for `PedAnovaImportanceEvaluator` (5312)
- Fix doc for `WilcoxonPruner` (5313)
- Fix doc example in `WilcoxonPruner` (5315)

Examples

- Remove Python 3.7 and 3.8 from tensorboard CI (https://github.com/optuna/optuna-examples/pull/231)
- Specify black version in the CI (https://github.com/optuna/optuna-examples/pull/232)
- Apply Black 2024 to codebase (https://github.com/optuna/optuna-examples/pull/236)
- Remove MXNet examples (https://github.com/optuna/optuna-examples/pull/237)
- Add an example of Wilcoxon pruner (https://github.com/optuna/optuna-examples/pull/238)
- Make Keras examples Keras 3 friendly (https://github.com/optuna/optuna-examples/pull/239)
- Remove a comment for keras that is not used anymore in this file (https://github.com/optuna/optuna-examples/pull/240)
- Use Keras 3 friendly syntax in MLflow example (https://github.com/optuna/optuna-examples/pull/242)
- Remove `-pre` option in the `rl` integration (https://github.com/optuna/optuna-examples/pull/243)
- Hotfix CI by adding version constraints to `dask` and `tensorflow` (https://github.com/optuna/optuna-examples/pull/245)

Tests

- Unify the implementation of `_create_frozen_trial()` under `testing` module (5157)
- Remove the Python version constraint for PyTorch (5278)

Code Fixes

- Fix unused (and unintended) import (https://github.com/optuna/optuna-integration/pull/68)
- Add Dask to `__init__.py` and fix its documentation generation (https://github.com/optuna/optuna-integration/pull/71)
- Replace `optuna.integration` with `optuna_integration` in the doc and the issue template (https://github.com/optuna/optuna-integration/pull/73)
- Fix the doc for TensorFlow (https://github.com/optuna/optuna-integration/pull/76)
- Add skopt dependency (https://github.com/optuna/optuna-integration/pull/78)
- Fastai readme fix (https://github.com/optuna/optuna-integration/pull/82, thanks DanielAvdar!)
- Fix `__init__.py` (https://github.com/optuna/optuna-integration/pull/86)
- Apply Black 2024 to codebase (https://github.com/optuna/optuna-integration/pull/87)
- Change the order of dependencies by name (https://github.com/optuna/optuna-integration/pull/92)
- Remove the deprecated decorator of `KerasPruningCallback` (https://github.com/optuna/optuna-integration/pull/93)
- Remove `UserWarning` by `tests/test_keras.py` (https://github.com/optuna/optuna-integration/pull/94)
- Refactor `TPESampler` for more clarity before c-TPE integration (5117)
- Fix `Checks(integration)` failure (5167)
- Fix type annotation of logging (5176)
- Update NamedTuple in `_ParzenEstimatorParameters` to more modern style (5193)
- Apply Black 2024 to codebase (5252)
- Simplify annotations in `optuna/study/_optimize.py` (5261, thanks shahpratham!)
- Unify and refactor `plot_timeline` test (5281)

Continuous Integration

- Remove non oldest and latest Python versions from tests (https://github.com/optuna/optuna-integration/pull/44)
- Fix flake8 failure in CI (https://github.com/optuna/optuna-integration/pull/55)
- Delete workflow dispatch input (https://github.com/optuna/optuna-integration/pull/57)
- Fix default branch (https://github.com/optuna/optuna-integration/pull/58)
- Fix coverage source path (https://github.com/optuna/optuna-integration/pull/60)
- Not use `black 24.*` (https://github.com/optuna/optuna-integration/pull/64)
- Simplify integration test (https://github.com/optuna/optuna-integration/pull/95)
- Hotfix the version of `botorch<0.10.` for CI failures (https://github.com/optuna/optuna-integration/pull/96)
- Hotfix the CI error by adding version constraint to dask (https://github.com/optuna/optuna-integration/pull/99)
- Fix tests with MPI (5166)
- Fix Checks (Integration) CI for NumPy 1.23.5 (5177)
- Add version constraint for black (5210)
- Skip the reproducibility tests for lightgbm (5214)
- Fix the errors in mypy for the `Checks (Integration)` CI (5217)
- Add a version constraint for Torch (5221)
- Hotfix mypy error in integration (5232)
- Skip `test_reproducible_in_other_process` for `GPSampler` with Python 3.12 (5251)
- Add CI settings to test Matplotlib without Plotly (5263, thanks DanielAvdar!)
- Unify indent size, two in toml file (5271)
- Follow up for split integrations (5277)
- Add a version constraint to `fakeredis` (5307)

Other

- Bump up version number to 3.6.0.dev (https://github.com/optuna/optuna-integration/pull/53)
- Bump up version number to 3.6.0 (https://github.com/optuna/optuna-integration/pull/100)
- Bump the version up to v3.6.0.dev (5143)
- Ignore auto generated files by Sphinx (5192)
- Delete `labeler.yml` to disable the `triage` action (5240)
- Bump up to version number 3.6.0 (5318)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

Alnusjaponica, DanielAvdar, HarshitNagpal29, HideakiImamura, SimonPop, adjeiv, buruzaemon, c-bata, contramundum53, dheemantha-bhat, eukaryo, gen740, hrntsm, knshnb, nabenabe0928, not522, nzw0301, porink0424, ryota717, shahpratham, toshihikoyanase, y0z