Sklearn-genetic-opt

This release comes with some requested features and enhancements.

Features:

* Class `GAFeatureSelectionCV` now has a parameter called `max_features`, int, default=None. If it's not None, it will penalize individuals with more features than max_features, putting a "soft" upper bound to the number of features to be selected.

* Classes `GASearchCV` and `GAFeatureSelectionCV` now support multi-metric evaluation the same way scikit-learn does; you will see this reflected on the `logbook` and `cv_results_` objects, where now you get results for each metric. As in scikit-learn, if multi-metric is used, the `refit` parameter must be a str specifying the metric to evaluate the cv-scores.

* Training gracefully stops if interrupted by some of these exceptions: ``KeyboardInterrupt``, ``SystemExit``, ``StopIteration``.
When one of these exceptions is raised, the model finishes the current generation and saves the current best model. It only works if at least one generation has been completed.

API Changes:

* The following parameters changed their default values to create more extensive and different models with better results:

- population_size from 10 to 50

- generations from 40 to 80

- mutation_probability from 0.1 to 0.2

Docs:

* A new notebook called Iris_multimetric was added to showcase the new multi-metric capabilities.

This is an exciting release! It introduces features selection capabilities to the package

Features:

* `GAFeatureSelectionCV` class for feature selection along with any scikit-learn classifier or regressor. It optimizes the cv-score while minimizing the number of features to select. This class is compatible with the mlflow and tensorboard integration, the Callbacks, and the plot_fitness_evolution function.

API Changes:

The module mlflow was renamed to mlflow_log to avoid unexpected errors on name resolutions

This is a minor release that fixes a couple of bugs and adds some minor options.

Features:

* Added the parameter `generations` to `DeltaThreshold`. Now it compares the maximum and minimum values of a metric from the last generations, instead of just the current and previous ones. The default value is 2, so the behavior remains the same as in previous versions.

Bug Fixes:

* When a param_grid of length 1 is provided, a user warning is raised instead of an error. Internally it will swap the crossover operation to use the DEAP's `tools.cxSimulatedBinaryBounded`.
* When using `Continuous` class with boundaries `lower` and `upper`, a uniform distribution with limits `[lower, lower + upper]` was sampled, now, it's properly sampled using a `[lower, upper]` limit.

This is a big release with several new features and enhancements! 🎊

Features:

* Added the `ProgressBar` callback, it uses tqdm progress bar to shows how many generations are left in the training progress.
* Added the `TensorBoard` callback to log the generation metrics, watch in real-time while the models are trained, and compare different runs in your TensorBoard instance.
* Added the `TimerStopping` callback to stop the iterations after a total (threshold) fitting time has been elapsed.
* Added new parallel coordinates plot using `plot_parallel_coordinates` by Raul9595
* Now if one or more callbacks decides to stop the algorithm, it will print its class name to know which callbacks were responsible of the stopping.
* Added support for extra methods coming from scikit-learn's BaseSearchCV, like `cv_results_`,
`best_index_` and `refit_time_` among others.
* Added methods `on_start` and `on_end` to `BaseCallback`. Now the algorithms check for the callbacks like this:

- **on_start**: When the evolutionary algorithm is called from the GASearchCV.fit method.

- **on_step:** When the evolutionary algorithm finishes a generation (no change here).

- **on_end:** At the end of the last generation.

Bug Fixes:

* A missing statement was making that the callbacks start to get evaluated from generation 1, ignoring generation 0. Now this is properly handled and callbacks work from generation 0.

API Changes:

* The modules `sklearn_genetic.plots` and `sklearn_genetic.mlflow.MLflowConfig` now requires an explicit installation of seaborn and mlflow, now those are optionally installed using ``pip install sklearn-genetic-opt[all].``
* The GASearchCV.logbook property now has extra information that comes from the scikit-learn cross_validate function.
* An optional extra parameter was added to GASearchCV, named `return_train_score`: bool, default=False. As in scikit-learn, it controls if the `cv_results_` should have the training scores.

Docs:

* Edited all demos to be in the jupyter notebook format.
* Added embedded jupyter notebooks examples in read the docs page.
* The modules of the package now have a summary of their classes/functions in the docs.
* Updated the callbacks and custom callbacks tutorials to add a new TensorBoard callback and the new methods on the base callback.

Internal:

* Now the HallofFame (hof) uses the `self.best_params_` for the position 0, to be consistent with the
scikit-learn API and parameters like `self.best_index_`
* MLflow now has unit tests by Turtle24

Thanks to new contributors for helping in this project! Raul9595 Turtle24

Features:

* Build-in integration with MLflow using the class `sklearn_genetic.mlflow.MLflowConfig` and the new parameter `log_config` from the class `sklearn_genetic.GASearchCV`

* Implemented the callback `sklearn_genetic.callbacks.LogbookSaver` which saves the estimator.logbook object with all the fitted hyperparameters and their cross-validation score

* Added the parameter `estimator` to all the functions on the module `sklearn_genetic.callbacks`

Docs:

* Added user guide "Integrating with MLflow"
* Update the tutorial "Custom Callbacks" for new API inheritance behavior

Internal:


* Added a base class `sklearn_genetic.callbacks.base.BaseCallback` from which all Callbacks must inherit from
* Now coverage report doesn't take into account the lines with pragma: no cover and noqa

Docs:

* Added user guide on "Understanding the evaluation process"
* Several guides on contributing, code of conduct
* Added important links
* Docs requirement are now independent of package requirements

Internal:

* Changed test ci from travis to Github actions