Crepes

Features

- The `calibrate` methods of the classes `WrapClassifier` and `WrapRegressor` now take an additional argument `seed`, for setting the state of the random number generator. This allows for `predict_p` and `predict_set` of the former class and `predict_int` and `predict_cps` of the latter class as well as `evaluate` for both classes to become deterministic. The methods `predict_p`, `predict_set`, `predict_int` ,`predict_cps` and `evaluate` of these classes also have an argument `seed`, which can be used to over-ride any setting by the `calibrate` method. In addition, the corresponding methods of the classes `ConformalClassifier` and `ConformalPredictiveSystem` also include the argument `seed` for the same purpose. (The methods of `ConformalRegressor` currently contains no stochastic components and there is hence no need for a seed.) Thanks to egonmedhatten and tuvelofstrom for suggesting this extension.

- The `predict_p` methods of the classes `ConformalClassifier` and `WrapClassifier` now takes an optional argument `smoothing` to allow for generating both smoothed and non-smoothed p-values (default: `smoothing=True`).

- The default value for the parameter `smoothing` of the `predict_set` and `evaluate` methods has been changed to `True`.

- The class `DifficultyEstimator` in `crepes.extras` now includes a parameter `f` for providing a function to compute the difficulty estimates.

Fixes

- The documentation for the class `MondrianCategorizer` in `crepes.extras` has been corrected.

Features

- The class `MondrianCategorizer` was added to `crepes.extras`, for generating categories to be used by Mondrian conformal classifiers, regressors and predictive systems. See [the documentation](https://crepes.readthedocs.io/en/latest/crepes.extras.html) for the interface to objects of the class through the `fit` and `apply` methods.

- The class `WrapRegressor` has been updated so that the method `calibrate` allows for specifying an (optional) difficulty estimator and (optional) Mondrian categorizer, which are used both for calibration and making predictions, instead of requiring that difficulty estimates and Mondrian categories are provided separately for these tasks. The methods `predict_int`, `predict_cps` and `evaluate` no longer require `sigmas` and `bins` to be provided for normalized and Mondrian conformal regressors and predictive systems. Thanks to tuvelofstrom for suggestions along these lines.

- The class `WrapClassifier` has been updated so that the method `calibrate` allows for specifying an (optional) Mondrian categorizer, which is used both for calibration and making predictions. The methods `predict_p`, `predict_set` and `evaluate` no longer require `bins` to be provided for Mondrian conformal classifiers.

Fix

- Label vectors represented by pandas.Series are converted to NumPy arrays to avoid indexing issues. Thanks to valeman for pointing this out.

Fixes

- Fixed deprecated code for checking if an array is non-empty in the `ConformalPredictiveSystem` class. Thanks to tuvelofstrom for pointing this out.

Features

- The function `margin` for computing non-conformity scores for conformal classifiers has been added to `crepes.extras`.

Fixes

- Fixed a bug in the `DifficultyEstimator` class (in `crepes.extras`), which caused an error when trying to display a non-fitted object. Thanks to tuvelofstrom for pointing this out.

- Fixed an error in the documentation for the function `hinge`.

- The Jupyter notebooks `crepes_nb_wrap.ipynb` and `crepes_nb.ipynb` have been updated to illustrate the new `margin` function.

Features

- The classes `ConformalClassifier` and `WrapClassifier` have been added to `crepes`, allowing for generation of standard and Mondrian conformal classifiers, which produce p-values and prediction sets. The `calibrate` method of `WrapClassifier` allows for easily generating class-conditional conformal classifiers and using out-of-bag calibration. See [the documentation](https://crepes.readthedocs.io/en/latest/crepes.html) for the interface to objects of the class through the `calibrate`, `predict_p` and `predict_set` methods, in addition to the `fit`, `predict` and `predict_proba` methods of the wrapped learner. The method `evaluate` allows for evaluating the predictive performance using a set of standard metrics.

- The function `hinge` for computing non-conformity scores for conformal classifiers has been added to `crepes.extras`.

Fixes

- The class `Wrap` has changed name to `WrapRegressor` and the arguments to the `calibrate` method of this class have been changed to be in line with the `calibrate` method of `WrapClassifier`.

- The Jupyter notebooks `crepes_nb_wrap.ipynb` and `crepes_nb.ipynb` have been updated and extended

Fix

- Fixed a bug in the ``evaluate`` method of ``ConformalPredictiveSystem``, which caused an error when using ``CRPS`` as a single metric, i.e., when providing ``metrics=["CRPS"]`` as input. Thanks to Zeeshan-Khaliq for pointing this out.