Worc

------------------

Fixed
~~~~~
- Bug in computing confidence intervals when performance was always the same,
results now in a np.nan confidence interval.
- Error catched and message added when you provide images and/or features in the
test set, but not labels.
- SimpleWORC and BasicWORC now detect whether user has provided a separate training
and test set and thus bootstrapping should be used.
- Bug in PREDICT was fixed that mixed up the mode in shape feature extraction (2D / 2.5D)
- Bug in performance calculation of multiclass classification.
- Bugs in statistical feature testing.

Changed
~~~~~~~
- Statistical test feature selection before PCA: otherwise when combined,
it will select PCA components, not features.

Added
~~~~~
- Histogram equalization to preprocessing.
- Recursive feature elimination (RFE) feature selection.
- Workflow to provide evaluate a trained model on new data
- Option to set a fixed random seed in the hyperoptimization
for reproducibility.
- Various FAQs
- Updated user manual with more extensive debugging guide.
- Thoroughly updated user manual documentation on different data flows, e.g.
train-test setups, multiple segmentations per patient.
- function in SimpleWORC to add parameter files for elastix.

------------------

Fixed
~~~~~
- Bug when using statisticaltestthreshold in multiclass classification,
see Github Issue 73.

------------------

Fixed
~~~~~
- Bug when using elastix, dimensionality got wrong name in fastr network.
- Bug in BasicWORC when starting from features
- For statistical test thresholding, if refit does not work during ensembling, skip this method
instead of returning NaN.
- Createfixedsplits function was outdated, updates to newest conventions and added
to documentation.
- When using Dummy's, segmentix now still copies metadata information from image
to segmentation when required.

Changed
~~~~~~~
- When part of fiting and scoring a workflow fails, during optimization return NaN as performance,
but during refitting skip that step. During optimization, if we skip it, the relation between
the hyperparameter and performance gets disturbed. During refitting, we need to have a model,
so best option is to skip the step. Previously, there was only skipping.
- Set default of XGB estimator parallelization to single job.

Added
~~~~~
- Documentation updates
- Option to save the workflows trained on the train-validation training datasets, besides the option to save workflows
trained on the full training dataset. Not an option for SMAC due to implementation of SMAC.
- Validate function for fastr HDF5 datatype, as previously fastr deemed empty hdf5 types as valid.
- Option to eliminate features which are all NaN during imputation.
- Preflightcheck on number of image types provided.

------------------

Added
~~~~~~~
- Bayesian optimization through SMAC as alternative to random search. Due
to specific requirements for SMAC, this is not by default installed.
Instructions to install this component are provided in the documentation.
- Besides Top_N ensembling, various other methods are added:
'ForwardSelection', 'Caruana', and 'Bagging'
- LightGBM classifier
- Light fingerprinting approach to adjust config based on dataset.

Fixed
~~~~~
- When none of the workflows to be included in the workflow converges during
retraining, add the next best performing workflows.

------------------

Fixed
~~~~~
- Bug in plotting 2D images when ranking the posteriors or percentages.
- Bug in Evaluate for Regression.
- Bug in BasicWORC for supplying label files.
- There is a bug in SimpleITK 2.1.0, which leads it to not be able
to read certain NIFTI files. See: https://github.com/InsightSoftwareConsortium/ITK/issues/2674
Reverting to SimpleITK 2.0.2 for now

Changed
~~~~~~~
- Defaults for splitting jobs in hyperoptimization: now more parallelized.

Added
~~~~~~~
- Documentation on additional functionality.

------------------
Unit tests moved from Travis to Github Workflow from this verion on.

Fixed
~~~~~
- Bug in plotting images when ranking the posteriors or percentages.
- Linting issues.

Changed
~~~~~~~
- Removed deprecated scatterplot, RankedSVM functions.