Annif

This release changes the Python version requirement to 3.6+ and drops the usage of Pipenv in development installations. The TensorFlow library has been upgraded to version 2.2, which means that all features are now supported also under Python 3.8.

The `eval` command is supplemented by introducing weighted subject average as a metric and the possibility to output metrics separately for each subject (thus allowing to explore e.g. how often a subject was suggested correctly); also some metrics are given more specified interpretation in the output.

Other changes include the possibility to display notation codes (when available) in web UI as well as minor improvements, bug fixes, and maintenance tasks.

New features:
- 392 Evaluate samples: specify interpretation of metrics (credit: Veldhoen)
- 391/393 Evaluation per-subject (credit: Veldhoen)
- 390/397 Show notations in web UI results list

Improvements:
- 405/403 Upgrade to TensorFlow 2.2, Python 3.6+, drop Pipenv
- 395/396 Don't give suggestions for empty input
- 389/401/402 Improved error handling in maui backend
- 399 Miscellaneous minor improvements for readthedocs builds

Maintenance:
- 400 Dockerfiles reorg and cleanup
- 407 Adding secrets needed by new Drone instance

Bug fixes:
- 394 Fix click 7.1 compatibility in tests
- 398 Fix silently failing readthedocs builds

This release includes improvements in training by reducing memory usage and adds the `--cached` option to `train` command to reuse the already preprocessed data from the previous run. Vocabulary management is improved by allowing to update the labels in an already existing vocabulary (renaming labels and removing subjects) without the need to retrain the project. Support of notation codes used in classifications (e.g. [UDC](https://en.wikipedia.org/wiki/Universal_Decimal_Classification) or YKL) is added.


New features:
- 342/376 --cached option to reuse preprocessed training data
- 274/383 Retain subject IDs when loading vocabulary over existing one
- 157/385 Support for notation codes


Improvements:
- 363/381 Use LMDB to store vectors in nn_ensemble
- 379 Use sparse vectors in PAV backend
- 382 Fix sonarqube errors (code quality problems)

Bug fixes:
- 386 Fix invalid "fasttext" package being installed
- 384 Remove duplicated be param option in `optimize` CLI

This patch release includes the changes necessitated by the update of `api.annif.org`:
- enabling `https` in Swagger
- installing `curl` needed by Docker healthcheck

This bugfix release fixes a problem with the Maui backend that was introduced by the parameter overriding support in 0.45:

* 372/373: Adapt the Maui backend for parameter overriding

(Annif 0.45.1 was an intermediate patch release where a Docker image build issue was fixed, with no changes in the Python codebase)

This release includes a new [omikuji backend](https://github.com/NatLibFi/Annif/wiki/Backend%3A-Omikuji) to support tree-based extreme multilabel classification machine learning algorithms, which give a big improvement to the quality of the subject indexing results. The `--backend-param/-p` option is introduced to the CLI `train` and `learn` commands (previously that option was only available for `suggest` and `eval`); the option can be used to override the parameters from the config file. Also Python 3.8 support is introduced - however, the `nn_ensemble` backend requires TensorFlow 2.0, which is not yet available for Python 3.8. The Vowpal Wabbit ensemble backend has been removed, as the neural network ensemble has similar features and gives better results.

New features:
- 343/366/368/371 Omikuji backend
- 250/289 Support backend param option in train and learn commands
- 345/370 Support for Python 3.8

Bug fixes:
- 369 Fix for spurious "analyzer setting is missing" errors under WSGI
- 360/361 Launching Gunicorn

Improvements/Maintenance:
- 367 Disable unnecessary Drone build dryruns for pushes
- 365 Remove vw_ensemble backend
- 359 Refactor backend project
- 358 Mauiserver dockerization

This release includes a new `maui` backend for integrating Annif with [Maui Server](https://github.com/TopQuadrant/MauiServer), a REST service wrapper for the Maui tool that will replace the similar but more limited Maui Service. The `eval` command has been enhanced by adding the F15 metric (F1 score for the top 5 suggestions) that is commonly used for comparing algorithms. There are also small improvements to the nn_ensemble backend and some bug fixes.

New features
* 269/344/352 Add Maui Server backend
* 354 Always compute F15 metric when evaluating

Improvements
* 355/356 Support `learn-epochs` parameter in nn_ensemble backend

Bug fixes
* 350/351 Fail gracefully if trying to evaluate an empty corpus
* 307/353 Accept UTF-8 files with Byte Order Mark (BOM)