Fixed - Only try to to merge the `id2label` and `label2id` conversions if the model is finetuned. This caused some errors when a model was not finetuned but somehow still had conversion dictionaries.
1.5.2
Not secure
Fixed - Deal with models with tasks `feature-extraction` or `sentence-similarity` as if they were `fill-mask`, meaning assume that they are merely pretrained models, rather than finetuned.
1.5.1
Not secure
Fixed - Fixed bug when evaluating a finetuned model.
1.5.0
Not secure
Changed - Added progress bar description when evaluating models without finetuning them first. - Lowered the package requirements to the earliest possible versions.
Removed - Removed support for TensorFlow and Jax models, due to them not working properly anyway. They might be included at a later point, properly.
1.4.0
Not secure
Changed - Now also outputting aggregated metrics in the resulting `scandeval_benchmark_results.json` file. This `json` file now has keys `raw_metrics` and `total`, with `raw_metrics` containing the previous (raw) scores, and the value of the new `total` key has aggregated scores (means and standard errors).
1.3.8
Not secure
Changed - All training/evaluation progress bars are now removed when they are finished, and the training progress bar has no total anymore, as it was misleading.