Scandeval

Latest version: v12.10.8

Safety actively analyzes 642283 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 17 of 21

0.16.0

Not secure
Fixed
- Upped the number index tokens for dependency parsing from 100 to 512. This
will need to be done better in the future, but is a fix for now.

Added
- Added the random models `random-roberta-sequence-clf` and
`random-roberta-token-clf` to the default list of model IDs when benchmarking
all models.

0.15.1

Not secure
Fixed
- The list of dependency tags in the `ndt-nb-dep` and `ndt-nn-dep` were wrong.
They have now been changed to all the tags occurring in the training sets.
- The `europarl_sent` data folder has now been renamed to `europarl`, so that
it can be loaded correctly with `load_dataset`.

0.15.0

Not secure
Added
- Added the Bokmål and Nynorsk POS and DEP parts of the Norwegian Dependency
Treebank dataset (NDT). They can be loaded as `ndt-nb-pos`, `ndt-nn-pos`,
`ndt-nb-dep` and `ndt-nn-dep`, respectively, from the CLI and the `Benchmark`
class.

Removed
- Removed the `EuroparlSubj` and `TwitterSubj` datasets, as they were too easy
and did not really differentiate models.
- Removed the abstract `SentimentClassificationBenchmark` and
`BinaryClassificationBenchmark`, to simplify the classes. There is now only
one `TextClassificationBenchmark`, which always evaluates with macro-F1.

Changed
- Changed the name of `europarl-sent` to `europarl`, as `europarl-subj` now
does not exist anymore.
- Changed the `nordial` dataset to the original 4-way classification dataset.

0.14.1

Not secure
Fixed
- Remove duplicate model IDs when calling the CLI or `Benchmark` class without
any specified model IDs.

0.14.0

Not secure
Added
- Added the Bokmål and Nynorsk parts of the NorNE dataset, for named entity
recognition. They can be loaded with the `norne-nb` and `norne-nn` names.
- There is now a `load_dataset` function, which can load any dataset, using the
dataset's name (same name as in the CLI). For instance,
`load_dataset('angry-tweets')` loads the `AngryTweets` dataset. This can be
imported directly from the package: `from scandeval import load_dataset`. The
individual dataset loading functions can still be imported as before; e.g.,
`from scandeval.datasets import load_angry_tweets`.

Changed
- Refactored folder structure with benchmarks and datasets.
- Separated `dane` and `dane-no-misc` into two distinct benchmark classes. The
`dane-no-misc` can now also be loaded with the `load_dataset` function.

0.13.0

Not secure
Added
- Added the Norwegian Review Corpus (NoReC), a sentiment classification dataset
in Norwegian.
- Added the Bokmål/Nynorsk part of the Norwegian Dialect dataset (NorDial), a
binary classification dataset in Norwegian.

Changed
- Changed the early stopping patience to `2 + 1000 // len(train)` from `2 + 250
// len(train)`, to allow more patience (and thus, more stability), for
smaller datasets.

Page 17 of 21

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.