Mteb

Latest version: v1.36.22

Safety actively analyzes 715032 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 80 of 82

1.5.1

Fix

* fix: Added tests for checking datasets (307)

* fix: Fixed hf_hub_name for WikiCitiesClustering

* Added points for this PR and a 3 other minor dataset fixes

* feat: Added tests which validated that datasets are available

* fix: Updated hf references and revisions to multiple datasets

* Added points for submission

* fix: Added suggestions from the review

* Apply suggestions from code review

Co-authored-by: Imene Kerboua <33312980+imenelydiakerusers.noreply.github.com>

* fix: sped up async test for whether datasets exist

* fix: Updated revisions

* fix: reuploaded scandeval datasets

* fix: Applied formatter

---------

Co-authored-by: Imene Kerboua <33312980+imenelydiakerusers.noreply.github.com> ([`8d804f4`](https://github.com/embeddings-benchmark/mteb/commit/8d804f4956ea66b68b25c14ab05bc5ad2e102ff7))

1.5.0

Feature

* feat: Allow extending the load_dataset parameters in custom tasks inheriting AbsTask (299)

* Allow extending the load_dataset parameters

* format

* Fix test

* remove duplicated logic from AbsTask, now handled in the metadata

* add tests

* remove comments, moved to PR

* format

* extend metadata dict from super class

* Remove additional load_data

* test: adding very high level test

* Remove hf_hub_name and add test

* Fix revision in output file

---------

Co-authored-by: gbmarc1 <marcantoine.belangershopify.com> ([`953780d`](https://github.com/embeddings-benchmark/mteb/commit/953780dd680309cbae78f6543c005965ad9caf01))

1.4.1

Fix

* fix: hf_hub_name for WikiCitiesClustering (305)

* fix: Fixed hf_hub_name for WikiCitiesClustering

* Added points for this PR and a 3 other minor dataset fixes ([`b447235`](https://github.com/embeddings-benchmark/mteb/commit/b447235a18d2fb7261c6cb6cbb214e0bbd15452c))

1.4.0

Feature

* feat: Added windows support by replacing pytrec-eval with pytrec-eval-terrier (292)

* ci: Added windows to test suite

* feat: Changed to pytrec-eval-terrier to add support for windows installs ([`fc0e105`](https://github.com/embeddings-benchmark/mteb/commit/fc0e105e737352e0afd3a144717e7129654f8c90))

1.3.4

Fix

* fix: Update MindSmallReranking.py to have the correct hf reference (303) ([`102e24e`](https://github.com/embeddings-benchmark/mteb/commit/102e24e093c4414e3ecef49cb9d0ba94995f2eb3))

1.3.3

Documentation

* docs: Added information related to the automatic release (290)

* docs: added information related to the automatic release

* docs: removed test-parallel from docs

* docs: minor additions to contributing guidelines

* ci: removed changelog

As it already present in the git releases

* Apply suggestions from code review

Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com>

---------

Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com> ([`6821d23`](https://github.com/embeddings-benchmark/mteb/commit/6821d2324f41fed2bb686eedf074e4dc6803ece6))

Fix

* fix: fixed bug introduced in TatoebaBitextMining causing it to use a different dataset (297) ([`d0549a3`](https://github.com/embeddings-benchmark/mteb/commit/d0549a35f9697663a3e4bc6bdbe5990977d5c2e2))

* fix: Fixed mispecified rev. id for datasets (298)

* fix: fixed wrong rev. id for ToxicConversationsClassification

* fix: fixed wrong rev. id with RedditClusteringP2P ([`e1ae0d3`](https://github.com/embeddings-benchmark/mteb/commit/e1ae0d367eda70e6be31f0a43e20aed7cf75fedd))

Page 80 of 82

Links

Releases

Has known vulnerabilities

© 2025 Safety CLI Cybersecurity Inc. All Rights Reserved.