Mteb

Fix

* fix: Missing fixes for 2086 - change MultilingualSentiment split from test to validation in CMTEB (2088)

* fix: Fixed previous incorrect specification of splits for CMTEB ( MTEB(cmn, v1) )

Fixes 2064

* change MultilingualSentiment split from test to validation in CMTEB ([`6637ff9`](https://github.com/embeddings-benchmark/mteb/commit/6637ff95945b12a61594d09eafe88c82d3dfe4e4))

* fix: Smarter leaderboard caching with cachetools (2085)

* Added smarter caching to callbacks

* Added cachetools as a dependency

* Ran linting

* Removed debugging print statement

* Bumped Gradio version

* Dependency fixes

* Dependency fixes

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com> ([`1006770`](https://github.com/embeddings-benchmark/mteb/commit/1006770c098869d5c9db2e1b3b13c2c190c34a26))

Unknown

* Remove duplicated string in docstring of TaskMetadata class (2087)

* Remove duplicated string in docstring of TaskMetadata class

* Remove duplicated dataset field ([`c6e5123`](https://github.com/embeddings-benchmark/mteb/commit/c6e51230a9a65b9e5c25bae54e8d4b2c8095c7f8))

Fix

* fix: Fixed previous incorrect specification of splits for CMTEB ( MTEB(cmn, v1) ) (2086)

Fixes 2064 ([`12d9b96`](https://github.com/embeddings-benchmark/mteb/commit/12d9b96842d64a159dba39013b2a121e7b436f9b))

Fix

* fix: Freeze model/rank columns in leaderboard (2044)

* fix: freeze model/rank columns in leaderboard

* freezing zero-shot column

* update min gradio version to 5.16.0 in pyproject.toml

---------

Co-authored-by: Shikhar Shiromani <sshiromanisshiromani-mlt.client.nvidia.com> ([`07562f4`](https://github.com/embeddings-benchmark/mteb/commit/07562f4d27c8319249c2a28ff9903ad5bf3b4173))

Fix

* fix: Ensure voyage model uses different naming scheme (2083)

* fix: Added make command for running leaderboard locally

* fix: Ensure voyage models doesn&39;t re-use the name ([`b14963f`](https://github.com/embeddings-benchmark/mteb/commit/b14963fe6ace93ca9c1e7e066577f7e44250f823))

Fix

* fix: add missing `e5` training datasets (2065)

add missing training datasets ([`efe2578`](https://github.com/embeddings-benchmark/mteb/commit/efe2578c06265419d6ea613108d156bb4f124f8f))

Ci

* ci: Rerun tests that fail due to networking issues. (2029)

* fix: rerun tests that fail - Networking

* update tests to use tmp_path

* set versions for dev dependencies

* add pytest options to pyproject.toml

* add rerun json.decoder.JSONDecodeError

* remove JSONDecodeError from pyproject.toml

* add huggingface_hub.errors.HfHubHTTPError

* add huggingface_hub.errors.LocalEntryNotFoundError
https://github.com/embeddings-benchmark/mteb/actions/runs/13298535701/job/37139767443?pr=2044

* FileNotFoundError
https://github.com/embeddings-benchmark/mteb/actions/runs/13302915091/job/37147507251?pr=2029

* add doc to pytest rerun

---------

Co-authored-by: sam021313 <40773225+sam021313users.noreply.github.com> ([`efaa990`](https://github.com/embeddings-benchmark/mteb/commit/efaa990b6c7a4916c3349bda4204b2b323927966))

Fix

* fix: generate metadata (2063)

* fix: generate metadata

* use logging not print for script

* lint

* add iso639 to dev pyproject

* fix import

* add memory_usage_mb

* set version for iso639

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

---------

Co-authored-by: sam021313 <40773225+sam021313users.noreply.github.com>
Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>
Co-authored-by: Roman Solomatin <samoed.romangmail.com> ([`26360a0`](https://github.com/embeddings-benchmark/mteb/commit/26360a0b856d97cf8b589218365288bdf55ae791))

Unknown

* Update tasks table ([`3dbdeb1`](https://github.com/embeddings-benchmark/mteb/commit/3dbdeb1f2664e3502ef68bcc84bd05f8d7c0c342))

* misc: Add all image classification descriptive stats (2073)

* add most image classification descr stats

* revert changes to encoder

* add stats

---------

Co-authored-by: Roman Solomatin <36135455+Samoedusers.noreply.github.com> ([`04c9993`](https://github.com/embeddings-benchmark/mteb/commit/04c9993f3c5478f76253141ec1b0170522d821a2))

* misc: update model names to adjust for adding to results repo (2074)

* update model names to adjust for adding to results repo

* update model meta script ([`50cc1c9`](https://github.com/embeddings-benchmark/mteb/commit/50cc1c995765dacc820d19f632bdea2f5fbef260))

* Update tasks table ([`dbda3c5`](https://github.com/embeddings-benchmark/mteb/commit/dbda3c59024c4042c2d3e37c61a5e2ee16e4e8ed))

* Add datasets for a benchmark newly introduced for &34;Engineering&34; domain (1911)

* adding clustering tasks (built-bench-clustering S2S & P2P)

* updated built-bench-clustering tasks

* Updated BuiltBenchClustering tasks

* Added &34;Engineering&34; as new domain to TaskMetadata.py
* Updated tasks table in docs
* Updated task metadata for BuiltBenchClustering S2S and P2P

* updated metadata for clustering tasks

* Add/update BuiltBench tasks

- Add BuiltBenchRetrieval task
- Add BuiltBenchReranking task
- Update metadata for BuiltBenchClusterinP2P
- Update metadata for BuiltBenchClusterinS2S

* update BuiltBench benchmark

* Update mteb/benchmarks/benchmarks.py

Co-authored-by: Roman Solomatin <samoed.romangmail.com>

* Update mteb/tasks/Clustering/eng/BuiltBenchClusteringS2S.py

Co-authored-by: Roman Solomatin <samoed.romangmail.com>

* Update mteb/tasks/Clustering/eng/BuiltBenchClusteringP2P.py

Co-authored-by: Roman Solomatin <samoed.romangmail.com>

* Update mteb/benchmarks/benchmarks.py

Co-authored-by: Isaac Chung <chungisaac1217gmail.com>

* Fix formatting via ruff

---------

Co-authored-by: Roman Solomatin <samoed.romangmail.com>
Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`5f4b593`](https://github.com/embeddings-benchmark/mteb/commit/5f4b593416ca8615f128a04844b50eb2dd4c09b9))