Mteb

Documentation

* docs: typos; Standardize spacing; Chronological order (2436)

* Fix typos; add chrono order

* Fix spacing ([`0db0a20`](https://github.com/embeddings-benchmark/mteb/commit/0db0a20d48d422d3f3db25d50b73ada2525ccc19))

Fix

* fix: Add model specific dependencies in pyproject.toml (2424)

* Add model specific dependencies in pyproject.toml

* Update documentation ([`8a024be`](https://github.com/embeddings-benchmark/mteb/commit/8a024be2e69a4f42c8cc281a5af21e08d69ff0ff))

Unknown

* Added Memory Usage column on leaderboard (2428) ([`d3eab6f`](https://github.com/embeddings-benchmark/mteb/commit/d3eab6f1f483e4e5b9e1e871289397c8e114f713))

* add richinfoai models (2427)

* add richinfoai models

add richinfoai models

* format codes by linter

format codes by linter ([`98ab0ef`](https://github.com/embeddings-benchmark/mteb/commit/98ab0ef357a1233441b13f4137f3a7a9d5c28cc4))

* Update speed dependencies with new setuptools release (2429) ([`731c4fc`](https://github.com/embeddings-benchmark/mteb/commit/731c4fccf82f5841d30a50210b6945255d5dea58))

* Update tasks table ([`55c542b`](https://github.com/embeddings-benchmark/mteb/commit/55c542be5cd577159a74c0172e470daf16821d25))

* add __init__.py Clustering > kor folder, And edit __init__.py in Clustering folder (2422)

* add PatentFnBClustering.py

* do make lint and revise

* rollback Makefile

* Update mteb/tasks/Clustering/kor/PatentFnBClustering.py

Co-authored-by: Roman Solomatin <samoed.romangmail.com>

* klue_mrc_domain

* make lint

* klue_modified_clustering_dataset

* clustering & kor folder add __init.py

* clustering & kor folder add __init__.py

* task.py roll-back

* correct text_creation to sample_creation & delete form in MetaData

* correct task_subtype in TaskMetaData

* delete space

* edit metadata

* edit task_subtypes

---------

Co-authored-by: Roman Solomatin <samoed.romangmail.com> ([`39cee62`](https://github.com/embeddings-benchmark/mteb/commit/39cee62dc20c60f78143e261aae0a03784b06b17))

* HOTFIX: pin setuptools (2423)

* pin setuptools

* pin setuptools

* pin setuptools in makefile

* try ci

* fix ci

* remove speed from installs ([`071741d`](https://github.com/embeddings-benchmark/mteb/commit/071741d201643f5e2537cb4ac20545032914d5f4))

Fix

* fix: Add VDR Multilingual Dataset (2408)

* Added VDR Multilingual Dataset

* address comments

* make lint

* Formated Dataset for retrieval

* Update mteb/tasks/Retrieval/multilingual/VdrMultilingualRetrieval.py

Co-authored-by: Roman Solomatin <samoed.romangmail.com>

* Update mteb/tasks/Retrieval/multilingual/VdrMultilingualRetrieval.py

Co-authored-by: Roman Solomatin <samoed.romangmail.com>

* make lint

* corrected date

* fix dataset building

* move to image folder

---------

Co-authored-by: Roman Solomatin <samoed.romangmail.com>
Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`9d9b0b4`](https://github.com/embeddings-benchmark/mteb/commit/9d9b0b4329f29351b4c6dc596845205a97893744))

Unknown

* Update tasks table ([`34edcd5`](https://github.com/embeddings-benchmark/mteb/commit/34edcd552e9d4a055d794cdef5c985701b572892))

Fix

* fix: Add option to remove benchmark from leaderboard (2417)

fix: Add option to remove leaderboard from leaderboard

fixes 2413

This only removed the benchmark from the leaderboard but keep it in MTEB. ([`e8faf3f`](https://github.com/embeddings-benchmark/mteb/commit/e8faf3fb1c132d17caba4da5b6f570739f005a91))

Unknown

* Update tasks table ([`065159d`](https://github.com/embeddings-benchmark/mteb/commit/065159d999dacc2dce87beb5f876d2cc9ebcfc13))

* [MIEB] &34;capability measured&34;-Abstask 1-1 matching refactor [1/3]: reimplement CV-Bench (2414)

* refactor CV-Bench

* reimplement CV Bench

* remove abstask/evaluator/tests for Any2TextMultipleChoice

* rerun descriptive stats ([`2833138`](https://github.com/embeddings-benchmark/mteb/commit/283313845fdde1b07b33b280e74b7602bdcf79f3))

Fix

* fix: Add validation to model_name in `ModelMeta` (2404)

* add test for name validation

* upd docs

* upd cohere name

* fix tests

* fix name for average_word_embeddings_komninos

* fix name for average_word_embeddings_komninos

* fix reranker test

* fix reranker test ([`095851f`](https://github.com/embeddings-benchmark/mteb/commit/095851f66fbb87560fd677b04332b57293c3da61))

Unknown

* Min torchvision>0.2.1 (2410)

matching torch>1.0.0 ([`146a893`](https://github.com/embeddings-benchmark/mteb/commit/146a893c3fdcb0833fc1bd625ed567d6c2dba93d))

* remove Arabic_Triplet_Matryoshka_V2.py (2405) ([`811dbf6`](https://github.com/embeddings-benchmark/mteb/commit/811dbf6e16070731940c84de67ffb5bdbc256dcb))

Fix

* fix: Major updates to docs + make mieb dep optional (2397)

* fix: renaming Zeroshot -> ZeroShot

Adresses 2078

* fix: minor style changes

Adresses 2078

* fix: Major updates to documentation

This PR does the following:
- This introduced other modalities more clearly in the documentation as well as make it easier to transition to a full on documentation site later.
- added minor code updates due to discovered inconsistencies in docs and code.
- Added the MMTEB citation where applicable
- makes the docs ready to move torchvision to an optional dependency

* Moved VISTA example

* rename 1

* rename 2

* format

* fixed error

* fix: make torchvision optional (2399)

* fix: make torchvision optional

* format

* add docs

* minor fix

* remove transform from Any2TextMultipleChoiceEvaluator

---------

Co-authored-by: Isaac Chung <chungisaac1217gmail.com>

* move Running SentenceTransformer model with prompts to usage

---------

Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`cae1575`](https://github.com/embeddings-benchmark/mteb/commit/cae157558e984c8dd6472e0b65df786b8cd217ae))

Unknown

* Update tasks table ([`5b0bd56`](https://github.com/embeddings-benchmark/mteb/commit/5b0bd56748be08be89b569e680b842d27a66b95a))

* Added new dataset and tasks - ClusTREC-covid , clustering of thematic covid related scientific papers (2302)

* Clustrec covid new dataset and task

* fix

* fix

* fix

* fix

* fix

* descriptive stats

* change all mentions of clustrec-covidp2p to clustrec-covid

* change &39; to &34; ([`e2476d2`](https://github.com/embeddings-benchmark/mteb/commit/e2476d2644e2753cf0671f3b221032a452ab906e))

* Minor style changes (2396)

* fix: renaming Zeroshot -> ZeroShot

Adresses 2078

* fix: minor style changes

Adresses 2078

* rename 1

* rename 2

* format

* fixed error

---------

Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`8be95b7`](https://github.com/embeddings-benchmark/mteb/commit/8be95b78c6de0023d0429bb0af4ffc4b202584cf))

Fix

* fix: Update AmazonPolarityClassification license (2402)

Update AmazonPolarityClassification.py ([`cf84a79`](https://github.com/embeddings-benchmark/mteb/commit/cf84a79085c1499235c519d5fb913019870c060e))

Unknown

* fix b1ade name (2403) ([`a0990cb`](https://github.com/embeddings-benchmark/mteb/commit/a0990cb21dd77583ca5a283c262ce08cfb7759d3))