Mteb

Fix

* fix: Voyage bclavie/mmarco-japanese-hard-negatives (980)

* no message

* no message

* no message

* update metadata

* change metadata

* add results

* update metadata

* update metadata

* update metadata

* update n_samples and remove line ([`06e0a8b`](https://github.com/embeddings-benchmark/mteb/commit/06e0a8b54885f8886a6ffb57af2f17ed5c1743a7))

Unknown

* Update tasks table ([`4bef147`](https://github.com/embeddings-benchmark/mteb/commit/4bef147c5d9e702414881129476b7fa2e0c68a2c))

Fix

* fix: ensure that results from parallel datasets are formatted correctly (974)

* fix: ensure that results from parallel datasets are formatted corrected.
Additionally updated a few results.
* add pytest coverage
* remove unfinished results file
* Add test for multilingual subset loader
* removed upper bound on numpy
* sped up tests
* add trust remote code for new datasets
---------
Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`6004ec7`](https://github.com/embeddings-benchmark/mteb/commit/6004ec7b6e99afb2d31a41784ac0b3d4a6ded935))

Unknown

* Update points table ([`ae27b5c`](https://github.com/embeddings-benchmark/mteb/commit/ae27b5c16bd2655663be28f86657569e049d7ea4))

* LLM2Vec models (926)

* adding llm2vec model loader

* fix merge

* update import error

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* llm2vec use instructions

* prompt_name separate out of kwargs

* format

* scores

* use flash attention if available

* fixed bug for retieval

* user can provide instructions for LLM2Vec

* fix type error

* making code py 3.8 and 3.9 compatible

* proper storing results

* type combination compliant with py 3.8, 3.9

* add points

* add semicolon

* updated scores

* format

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com> ([`a6c784c`](https://github.com/embeddings-benchmark/mteb/commit/a6c784c1835ce11a9b1a974f8a6d43439d165805))

Fix

* fix: GritLM Retrieval instructions (981)

* Fix GritLM instructions

* Simplify

* Simplify

* Simplify

* Format

* Added e5 conditional encode for corpus/query

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com> ([`f1c9fc7`](https://github.com/embeddings-benchmark/mteb/commit/f1c9fc775ef0edfa83e6d32e365d1e12663273b8))

Unknown

* Update tasks table ([`b9f9ea7`](https://github.com/embeddings-benchmark/mteb/commit/b9f9ea77e3080e061e9da9f08d264d952f8511f6))

* Update points table ([`4f5542a`](https://github.com/embeddings-benchmark/mteb/commit/4f5542af892de26a55c3912c971853e7c80d489c))

* Move speedtask imports (982)

* move speedtask import
* move speedtask import
* add points file
* Update docs/mmteb/points/982.jsonl
Co-authored-by: Isaac Chung <chungisaac1217gmail.com>
---------
Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`58c7866`](https://github.com/embeddings-benchmark/mteb/commit/58c78667e4ec4e1d6add18ca2fd57fb01534a5e2))

Fix

* fix: add max_fraction_of_documents_to_embed to clustering datasets with `max_document_to_embed` (977)

* add max_fraction_of_documents_to_embed
* add points file
* add tests for clusteringfast
* review points
---------
Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`5367193`](https://github.com/embeddings-benchmark/mteb/commit/536719374e0e179fb1354f02e0a511d7ceefaa70))

Unknown

* Update points table ([`44af47a`](https://github.com/embeddings-benchmark/mteb/commit/44af47a04c953c4c1cdb627be4ed29609bce1f73))

* Update points table ([`e169fb3`](https://github.com/embeddings-benchmark/mteb/commit/e169fb30e4f2cb01cdd1fa0667a325a6a73b5c01))

* add points (979)

* add points for previous PRs.
* Update docs/mmteb/points/906.jsonl
Co-authored-by: Isaac Chung <chungisaac1217gmail.com>
* Update docs/mmteb/points/873.jsonl
Co-authored-by: Isaac Chung <chungisaac1217gmail.com>
---------
Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`1c8447d`](https://github.com/embeddings-benchmark/mteb/commit/1c8447dd1e9ad41748ed7bdaacf36cafa6a1754d))

* Fix GritLM instructions (976)

* Fix GritLM instructions

* make lint

---------

Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`a3410e9`](https://github.com/embeddings-benchmark/mteb/commit/a3410e919c1987521f5eb4a54ed7464fd44875e6))

* Update points table ([`8b6b08e`](https://github.com/embeddings-benchmark/mteb/commit/8b6b08e699fe74b0de535f847d87a3a733f977b5))

* Improve belebele retrieval task (894)

* Multilingual/BelebeleRetrieval: Fix self.langs to self.hf_subsets
* Multilingual/BelebeleRetrieval: Use lang+script code
* Multilingual/BelebeleRetrieval: Use 7 more scripts
* Multilingual/BelebeleRetrieval: convert to cross lingual task
* Multilingual/BelebeleRetrieval: add sample results
* Multilingual/BelebeleRetrieval: update historic results language codes
* Multilingual/BelebeleRetrieval: code formattin
* Multilingual/BelebeleRetrieval: update results
* Multilingual/BelebeleCrossRetrieval: Creating new version of Belebele task
* Multilingual/BelebeleCrossRetrieval: Restricting langauge pairs
* Multilingual/BelebeleCrossRetrieval: Formatting changes
* Multilingual/BelebeleRetrieval: Replace old task
* Multilingual/BelebeleRetrieval: Use underscore between language-script in language pair
* Add points
* points
---------
Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`3137f96`](https://github.com/embeddings-benchmark/mteb/commit/3137f96008acd07e1db438ce750b2cead45579cd))

Documentation

* docs: fix cli commands (966) ([`d403580`](https://github.com/embeddings-benchmark/mteb/commit/d40358089c959f812d150e40f9338f2d784bd73c))

Fix

* fix: Add Speed tasks for CPU and GPU & system info (967)

* move add_main_score to AbsTask
* add speed tasks
* add results
* Revert &34;move add_main_score to AbsTask&34;
This reverts commit 77cbb569b1e5f7d748b05e22d4b5a544ca064f86.
* fix tests
* add to requirements
* add psutil
* move to optional
* add comment on virtual_memory() returning Bytes
* add optional installs when testing
* make lint
* address review comments
* we still do points right? ([`d51fa81`](https://github.com/embeddings-benchmark/mteb/commit/d51fa818ad93588049b8e44b9845c044cb0eb93d))

* fix: bug in calculate_metadata_metrics for some retrieval datasets (965)

fix calculate_metadata_metrics ([`f3b5b1c`](https://github.com/embeddings-benchmark/mteb/commit/f3b5b1c77febd326615d645afb13635088c53474))

Unknown

* Update points table ([`88ee978`](https://github.com/embeddings-benchmark/mteb/commit/88ee978f1b900c4e1a39c1073ec4b178762e856b))

* Fix GritLM kwarg (970)

* Fix kwarg

* Default to None

Co-authored-by: Isaac Chung <chungisaac1217gmail.com>

* Format

---------

Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`5706956`](https://github.com/embeddings-benchmark/mteb/commit/5706956650a05af8aa33ab4e7db981321bab3f42))

* Rmv dup ([`81882aa`](https://github.com/embeddings-benchmark/mteb/commit/81882aa6bab85dbf12420cb196cb5bfa13deb550))

Fix

* fix: Add E5 model test case in test CLI (958)

* add multilingual e5 small in test case
* switch order of kwargs
* make lint
* default to None
* add prompts to class
* correct path with model names
* add points
* no need to comment
* remove __init__ method ([`6c0f597`](https://github.com/embeddings-benchmark/mteb/commit/6c0f597e62e165f6ecef3116c0002093fbdd610c))

Unknown

* Update points table ([`a14a03b`](https://github.com/embeddings-benchmark/mteb/commit/a14a03b1b2fba51f0ffdf84c6b0f89bc78ce1f18))