Mteb

Fix

* fix: Update annotations for multilingual classification tasks (923)

* Update MTOPIntentClassification.py

* Update MTOPDomainClassification.py

* Update MassiveIntentClassification.py

* Update MassiveScenarioClassification.py

* Update MassiveScenarioClassification.py

* Update MassiveIntentClassification.py

* add points

---------

Co-authored-by: Tikhonova Maria <m_tikhonova94mail.ru> ([`568651b`](https://github.com/embeddings-benchmark/mteb/commit/568651bab6dd57ae8fc2216d5e192da351f44a98))

Unknown

* Update tasks table ([`42d4ecd`](https://github.com/embeddings-benchmark/mteb/commit/42d4ecd63d50301ea0f094e3189457fb18cd6c39))

* Update points table ([`53fb2f8`](https://github.com/embeddings-benchmark/mteb/commit/53fb2f8ed6de90b1d2a9a2e8723771aaefcfaede))

Fix

* fix: Added the ability to make task specific instructions, tested using e5 instruct (888)

* Added prompting along with e5 instruct

All encode calls now include a prompt_name (following the variable name in sentence transformer&39;s encode), which provided the task name. The model can then use a custom prompt pr. task if they wish (or encode based on e.g. task type).

Additionally to test it out I have also added the two e5 instruct models.

* format

* updated based on review

* format

* Update mteb/models/instructions.py

Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com>

* Update mteb/models/__init__.py

Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com>

* Added additional tests for instructions

---------

Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com> ([`40208cf`](https://github.com/embeddings-benchmark/mteb/commit/40208cfaf0bf588515e3a077b4d7e5fc5fa98200))

Unknown

* Update points table ([`b97ad77`](https://github.com/embeddings-benchmark/mteb/commit/b97ad779c3eac25d9ac8098bf84300197964666f))

Fix

* fix: Fix create_meta function in cli.py (912)

* Update cli.py

Fix create_meta function

* Update test files ([`af365b7`](https://github.com/embeddings-benchmark/mteb/commit/af365b795e2c54b25b0e02bbb0788ba9d3c403c7))

Unknown

* format ([`c47d279`](https://github.com/embeddings-benchmark/mteb/commit/c47d2795b4297792aa55f059b833055a4a13106c))

* Fix GritLM Loading (925)

* add gritlm to init file

* sort

---------

Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com> ([`55fb52e`](https://github.com/embeddings-benchmark/mteb/commit/55fb52e514109aef8a8077bf88ffd16c950eb4c2))

Documentation

* docs: Add points for 911 (913)

Add poinhts ([`5deeb3c`](https://github.com/embeddings-benchmark/mteb/commit/5deeb3c6351fbedde24dc77d8981e32562d9b516))

Fix

* fix: Update annotations for English STS tasks (908)

* Update STS16STS.py

* Update STS15STS.py

* Update STS14STS.py

* Update STS13STS.py

* add points for 902&908 prs

* Update STS16STS.py

* Update STS14STS.py

* Update STS14STS.py

* Update STS16STS.py

---------

Co-authored-by: Tikhonova Maria <m_tikhonova94mail.ru> ([`f1dd8bb`](https://github.com/embeddings-benchmark/mteb/commit/f1dd8bb60fe730d34fcd7bb909834ad1d9daba80))

Unknown

* Update tasks table ([`844e743`](https://github.com/embeddings-benchmark/mteb/commit/844e7432de16d4ffea07bd4312f18da1e7bc6fdc))

* Update points table ([`cffd94c`](https://github.com/embeddings-benchmark/mteb/commit/cffd94c1699fd9a4f954950eb7adcebee169da45))

Documentation

* docs: Update annotations for multilingual STS tasks (902)

* Update STS12STS.py

* Update STSBenchmarkMultilingualSTS.py

* Update STS22CrosslingualSTS.py

* Update STS17CrosslingualSTS.py

* Update STS22CrosslingualSTS.py

* Update STSBenchmarkMultilingualSTS.py

* Update STS17CrosslingualSTS.py

* Update STS12STS.py

* Update STS17CrosslingualSTS.py

* Update STSBenchmarkMultilingualSTS.py

* Update STS22CrosslingualSTS.py

---------

Co-authored-by: Tikhonova Maria <m_tikhonova94mail.ru> ([`4383fd3`](https://github.com/embeddings-benchmark/mteb/commit/4383fd3a65c16df9598581f32643408e33273cb0))

Fix

* fix: Incorrect handling of qrel_revision fix: 909 (911)

* fix: 909

* fix: 909

* add: points

* clean ([`63cd4b7`](https://github.com/embeddings-benchmark/mteb/commit/63cd4b7f99e9371a90e26014a328231b1f3db810))

Unknown

* Update tasks table ([`30433a9`](https://github.com/embeddings-benchmark/mteb/commit/30433a9a7d8ef6b2b47a95cf2ff9b002088b8a03))

* Update points table ([`d771437`](https://github.com/embeddings-benchmark/mteb/commit/d77143791408269ea69fd6455f4dd5a6782d1805))

* (1) Add WebLINX Candidate Elements Reranking Task; (2) move `convert_conv_history_to_query` from `mteb.evaluation.evaluators.RetrievalEvaluator` to `mteb.evaluation.evaluators.utils` (820)

* Initial file (WIP)

* Update dates, and indicate dotos

* Update mteb/tasks/Reranking/eng/WebLINXCandidatesReranking.py

* Move `convert_conv_history_to_query` from RetrievalEvaluator to utils

* Add WebLINXCandidatesReranking to mteb.tasks.Reranking

* Lint RetrievalEvaluator&39;s imports

* lint mteb.evaluation.evaluator.utils

* Update dataset path, name and revision, n_samples and avg_character_lenght

* Update revision, add load_data method

* Fix typo

* Change main score to mrr, since recall10 is not supported

* Add average results

* Add points

* Run `make lint` ([`e34ddaa`](https://github.com/embeddings-benchmark/mteb/commit/e34ddaac76777e4f3a360a30b5ee2d111c5a9f61))

* Update tasks table ([`b16d7d5`](https://github.com/embeddings-benchmark/mteb/commit/b16d7d5620b987992e16ee663e5d82ee007acd8e))

Documentation

* docs: Add points for paper writing (901)

* add points for paper writing

* add PR number

* correct typo ([`fbbc44b`](https://github.com/embeddings-benchmark/mteb/commit/fbbc44b51702930d3dafbc6f7529233c5de7a138))

Fix

* fix: Backfilled bibtex citations data (900)

* Added IndicNLP News Classificaiton

* Added IndicNLP News Classificaiton

* Added results

* Updated dataset version

* Small fixes

* Small fix

* Small fix

* Updated results

* Fix linting issues

* Added points

* Resolve conflict

* Update 610.jsonl

* Backfilled missing bibtex citations.

* Backfilled missing bibtex citations.

* Remove non-present files

---------

Co-authored-by: Imene Kerboua <33312980+imenelydiakerusers.noreply.github.com>
Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com> ([`77d0e06`](https://github.com/embeddings-benchmark/mteb/commit/77d0e06e88e7645d14c278b25a9bbb6c97b8fed6))

Unknown

* Update points table ([`a7ce58f`](https://github.com/embeddings-benchmark/mteb/commit/a7ce58f81728e7adaba940d30f41fe124e4ad64d))

* Update tasks table ([`97eb8b3`](https://github.com/embeddings-benchmark/mteb/commit/97eb8b342f42a372dc74449c383ecc20c3c9f7a3))