Mteb

Fix

* fix: Get meta from CrossEncoder (1255)

* remove indent after return

* handle cross encoders for model meta

* make lint

* update filename since we now have model name ([`0ad5dad`](https://github.com/embeddings-benchmark/mteb/commit/0ad5dad6591ccbfbf3304525ea20fad9c2710cce))

Documentation

* docs: Small point changes & more contributors (1254)

* Update points.md

* Fix format

* Fix attribution ([`0d7664d`](https://github.com/embeddings-benchmark/mteb/commit/0d7664d22cf4eeedb5f84e162ab3aa0115dc7c7c))

Fix

* fix: Downsample large retrieval datasets (1236)

* most tasks

* lint

* fix other issues

* refactor

* lint and docs

* add polish

* keep case sensitive mteb paths

* add potential points

* fix points

* fix test about metadata

* update tasks and stats

* lint ([`b754f1a`](https://github.com/embeddings-benchmark/mteb/commit/b754f1a578cc6f6868f0666e7a0e2ac9158fe13c))

Unknown

* Update tasks table ([`e875401`](https://github.com/embeddings-benchmark/mteb/commit/e8754010b6b47118d7d6553098b5e37860ec5414))

* Update points table ([`a155401`](https://github.com/embeddings-benchmark/mteb/commit/a15540116446c6e1d3e0878c92eda3483ae1584a))

* Update points table ([`55e18a8`](https://github.com/embeddings-benchmark/mteb/commit/55e18a80bd91474eff286042c538462c04e034d6))

Documentation

* docs: Added coordination point for Jimmy Lee (1253)

docs: Added coordination point for Jimmy lee for his work on the coordination of Crystina and Nandan ([`6b27ce0`](https://github.com/embeddings-benchmark/mteb/commit/6b27ce0a589e0e8f1370dacd414fa61cf882fb8c))

* docs: Update affiliation (1247)

Update points.md ([`45de3ec`](https://github.com/embeddings-benchmark/mteb/commit/45de3eca1c2487573104e6facd113d2c40907a0f))

Fix

* fix: Add multilingual Benchmark (1252)

* fix: Add multilingual bench

* Update mteb/benchmarks/benchmarks.py

Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com>

* format

---------

Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com> ([`6a6259c`](https://github.com/embeddings-benchmark/mteb/commit/6a6259c39eda1d831073b7516ec158c617f782f3))

Unknown

* Update points table ([`4dbf784`](https://github.com/embeddings-benchmark/mteb/commit/4dbf78447a67bd9a3b14fd50bdb1cb4466fe1f95))

* Update points table ([`35abd84`](https://github.com/embeddings-benchmark/mteb/commit/35abd8495faa10f165ca108ca84baecb60e365a6))

* Add final author list ([`9ac24b7`](https://github.com/embeddings-benchmark/mteb/commit/9ac24b7b27d6dc6daf0ded3f91bac1b30bc62769))

* Merge branch &39;main&39; of https://github.com/embeddings-benchmark/mteb ([`8582244`](https://github.com/embeddings-benchmark/mteb/commit/8582244752e341c9a17e3121b7b82d4f82b7346b))

* Added author-information ([`c02c8e5`](https://github.com/embeddings-benchmark/mteb/commit/c02c8e5bbe575306a0c5f6b881a9958cfad2e84e))

Documentation

* docs: Create benchmarks overview table (1245)

* fix get_benchmarks method

* add create benchmark script

* make lint ([`fda9be1`](https://github.com/embeddings-benchmark/mteb/commit/fda9be1c085b5d83be58253e32e4aca8e4a2d594))

* docs: Add MTEB(code) dataset (1237)

* docs: Add MTEB(code) dataset

* Fix linting ([`f808863`](https://github.com/embeddings-benchmark/mteb/commit/f808863f5e393fc472058539fb113efe47e0abf4))

* docs: Update points (1228)

* Fix case

* Fix casing

* Fix case

* Fix case

* Create 971.jsonl

* Update contrib

* Add contributors ([`a636dc2`](https://github.com/embeddings-benchmark/mteb/commit/a636dc28e968e5689f465983cbdad40481893e6f))

Fix

* fix: mrshu&39;s name in `points.md` (1246)

* Use the diacritic character to be inline with Slovak spelling.

Signed-off-by: mr.Shu <mrshu.io> ([`3c06694`](https://github.com/embeddings-benchmark/mteb/commit/3c06694cabb6f6b8d71543aace90f1086cf296e5))

Unknown

* Add contributor (1243) ([`18f7306`](https://github.com/embeddings-benchmark/mteb/commit/18f730696451a5aaa026494cecf288fd5cde9fd0))

* Update of my affiliation (1242)

Update points.md ([`7469afa`](https://github.com/embeddings-benchmark/mteb/commit/7469afabbbc3b40ca85949472267ecf2baa09612))

* Update points table ([`eece6ec`](https://github.com/embeddings-benchmark/mteb/commit/eece6ecdb3248b1fb5a044d33c904f3d48cd6eab))

* Update points table ([`dddf5c2`](https://github.com/embeddings-benchmark/mteb/commit/dddf5c249bd102cd22f46965e38c1ebe0896a33d))

Documentation

* docs: clarify adding a model (1222) ([`25b7a2f`](https://github.com/embeddings-benchmark/mteb/commit/25b7a2fd1c6c1c24a48bf3c2c4d8c00dfa2820b9))

Fix

* fix: Add RepLLaMA style models (1223)

* init commit

* working and reproducing

* lint

* update hashes

* warning

* add pyproject ([`bedcfb3`](https://github.com/embeddings-benchmark/mteb/commit/bedcfb3e3991c0573aebb05add24c4f627e14f92))

Unknown

* Update points table ([`694e563`](https://github.com/embeddings-benchmark/mteb/commit/694e5639c0622af3bf90de5855c1af1ab01d86df))

* Update points table ([`4de39ee`](https://github.com/embeddings-benchmark/mteb/commit/4de39ee917565c8b5da0a101c4769e5b025745e2))

* Update points table ([`b66aaeb`](https://github.com/embeddings-benchmark/mteb/commit/b66aaeb0919707530a41ad983c61413bd6933a53))

Fix

* fix: Allow benchmark to specify eval_splits (1217)

* fix: Allow benchmark to specify eval_splits

This PR allow for benchmarks to specify specific eval. splits. This allow us to fully specify a benchmark within the benchmark object.

To do this it add the following:
- added eval_splits to the Abstask object, which default to metadata.eval_splits
- use the task.eval_splits unless overwritten in mteb.MTEB.run
- added eval_splits arg to mteb.get_tasks, which filter the tasks based on splits
- updated documentation
- renamed the &34;Advanced Usage&34; to &34;Usage Documentation&34; to make it more accicible
- added tests where relevant

* Added correction based on feedback ([`00260b5`](https://github.com/embeddings-benchmark/mteb/commit/00260b5497b4c82583be6383d8b22a3fceb64b54))