Mteb

Documentation

* docs: Added a script to extract the bibtex citations and generate the consolidated bib file (904)

* Added IndicNLP News Classificaiton

* Added IndicNLP News Classificaiton

* Added results

* Updated dataset version

* Small fixes

* Small fix

* Small fix

* Updated results

* Fix linting issues

* Added points

* Resolve conflict

* Update 610.jsonl

* Backfilled missing bibtex citations.

* Backfilled missing bibtex citations.

* Remove non-present files

* Wrote a script to scrape through the bibtex citations and create the corresponding bib file

* Added missed dependency

* Added functionality to create latex table

* Fixed linting issue

---------

Co-authored-by: Imene Kerboua <33312980+imenelydiakerusers.noreply.github.com>
Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com> ([`ab23552`](https://github.com/embeddings-benchmark/mteb/commit/ab235525d39402d9e1460eb7662d96928e055927))

Fix

* fix: Add LinceMT Bitext Mining (MINERS) (948)

* add new task

* add metadata

* update init

* Add results

* Update TaskMetadata.py ([`6bd165e`](https://github.com/embeddings-benchmark/mteb/commit/6bd165e95d1fe2cc623ffa7faa79aae8201d5dc8))

* fix: Add Phinc Bitext Mining (MINERS) (947)

* add phinc

* update metadata

* add files

* add points ([`0e71110`](https://github.com/embeddings-benchmark/mteb/commit/0e711102dbe2bdef4063099ba83af99fef16839c))

* fix: pair classification inconsistency (945)

* first go at fixing PairClassification

* added transformation for consistency

* added IndicXnliPairClassification

* added IndicXnliPairClassification 2

* lint fixes

* points added ([`6660f43`](https://github.com/embeddings-benchmark/mteb/commit/6660f432bd501eb2bbdd131fe61d697ef547e755))

Unknown

* Update tasks table ([`3ca687f`](https://github.com/embeddings-benchmark/mteb/commit/3ca687ffe7be3ae08970f163174e3053379f72d2))

* Update tasks table ([`4e3d868`](https://github.com/embeddings-benchmark/mteb/commit/4e3d868e501c5ed2e95e8f9dbd5331c1389f07e4))

* Update points table ([`cbe8458`](https://github.com/embeddings-benchmark/mteb/commit/cbe8458e8b3328c4e83b2e517404fab625f34824))

* Update points table ([`bc9c094`](https://github.com/embeddings-benchmark/mteb/commit/bc9c094dc445120eb0686e4f4f50e47054cc55d7))

Fix

* fix: Merge miracl evaluator (906)

* start merge

* removing redundancy

* removing MIRACLevaluator

* add linting

* clean up method names

* correct arg bug.

* remove type annotation

* combine unique texts

* improve readability

* merge main

* add back main changes

* adjust for task_name changes

* add back

* lint

---------

Co-authored-by: Jordan Example <jordan.clive19gmail.com> ([`8ab4c14`](https://github.com/embeddings-benchmark/mteb/commit/8ab4c141313d65a6f9e265a156894889e3d32565))

Unknown

* Added overview figure in SVG and PNG (939)

* Added overview figure in SVG and PNG

* Added wide version

* Added centered overview and fixed svg files

* Made orange color darker on figures ([`e53821e`](https://github.com/embeddings-benchmark/mteb/commit/e53821e598790a90a55088a8b743641497dd303b))

Fix

* fix: Add jmteb (938)

* fix: correct label for sib200

* fix: Add JaGovFaqs and NLPJournal datasets (808)

* Add JaGovFaqs dataset

* Add NLPJournal datasets

* Add JAQKET dataset

* Add points

* Fix metadata

* Remove title from corpus for JAQKET dataset

* Update JAQKET scores (without title)

* Exclude JAQKET dataset

* Add points for review

---------

Co-authored-by: Ashwin Mathur <97467100+awinmlusers.noreply.github.com> ([`f38c79b`](https://github.com/embeddings-benchmark/mteb/commit/f38c79b33eb3d306a557c56234b43601a4307ffc))

Unknown

* Update tasks table ([`980dfa8`](https://github.com/embeddings-benchmark/mteb/commit/980dfa874a9fcf2ee118187d48fc944a510c968a))

* Update points table ([`a104512`](https://github.com/embeddings-benchmark/mteb/commit/a10451249a2528ade57ad9610e08051be97957cc))

Fix

* fix: temporarily limit numpy version due to bug in datasets (940)

see https://github.com/huggingface/datasets/pull/6976 ([`0b0705e`](https://github.com/embeddings-benchmark/mteb/commit/0b0705e0b485d3236703b68408139ca14f286b53))

Unknown

* Update tasks table ([`077ca56`](https://github.com/embeddings-benchmark/mteb/commit/077ca569c298b019db84331a1fd3b7a4d9ea9b79))

* Update points table ([`c666597`](https://github.com/embeddings-benchmark/mteb/commit/c666597b808197808a80ffdc5d6e6a3044f7102f))

* Fix bugs (898)

* Remove usage of deprecated langs: self.langs to self.hf_subsets

* Correct link of WikipediaRetrievalMultilingual task

* multilingual/MLQARetrieval: deduplicate corpus

* multilingual/MLQARetrieval: de-duplicate questions

* multilingual/MLQARetrieval: Separate spaces for query-ids and context-ids

* Add points

* Update docs/mmteb/points/898.jsonl

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com> ([`4fa6e9d`](https://github.com/embeddings-benchmark/mteb/commit/4fa6e9de4a9a3313639f33b6e9556d8652fa2133))

Documentation

* docs: Update annotations for tasks (936)

* annottations

* annotate

* annotate

* annotate

* annotate

* ann

* ann

* ann

* ann

* ann

* formatted

* add data

* add data

* Update docs/mmteb/points/936.jsonl

* lint

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com> ([`8823369`](https://github.com/embeddings-benchmark/mteb/commit/8823369b91c82d7641367cfb0b9cdfec075143b1))

Fix

* fix: RAR-b initial PR (929)

* RARb initial PR

* RAR-b initial PR

* RAR-b initial PR

* RAR-b initial PR

* fill metadata

* add metadata n samples&char length

* taskdata subtask reasoning as retrieval

* fix formatting errors

* metadata description made descriptive ([`b75a9c9`](https://github.com/embeddings-benchmark/mteb/commit/b75a9c9fe6c976f12a9e6d7219901b9fedb99a69))

Unknown

* Update tasks table ([`a098a59`](https://github.com/embeddings-benchmark/mteb/commit/a098a5917d77e6986a82a568ec0a9de881b79b3e))

* Update tasks table ([`ea62028`](https://github.com/embeddings-benchmark/mteb/commit/ea62028c6e29001a11ee776b62530c879502866f))

* Update points table ([`2529093`](https://github.com/embeddings-benchmark/mteb/commit/25290938ca7855d4a7cf40ed38ae29635509d775))

Unknown

* Merge branch &39;main&39; of https://github.com/embeddings-benchmark/mteb ([`677dc93`](https://github.com/embeddings-benchmark/mteb/commit/677dc930b250a4db4b19fce46757926b5d98d390))