Mteb

Fix

* fix: Ensure result are consistently stored in the same way (876)

* Ensure result are consistently stored in the same way

- (due to failing test): updated missing dataset references
- (to test with more than one model) Added e5 models base and large
- updated mteb.get_model to now include metadata in the model object
- ensure that model name is always included when saving (with a default when it is not available)
- use the ModelMeta for the model_meta.json

* format

* minor test fixes

* docs: Minor updated to repro. workflow docs

* fixed failing test

* format

* Apply suggestions from code review

Co-authored-by: Isaac Chung <chungisaac1217gmail.com>

* docs: update PR template

* fix: Added benchmark object (878)

* removed duplicate task

* Added benchmark object

* removed import for duplicate task

* fix dataset references

* added seb

* Added test for running benchmarks

* changed tasks to be an iterable

* format

* Apply suggestions from code review

Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com>
Co-authored-by: Isaac Chung <chungisaac1217gmail.com>

---------

Co-authored-by: Isaac Chung <chungisaac1217gmail.com>
Co-authored-by: Niklas Muennighoff <n.muennighoffgmail.com> ([`fb843d0`](https://github.com/embeddings-benchmark/mteb/commit/fb843d040e8af63b4d0c2d61b78a69ca55652bcd))

Unknown

* Update tasks table ([`dfbdfdc`](https://github.com/embeddings-benchmark/mteb/commit/dfbdfdce8423dddb9821ffa349a7ee1df8bdb2ca))

Fix

* fix: Add FaithDialRetrieval dataset (874)

* faithdial dataset

* results

* added metadata

* add points

* Apply suggestions from code review

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* Add reviewer points

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* only test set

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com> ([`582381b`](https://github.com/embeddings-benchmark/mteb/commit/582381bd75ec111ba1ed8c81dc2df21336091546))

Unknown

* Update tasks table ([`35dcec7`](https://github.com/embeddings-benchmark/mteb/commit/35dcec7358ee5038c968545976a2abee007b2713))

* Update points table ([`527c5eb`](https://github.com/embeddings-benchmark/mteb/commit/527c5eb718c7b8b5f82972a9f4eae4f32460d2ae))

Fix

* fix: Add feedbackQA dataset (856)

* add feedbackQA
* Update mteb/tasks/Retrieval/eng/FeedbackQARetrieval.py
Co-authored-by: Xing Han Lu <21180505+xhlucausers.noreply.github.com>
* Update mteb/tasks/Retrieval/eng/FeedbackQARetrieval.py
Co-authored-by: Xing Han Lu <21180505+xhlucausers.noreply.github.com>
* Update mteb/tasks/Retrieval/eng/FeedbackQARetrieval.py
Co-authored-by: Xing Han Lu <21180505+xhlucausers.noreply.github.com>
* add feedbackQA
* add feedbackQA
* Update mteb/tasks/Retrieval/eng/FeedbackQARetrieval.py
Co-authored-by: Xing Han Lu <21180505+xhlucausers.noreply.github.com>
* add feedbackQA
* points
* make lint
* typo
* missing datasets
---------
Co-authored-by: Xing Han Lu <21180505+xhlucausers.noreply.github.com>
Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`0796efa`](https://github.com/embeddings-benchmark/mteb/commit/0796efa4f987fa65920b3853455c1e592ca6b697))

Unknown

* Update tasks table ([`24e3d92`](https://github.com/embeddings-benchmark/mteb/commit/24e3d92f7a16a56a24fb8efe6feadce15d319aee))

* Update points table ([`8b52f03`](https://github.com/embeddings-benchmark/mteb/commit/8b52f0342ada6a8e81f1867bf70d50f8740fde03))

Documentation

* docs: Affiliation modification (871)

modified affiliation

Co-authored-by: Isaac Chung <chungisaac1217gmail.com> ([`6e43bbf`](https://github.com/embeddings-benchmark/mteb/commit/6e43bbf389b8c71e1d15b50054a0183789d6c580))

Fix

* fix: Add MIRACL reranking (830)

* fix: MIRACL reranking (641)

* miracl reranking

* clean

* reranking eval based on pytrec

* eval miracl-reranking using pytrec

* miracl reranking

* dataset name

* compute_metrics_batched

* dataset name typo

* clean

* make lint

* points

* Update mteb/evaluation/evaluators/RerankingEvaluator.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* Update mteb/evaluation/evaluators/RerankingEvaluator.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* Update mteb/evaluation/evaluators/RerankingEvaluator.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* Update mteb/tasks/Reranking/multilingual/MIRACLReranking.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* solve part of PR comments

* docstring

* n-samples and avg_char_length

* make lint

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* Addressed changes from main PR

* remove unused import

* removed abstract from citation

* added *(MIRACL) to results

* fix: Miracl reranking fix (870)

* change evaluate and evaluate_custom back to staticmethod

* using MIRACLRerankingEvaluator rather than RerankingEvaluator; adapt to latest RetrievalEvaluator

* formatted

* fix: fixed missing references

---------

Co-authored-by: Crystina Xinyu Zhang <x978zhanuwaterloo.ca>
Co-authored-by: Your Name <youexample.com> ([`4218675`](https://github.com/embeddings-benchmark/mteb/commit/421867588007e7664a1584e3f273d2978b3bade1))

Unknown

* Update tasks table ([`2291242`](https://github.com/embeddings-benchmark/mteb/commit/2291242b8c3e99f84826adf43a427e9e42dd8f4e))

* Update points table ([`77badee`](https://github.com/embeddings-benchmark/mteb/commit/77badee59c439d537486317fd490ed801c441108))

Fix

* fix: dataset not available (872)

fix dataset not available ([`397519b`](https://github.com/embeddings-benchmark/mteb/commit/397519b4aea562dc2912d49874ec819550f2e28f))

Fix

* fix: main score for longEmbed (869)

* Fix main score

* Fix main score ([`db8bb5e`](https://github.com/embeddings-benchmark/mteb/commit/db8bb5ecaa6f5038cb090aba224461ab20015741))