Mteb

Latest version: v1.20.0

Safety actively analyzes 682532 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 10 of 58

1.14.10

Fix

* fix: add multilingual mocks and test descriptive_stats (1173)

* add multilingual mocks and test descriptive_stats

* fix task

* lint

* remove comment ([`f941aa0`](https://github.com/embeddings-benchmark/mteb/commit/f941aa0c6b405e4b377c74487271fb927bd4a05b))

* fix: Remove unavailable test split from OCNLI (1175)

fix ocnli ([`b21223a`](https://github.com/embeddings-benchmark/mteb/commit/b21223a39baa1d403fe6f6ebe1ef2fef9e15882f))

Unknown

* Update tasks table ([`5ddff62`](https://github.com/embeddings-benchmark/mteb/commit/5ddff6230d58f92d9a1cb2db7bea16d2236b8476))

1.14.9

Fix

* fix: Added multilingual and indic benchmark

This adds outlines for the multilingual and indic benchmarks as well as a few adjustments to existing benchmarks due to a few discovered bugs in the previous implementation.

It also add utilites for easily creating a table from a list of tasks:


tasks = mteb.get_tasks(tasks=[...])
tasks.to_latex()
tasks.to_dataframe()
([`8c9dada`](https://github.com/embeddings-benchmark/mteb/commit/8c9dadac6b23dc5f05656cc3284109f7d79e03b7))

Unknown

* Update tasks table ([`4699b43`](https://github.com/embeddings-benchmark/mteb/commit/4699b43a232fcdea096be08c899ce7f6479f4914))

1.14.8

Fix

* fix: Added contruction of MTEB(eng, v2) and its lite version (1168)

* Added metadata

* Added EU benchmark given current tasks

* Updated task selection and benchmark

* Added csv

* Added benchmark scores

* Added evaluation time

* format

* fixed baed on corrections

* fix: Added construction of MTEB(eng, v2) and MTEB(eng, v2 lite)

* Added v2 lite comparison with v1

* format

* format

* reformulate to not introduce mtebv2 ([`e861ef2`](https://github.com/embeddings-benchmark/mteb/commit/e861ef24524858800f96012edc887fbd4e67750a))

1.14.7

Fix

* fix: Added task selection for the EU benchmark (1166)

* Added metadata

* Added EU benchmark given current tasks

* Updated task selection and benchmark

* Added csv

* Added benchmark scores

* Added evaluation time

* format

* fixed baed on corrections

* Added romani

* format ([`0f4f2df`](https://github.com/embeddings-benchmark/mteb/commit/0f4f2dff35381632097ded347ffe5b4b8a767b76))

Unknown

* Update tasks table ([`c220741`](https://github.com/embeddings-benchmark/mteb/commit/c2207410c3e6dd5a161d89808404b1a50a6d1d9b))

1.14.6

Fix

* fix: Add function for metadata create (1167)

* init calculate metadata

* add bitextmetadata

* add classification

* add calculate metrics

* add docs

* fix speed

* fix dummy task

* add typed dict

* update stats

* Apply suggestions from code review

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* Update mteb/abstasks/AbsTaskClassification.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com>

* fix docstrings

* add overall

* update stats

* fix retrieval dict

* change to typeddict

* rename lang to hf_subset

* update task table

---------

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsengmail.com> ([`56382b5`](https://github.com/embeddings-benchmark/mteb/commit/56382b53ef33cd747b3fc65aa0d517480c52ef17))

1.14.5

Fix

* fix: Merge pr 1128 (1169)

* Create arctic_models.py

* Update run_mteb_english_cluster.py

* updates

---------

Co-authored-by: Daniel Campos <daniel.campossnowflake.com> ([`04dbcbb`](https://github.com/embeddings-benchmark/mteb/commit/04dbcbb8535cb81bde1d86ec0e05c0fa865a38d5))

* fix: Update COIR datasets (1159)

* integrate mteb

* fix bug

* update monolingual tasks for coir

* fix bugs

* update

* update

* fix

* remove title

* rm dates

* fix typo

* fix formatting

* edit metadata

* fix

* update new paths

* update revision

* fix

* update

* fix revision

* fix metadata

* add coir codesearchnet

* fix metclass error

* fix metaclass error

* fix hf commit

---------

Co-authored-by: l00856372 <lee.yi.quanhuawei.com> ([`c5bd338`](https://github.com/embeddings-benchmark/mteb/commit/c5bd3381f2efdc985d1538038c50300d733d1923))

Unknown

* Update tasks table ([`ecdc240`](https://github.com/embeddings-benchmark/mteb/commit/ecdc240d7542d18229e05d3e1f60d31a812a0494))

* Format recent additions to the codebase ([`bd2db98`](https://github.com/embeddings-benchmark/mteb/commit/bd2db98cbacc65b2a97a8cf56ab977d19666be44))

Page 10 of 58

Links

Releases

Has known vulnerabilities

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.