Ai21

Chore

* chore(release): v0.11.1 [skip ci] ([`07bf0cd`](https://github.com/AI21Labs/ai21-tokenizer/commit/07bf0cd237d566a43fac94513ce30d8d20668fd7))

Fix

* fix: Added backwards compatibility for jamba tokenizer (88)

* fix: Added backwards compatibility for jamba tokenizer

* fix: Added to async

---------

Co-authored-by: asafg <asafgai21.com> ([`9ad3475`](https://github.com/AI21Labs/ai21-tokenizer/commit/9ad34759295dc8ccde30fe273c508b617ea81175))

Chore

* chore(release): v0.11.0 [skip ci] ([`0df321c`](https://github.com/AI21Labs/ai21-tokenizer/commit/0df321c8491404c595f9a30cc413ccebdcf24551))

Feature

* feat: Async tokenizer (86)

* feat: support async, wip

* feat: fix and add tests, examples, update readme

* fix: poetry lock

* fix: anyio -> aiofiles

* fix: try 3.8

* fix: remove 3.7 from tests

* fix: poetry lock

* fix: add 3.7 back

* fix: poetry lock

* fix: poetry.lock

* ci: pipenv

* fix: pipenv

* fix: pipenv

* fix: pyproject

* fix: lock

* fix: version

* fix: Removed aiofiles

* ci: update python version,

* fix: switch from aiofiles to anyio, remove redundant comments

* chore: poetry lock

* fix: disable initializing async classes directly, cr comments

* test: fix import

* ci: add asyncio-mode to test workflow

* fix: to_thread -> run_in_executor

* ci: add asyncio

* fix: cr comments

* fix: cr comments

---------

Co-authored-by: asafg <asafgai21.com> ([`3006cda`](https://github.com/AI21Labs/ai21-tokenizer/commit/3006cda2305a5ad31d705d3f6e3fe5605820e2f5))

Chore

* chore(release): v0.10.0 [skip ci] ([`1178ba7`](https://github.com/AI21Labs/ai21-tokenizer/commit/1178ba7fd1d980681cf143292848dacd51cdc2d5))

Feature

* feat: remove python 3.7 support (87) ([`58482ab`](https://github.com/AI21Labs/ai21-tokenizer/commit/58482abbda7767055345274b38c58ba9fd33b42f))

Chore

* chore(release): v0.9.1 [skip ci] ([`a260d76`](https://github.com/AI21Labs/ai21-tokenizer/commit/a260d76aece9992f03ac4f004edf6a6dd83f5f13))

Fix

* fix: depend on less restrictive version of tokenizers (85) ([`eab6a14`](https://github.com/AI21Labs/ai21-tokenizer/commit/eab6a14a6a788efbff708df0cded59d2392ef7f2))

Chore

* chore(release): v0.9.0 [skip ci] ([`974df9e`](https://github.com/AI21Labs/ai21-tokenizer/commit/974df9e8e9c46f5545dfb1a2603d51bac0eab750))

Feature

* feat: Jamba instruct tokenizer (84) ([`88ff9af`](https://github.com/AI21Labs/ai21-tokenizer/commit/88ff9aff504caa8928d68bcde4430c24fbbc23f1))

Chore

* chore(release): v0.8.2 [skip ci] ([`1146741`](https://github.com/AI21Labs/ai21-tokenizer/commit/11467416dd263b824f6a8711983eb5588fb037dc))

Ci

* ci: add Python 3.12 to test matrix (82)

* ci: add Python 3.12 to test matrix

* chore: use sentencepiece 0.2.0 or higher

* fix: update poetry.lock ([`8084117`](https://github.com/AI21Labs/ai21-tokenizer/commit/8084117c74813a99b79ecefd12888817470e1838))

Fix

* fix: docs (83) ([`c26949a`](https://github.com/AI21Labs/ai21-tokenizer/commit/c26949a62d5e612a7ff8132c6e6896b263be7b28))

Unknown

* Update issue templates ([`86ea6e7`](https://github.com/AI21Labs/ai21-tokenizer/commit/86ea6e79a5670c0e8049ac587ed1b5f4b8790ae9))