Chore
* chore(release): v0.2.0 [skip ci] ([`8988faa`](https://github.com/AI21Labs/ai21-tokenizer/commit/8988faa1419069916aedcb317e7c813a57a7d03d))
* chore(deps-dev): bump black from 22.12.0 to 23.3.0 (32)
Bumps [black](https://github.com/psf/black) from 22.12.0 to 23.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/22.12.0...23.3.0)
---
updated-dependencies:
- dependency-name: black
dependency-type: direct:development
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <supportgithub.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]users.noreply.github.com>
Co-authored-by: asafgardin <147075902+asafgardinusers.noreply.github.com> ([`bb4986e`](https://github.com/AI21Labs/ai21-tokenizer/commit/bb4986e880f24eebc7be7eee050d85c4d56a1aa9))
Feature
* feat: Tokenizer factory (31)
* feat: Added tokenizer abc and factory
* fix: api to receive default and none
* fix: example
* fix: factory and tests
* fix: rename base
* fix: rename base class
* fix: rename package
* fix: example
* fix: readme and tasks
* docs: factory class
* docs: renames
* fix: directory hierarchy in tests
* fix: rename package
* chore(release): v0.1.2 [skip ci]
* fix: rename package
* ci: example
* fix: assert in example
* fix: src_path
---------
Co-authored-by: github-actions <github-actionsgithub.com> ([`e55cd1d`](https://github.com/AI21Labs/ai21-tokenizer/commit/e55cd1dac5ad501a0a48c369a28d061f37950f5f))
Fix
* fix: token name (34) ([`2b229b2`](https://github.com/AI21Labs/ai21-tokenizer/commit/2b229b28ace8ac72dcc9bc727d187350020a2e12))