Autotiktokenizer

Latest version: v0.2.1

Safety actively analyzes 685525 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

0.2.1

What's Changed
* Add support for WordPiece Tokenizers + Aya Family by bhavnicksm in https://github.com/bhavnicksm/autotiktokenizer/pull/9


**Full Changelog**: https://github.com/bhavnicksm/autotiktokenizer/compare/v0.2.0...v0.2.1

0.2.0

What's Changed
* Remove tokenizers dependency from AutoTikTokenizers by bhavnicksm in https://github.com/bhavnicksm/autotiktokenizer/pull/8


**Full Changelog**: https://github.com/bhavnicksm/autotiktokenizer/compare/v0.1.2...v0.2.0

0.1.2

What's Changed
* Update README.md by bhavnicksm in https://github.com/bhavnicksm/autotiktokenizer/pull/5
* Update README.md by bhavnicksm in https://github.com/bhavnicksm/autotiktokenizer/pull/6
* Bump version to 0.1.2 and update package metadata by bhavnicksm in https://github.com/bhavnicksm/autotiktokenizer/pull/7


**Full Changelog**: https://github.com/bhavnicksm/autotiktokenizer/compare/v0.1.1...v0.1.2

0.1.1

What's Changed
* Bump version to 0.1.1 and enhance _AutoTikTokenizer functionality by bhavnicksm in https://github.com/bhavnicksm/autotiktokenizer/pull/4


**Full Changelog**: https://github.com/bhavnicksm/autotiktokenizer/compare/v0.1.0...v0.1.1

0.1.0

What's Changed
* Add logo + Bump version to v0.1.0 by bhavnicksm in https://github.com/bhavnicksm/autotiktokenizer/pull/3


**Full Changelog**: https://github.com/bhavnicksm/autotiktokenizer/compare/v0.0.1a0...v0.1.0

0.0.1a0

What's Changed
* Initial Commit by bhavnicksm in https://github.com/bhavnicksm/autotiktokenizer/pull/1
* Add version and author metadata to AutoTikTokenizer by bhavnicksm in https://github.com/bhavnicksm/autotiktokenizer/pull/2

New Contributors
* bhavnicksm made their first contribution in https://github.com/bhavnicksm/autotiktokenizer/pull/1

**Full Changelog**: https://github.com/bhavnicksm/autotiktokenizer/commits/v0.0.1a0

Links

Releases

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.