Totokenizers

Latest version: v1.5.4

Safety actively analyzes 687918 Python packages for vulnerabilities to keep your Python projects secure.

1.5.2

What's Changed
* Fix tool toke count if message has 'tool_calls' in it by jojo0severo in https://github.com/TeiaLabs/totokenizers/pull/18

**Full Changelog**: https://github.com/TeiaLabs/totokenizers/compare/1.5.1...1.5.2

1.5.1

What's Changed
* Tolls tokens counting by jojo0severo in https://github.com/TeiaLabs/totokenizers/pull/16
* Use default token values when counting by jojo0severo in https://github.com/TeiaLabs/totokenizers/pull/17

**Full Changelog**: https://github.com/TeiaLabs/totokenizers/compare/v1.4.0...1.5.1

1.4.0

Releases

Has known vulnerabilities

1.5.2
1.5.1
1.4.0