Workflow

Naive-tokenizers

Latest version: v0.0.1

Safety actively analyzes 681866 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

0.0.1

Contents:

* Add abstract interface for `Tokenizer`
* Wrapper `jieba` as `JiebaTokenizer`
* Add `TransformerTokenizer`
* Add `BertTokenizer`

Links

Releases

Has known vulnerabilities

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.