- **Potentially breaking change:** The somajo-tokenizer script is
automatically created upon installation and bin/somajo-tokenizer is
removed. For most users, this does not make a difference. If you
used to run your own modified version of SoMaJo directly via
bin/somajo-tokenizer, consider installing the project in editable
mode (see Development section in README.md).
- Switch from setup.py to pyconfig.toml and restructure the project
(source in src, tests in tests).
- When creating a Token object, only known token classes can be
passed.
- Fix issue 25 (dates at the end of sentences)