Thainlp

- Clean word lists in `pythainlp.corpus` (remove duplicates, etc.)
- Fix/add return type hinting for functions in `pythainlp.corpus`
- Fix deprecated inline flag for regular expression in `pythainlp.corpus.tnc` (Thai National Corpus)
- Bug fix: reorder condition checks in `pythainlp.tokenize.dict_trie` so it catch `Trie` before `Iterable`

- `word_tokenize()`'s argument `whitespaces` is now `keep_whitespace` to make is more explicit, default behavior is to keep whitespaces
- `word_tokenize()` can now take a custom dictionary throught `custom_dict` parameter
- `dict_word_tokenize()` will be deprecated soon

- Fix TCC (Thai Textbook Corpus) corpus always downloading new file issue
- Words and their frequencies from TTC (Thai Textbook Corpus) now has a local copy at `ttc_freq.txt` inside `pythainlp.corpus`.
- Other refactoring and code improvements, including ones related to subword tokenization (Thai Character Cluster / TCC and ETCC), see 193

- Fixed tree map
- Subword tokeniser documentation improvement https://github.com/PyThaiNLP/pythainlp/pull/190

- Add Tokenizer from pythainlp.tokenize.Tokenizer 79432c2
- NER fixes, code cleaning, and type hinting 186

- Consolidate documentation files
- Thai2fit (Upgrade ULMFiT-related codes to fastai 1.0)
- Remove Python 2 compatibility code
- Remove temporary files, experiment files, and obsoleted files
- Remove sentiment analysis
- More consistent indentations in source code
- Improved word_tokenize (newmm, mm) and dict_word_tokenize
- Improved POS-tagging
- More and improved examples
- Improved test coverages with more test case

More details https://github.com/PyThaiNLP/pythainlp/issues/118

Install


pip install https://github.com/PyThaiNLP/pythainlp/archive/2.0b.zip


Docs : https://thainlp.org/pythainlp/docs/2.0/index.html

Website : https://pythainlp.github.io/

GitHub : https://github.com/PyThaiNLP/pythainlp

Issues : https://github.com/PyThaiNLP/pythainlp/issues

Thank you for choosing us.

PyThaiNLP team