- UrlStore class added: data store containing URLs with relevant information - code cleaning and maintenance (bugs, simplification)
0.6.0
- reviewed code base: simplicity and execution speed - dropped support for Python 3.5
0.5.0
- more complex language heuristics, use langcodes - extended blacklists and whitelists - more precise filters and more efficient code - support for Python 3.10
0.4.2
- enhanced cleaning - fixed language filter
0.4.1
- keep trailing slashes to avoid redirection - fixes: normalization and crawlable URLs
0.4.0
- URL manipulation tools added: extract parts, fix relative URLs - filters added: language, navigation and crawls - more robust link handling and extraction - removed support for Python 3.4