Rftokenizer

Compatibility update for torch 2.1. Now supports newer versions (for neural models for Hebrew):


scikit-learn==1.3.2
joblib==1.3.2
pandas==2.1.2
xgboost==2.0.3
flair==0.13.0

* add transformer based features

Compatibility update for sklearn 0.23 + joblib

* Added Arabic model
* Re-arranged folder setup for PyPI deployment

* Single file implementation, no classes for multilabel encoder etc.
* Switch classifier to xgboost
* Add interface for getting probabilities instead of predictions
* Add hyperparameter optimization code
* Various minor improvements