Scikit-learn-intelex

The release of **Intel® Extension for Scikit-learn 2023.1** introduces the following changes:

📚Support Materials

- [Accelerating Barnes-Hut t-SNE Algorithm by Efficient Parallelization on Multi-Core CPUs](https://arxiv.org/abs/2212.11506)

🛠️ Library Engineering

- Reduced the size of **Intel® oneDAL** library by approximately ~30%

🚨 What's New

- Introduced new functionality for **Intel® Extension for Scikit-learn**:
- Enabled PCA, Linear Regression, Random Forest algorithms and SPMD policy as preview
- Scikit-learn 1.2 support
- [sklearn_is_patched()](https://github.com/intel/scikit-learn-intelex/blob/rls/2023.1.0-rls/examples/sklearnex/patch_sklearn.py#L24) function added to validate status of algorithms patching
- Improved performance for the following **Intel® Extension for Scikit-learn** algorithms:
- t-SNE for “Burnes-Hut” algorithm
- SVM algorithm for single row inference

❗ Known Issues

- In certain conditions DAAL SYCL interface might hang with L0 backend – please use oneDAL DPC interfaces instead. If older interfaces are required OpenCL backend can be used as workaround.

The release of **Intel® Extension for Scikit-learn 2023.0.1** introduces the following changes:

🚨 What's New

- Performance improvements for tSNE algorithm https://github.com/intel/scikit-learn-intelex/commit/5275ebac37c416ce110634c6cee7b56f872ed71b
- Fixes for balanced classes and number of iterations in SVM https://github.com/intel/scikit-learn-intelex/commit/14849ee7190f5701e4eab2ad923fce9125e904ff, https://github.com/intel/scikit-learn-intelex/commit/4872a8ea0afa22813f1a4446ef5fc0d608660283, https://github.com/intel/scikit-learn-intelex/commit/9d0a05b80c09aa3930107e2eca233cd7b872593c
- Fix for `gamma` parameter in KMeans https://github.com/intel/scikit-learn-intelex/commit/1dca20c3761197d082d548f27f698d6389e60fbe

The release of **Intel® Extension for Scikit-learn 2023.0** introduces the following changes:

🚨 What's New

- Introduced new **Intel® oneDAL** functionality:
- DPC++ interface for Linear Regression algorithm

❗ Known Issues

- **Intel® Extension for Scikit-learn** SVC.fit and KNN.fit do not support GPU
- Most Intel® Extension for Scikit-learn sycl examples fail when using GPU context
- Running the Random Forest algorithm with versions 2021.7.1 and 2023.0 of scikit-learn-intelex on the 2nd *Generation Intel®* Xeon® *Scalable* Processors, formerly *Cascade Lake* may result in an 'Illegal instruction' error.
- No workaround is currently available for this issue.
- Recommendation: Use an older version of scikit-learn-intelex until the issue is fixed in a future release.

The release **Intel® Extension for Scikit-learn 2021.7.1** introduces the following changes:

📚 Support Materials
- [[Tabular Playground Series - Sep 2022] Tuning of ElasticNet hyperparameters](https://www.kaggle.com/code/alex97andreev/tuning-of-elasticnet-hyperparameters)
- [Accelerated Random Forest for Rent Prediction](https://www.kaggle.com/code/alex97andreev/accelerated-random-forest-for-rent-prediction)

🚨 What's New
- oneAPI interface for kNN regression
- Fix for wrong column names of pandas DataFrame in `sklearn.model_selection.train_test_split` patched function

The release **Intel® Extension for Scikit-learn 2021.6** introduces the following changes:

📚 Support Materials

Kaggle kernels:
- [Fast Feature Importance using scikit-learn-intelex](https://www.kaggle.com/lordozvlad/fast-feature-importance-using-scikit-learn-intelex)
- [[Tabular Playground Series - December 2021] Fast Feature Importance with sklearnex](https://www.kaggle.com/lordozvlad/tps-dec-fast-feature-importance-with-sklearnex?scriptVersionId=82254284)
- [[Tabular Playground Series - December 2021] SVC with sklearnex 20x speedup](https://www.kaggle.com/alexeykolobyanin/tps-dec-svc-with-sklearnex-20x-speedup)
- [[Tabular Playground Series - January 2022] Fast PyCaret with Scikit-learn-Intelex](https://www.kaggle.com/lordozvlad/tps-jan-fast-pycaret-with-scikit-learn-intelex)
- [[Tabular Playground Series - February 2022] KNN with sklearnex 13x speedup](https://www.kaggle.com/alexeykolobyanin/tps-feb-knn-with-sklearnex-13x-speedup)
- [Fast SVM for Sparse Data from NLP Problem](https://www.kaggle.com/alex97andreev/fast-svm-for-sparse-data-from-nlp-problem)
- [Introduction to scikit-learn-intelex](https://www.kaggle.com/lordozvlad/introduction-to-scikit-learn-intelex)
- [[Datasets] Fast Feature Importance using sklearnex](https://www.kaggle.com/lordozvlad/datasets-fast-feature-importance-using-sklearnex)
- [[Tabular Playground Series - March 2022] Fast workflow using scikit-learn-intelex](https://www.kaggle.com/lordozvlad/tps-mar-fast-workflow-using-scikit-learn-intelex)

🛠️ Library Engineering

- Reduced the size of oneDAL python run-time package by approximately 8%
- Added Python 3.10 support for daal4py and Intel(R) Extension for Scikit-learn packages

🚨 What's new

- Improved performance for the following Intel® Extension for Scikit-learn algorithms:

- t-SNE for “Burnes-Hut” algorithm

- Introduced new functionality for Intel® Extension for Scikit-learn:

- Manhattan, Minkowski, Chebyshev and Cosine distances for KNeighborsClassifier and NearestNeighbors with “brute” algorithm

- Fixed the following issues in Intel® Extension for Scikit-learn:

- An issue with the search of common data type in pandas DataFrame
- Patching overhead of finiteness checker for specific small data sizes
- Incorrect values in a tree visualization with `plot_tree` function in RandomForestClassifier
- Unexpected error for device strings in `{device}:{device_index}` format while using config context

The release **Intel® Extension for Scikit-learn 2021.5** introduces the following changes:

📚 Support Materials
- Kaggle kernels:
- [Tabular Playground Series - Sep 2021] [Ridge with sklearn-intelex 2x speedup](https://www.kaggle.com/alexeykolobyanin/tps-sep-ridge-with-sklearn-intelex-2x-speedup)
- [Tabular Playground Series - Oct 2021] [Fast AutoML with Intel Extension for Scikit-learn](https://www.kaggle.com/lordozvlad/fast-automl-with-intel-extension-for-scikit-learn/notebook)
- [Titanic – Machine Learning from Disaster] [AutoML with Intel Extension for Sklearn](https://www.kaggle.com/lordozvlad/titanic-automl-with-intel-extension-for-sklearn/notebook)
- [Tabular Playground Series - Nov 2021] [AutoML with Intel® Extension](https://www.kaggle.com/lordozvlad/tps-nov-automl-with-intel-extension)
- [Tabular Playground Series - Nov 2021] [Log Regression with sklearnex 17x speedup](https://www.kaggle.com/alexeykolobyanin/tps-nov-log-regression-with-sklearnex-17x-speedup)
- [Tabular Playground Series - Dec 2021] [SVC with sklearnex 20x speedup](https://www.kaggle.com/alexeykolobyanin/tps-dec-svc-with-sklearnex-20x-speedup)
- [Tabular Playground Series - Dec 2021] [Fast Feature Importance with sklearnex](https://www.kaggle.com/lordozvlad/tps-dec-fast-feature-importance-with-sklearnex)
- Added [demo samples](https://github.com/intel/scikit-learn-intelex/tree/rls/2021.5-rls/examples/notebooks) of the Intel® Extension for Scikit-learn usage with the performance comparison to original Scikit-learn for ElasticNet, K-means, Lasso Regression, Linear regression, and Ridge Regression
- Added [demo samples](https://github.com/intel/scikit-learn-intelex/tree/rls/2021.5-rls/examples/notebooks/NYCTaxi-E2E-Demo) of the Modin usage

🛠️ Library Engineering
- Reduced the size of oneDAL library by approximately ~15%, this is a required dependency of Intel® extension for scikit learn.

🚨 New Features
- Scikit-learn 1.0 support

🚀 ​Improved performance
- [GPU] `KNN` algorithm prediction
- [GPU] `SVC` and `SVR` algorithms training

🐛 Bug Fixes
- Stabilized the results of Linear Regression in oneDAL and Intel® Extension for Scikit-learn
- Fixed an issue with RPATH on MacOS