Insurautoml

New Features

Previously, the model selection and hyperparameter optimization is achieved by `HyperOpt`, and now, the automation is achieved by `ray.tune`, which contains more features, plentiful interaction between different packages and better visualization for training process.

However, the HyperOpt version is still workable through legacy archives. The hyperopt pipelines can be accessed through:
python
from My_AutoML._legacy import AutoTabular, AutoTabularClassifier, AutoTabularRegressor


For the model. due to conflicting hyperparameter space construction process, which described in 1 , leads to a re-construction of entire default hyperparameter space. Now, both grid search/random search and hyperopt search are available for model selection and hyperparameter optimization.

Key features for `ray.tune` optimization procedure:

1. search algorithms

> RandomSearch, GridSearch, BayesOptSearch, AxSearch, BOHB, BlendSearch, CFO, DragonflySearch, HEBO, HyperOpt, Nevergrad, Optuna, SigOpt, Scikit-Optimize, ZOOpt, Reapter, ConcurrencyLimiter

2. search scheduler

> FIFOScheduler, ASHAScheduler, HyperBandScheduler, MedianStoppingRule, PopulationBasedTraining, PopulationBasedTrainingReplay, PB2, HyperBandForBOHB

3. progress reporter

> CLIReporter, JupyterNotebookReporter

Smaller Changes

1. Basic support for `ray.tune` using Grid Search. All temp workings stored in temp folders. 979a44040cba4489b57c44d7a6885ac869be30a7

2. add few more classification/regression models from sklearn 3

> List of new models:
> 1. LinearRegression
> 2. Lasso
> 3. Ridge
> 4. ElasticNet
> 5. BayesianRidge
> 6. LogisticRegression
> 7. ComplementNB
> 8. HistGradientBoostingClassifier
> 9. HistGradientBoostingRegressor

3. change the logic of legacy structure, so legacy can still work when later changes (even structure level changes) come. 960c5f0a96bc869364ced58bd0b793c48ec25560

Future Plan

1. Use MLP for Tabular classification/regression, RNN structure for text processing (partial code ready)

2. More imputation methods/ balancing methods/ scaling methods/ feature selection/ regression, classification models to evaluation

AutoML pipeline

The pipeline is targeted as a AutoML for tabular regression/classification tasks.

Basic workable pipeline insists of a pipeline: encoding, imputation, balancing, scaling, feature selection, regression/classification models.

The pipeline can achieve automated Model Selection and Hyperparameter Optimization by HyperOpt.

Current methods in pipeline (some methods are deprecated and not displayed below):

1. Encoding

> 1 DataEncoding

2. Imputation

> SimpleImputer, JointImputer, ExpectationMaximization, KNNImputer, KNNImputer, MissForestImputer, MICE, GAIN

3. Balancing

> SimpleRandomOverSampling SimpleRandomUnderSampling TomekLink EditedNearestNeighbor CondensedNearestNeighbor OneSidedSelection CNN_TomekLink Smote Smote_TomekLink Smote_ENN

4. Scaling

> MinMaxScale Standardize Normalize RobustScale PowerTransformer QuantileTransformer Winsorization

5. Feature Selection

> RBFSampler FeatureFilter ASFFS GeneticAlgorithm extra_trees_preproc_for_classification/ extra_trees_preproc_for_regression liblinear_svc_preprocessor polynomial select_percentile_classification/ select_percentile_regression select_rates_classification/ select_rates_regression truncatedSVD

6. Regression

> AdaboostRegressor ARDRegression DecisionTree ExtraTreesRegressor GaussianProcess GradientBoosting KNearestNeighborsRegressor LibLinear_SVR LibSVM_SVR MLPRegressor RandomForest SGD

7. Classification

> AdaboostClassifier BernoulliNB DecisionTree ExtraTreesClassifier GaussianNB GradientBoostingClassifier NearestNeighborsClassifier LDA LibLinear_SVC LibSVM_SVC MLPClassifier MultinomialNB PassiveAggressive QDA RandomForest SGD

Working

1. Use ray to tune the pipeline (workable, still work on outputs)

2. Use MLP for Tabular classification/regression, RNN structure for text processing (partial code ready)