Mlxtend

New Features

- New function implementing the resampled paired t-test procedure (`paired_ttest_resampled`)
to compare the performance of two models
(also called k-hold-out paired t-test). ([323](https://github.com/rasbt/mlxtend/issues/323))
- New function implementing the k-fold paired t-test procedure (`paired_ttest_kfold_cv`)
to compare the performance of two models
(also called k-hold-out paired t-test). ([324](https://github.com/rasbt/mlxtend/issues/324))
- New function implementing the 5x2cv paired t-test procedure (`paired_ttest_5x2cv`) proposed by Dieterrich (1998)
to compare the performance of two models. ([325](https://github.com/rasbt/mlxtend/issues/325))
- A `refit` parameter was added to stacking classes (similar to the `refit` parameter in the `EnsembleVoteClassifier`), to support classifiers and regressors that follow the scikit-learn API but are not compatible with scikit-learn's `clone` function. ([325](https://github.com/rasbt/mlxtend/issues/324))
- The `ColumnSelector` now has a `drop_axis` argument to use it in pipelines with `CountVectorizers`. ([333](https://github.com/rasbt/mlxtend/pull/333))

Changes


- Raises an informative error message if `predict` or `predict_meta_features` is called prior to calling the `fit` method in `StackingRegressor` and `StackingCVRegressor`. ([315](https://github.com/rasbt/mlxtend/issues/315))
- The `plot_decision_regions` function now automatically determines the optimal setting based on the feature dimensions and supports anti-aliasing. The old `res` parameter has been deprecated. ([309](https://github.com/rasbt/mlxtend/pull/309) by [Guillaume Poirier-Morency](https://github.com/arteymix))
- Apriori code is faster due to optimization in `onehot transformation` and the amount of candidates generated by the `apriori` algorithm. ([327](https://github.com/rasbt/mlxtend/pull/327) by [Jakub Smid](https://github.com/jaksmid))
- The `OnehotTransactions` class (which is typically often used in combination with the `apriori` function for association rule mining) is now more memory efficient as it uses boolean arrays instead of integer arrays. In addition, the `OnehotTransactions` class can be now be provided with `sparse` argument to generate sparse representations of the `onehot` matrix to further improve memory efficiency. ([328](https://github.com/rasbt/mlxtend/pull/328) by [Jakub Smid](https://github.com/jaksmid))
- The `OneHotTransactions` has been deprecated and replaced by the `TransactionEncoder`. ([332](https://github.com/rasbt/mlxtend/pull/332)
- The `plot_decision_regions` function now has three new parameters, `scatter_kwargs`, `contourf_kwargs`, and `scatter_highlight_kwargs`, that can be used to modify the plotting style. ([342](https://github.com/rasbt/mlxtend/pull/342) by [James Bourbeau](https://github.com/jrbourbeau))


Bug Fixes

- Fixed issue when class labels were provided to the `EnsembleVoteClassifier` when `refit` was set to `false`. ([322](https://github.com/rasbt/mlxtend/issues/322))
- Allow arrays with 16-bit and 32-bit precision in `plot_decision_regions` function. ([337](https://github.com/rasbt/mlxtend/issues/337))
- Fixed bug that raised an indexing error if the number of items was <= 1 when computing association rules using the conviction metric. ([340](https://github.com/rasbt/mlxtend/issues/340))

New Features

- New `store_train_meta_features` parameter for `fit` in StackingCVRegressor. if True, train meta-features are stored in `self.train_meta_features_`.
New `pred_meta_features` method for `StackingCVRegressor`. People can get test meta-features using this method. ([294](https://github.com/rasbt/mlxtend/pull/294) via [takashioya](https://github.com/takashioya))
- The new `store_train_meta_features` attribute and `pred_meta_features` method for the `StackingCVRegressor` were also added to the `StackingRegressor`, `StackingClassifier`, and `StackingCVClassifier` ([299](https://github.com/rasbt/mlxtend/pull/299) & [#300](https://github.com/rasbt/mlxtend/pull/300))
- New function (`evaluate.mcnemar_tables`) for creating multiple 2x2 contigency from model predictions arrays that can be used in multiple McNemar (post-hoc) tests or Cochran's Q or F tests, etc. ([307](https://github.com/rasbt/mlxtend/issues/307))
- New function (`evaluate.cochrans_q`) for performing Cochran's Q test to compare the accuracy of multiple classifiers. ([310](https://github.com/rasbt/mlxtend/issues/310))

Changes

- Added `requirements.txt` to `setup.py`. ([304](https://github.com/rasbt/mlxtend/issues/304) via [Colin Carrol](https://github.com/ColCarroll))


Bug Fixes

- Improved numerical stability for p-values computed via the the exact McNemar test ([306](https://github.com/rasbt/mlxtend/issues/306))
- `nose` is not required to use the library ([302](https://github.com/rasbt/mlxtend/issues/302))

v.0.9.1

Downloads

- [Source code (zip)](https://github.com/rasbt/mlxtend/archive/v0.9.1.zip)
- [Source code (tar.gz)](https://github.com/rasbt/mlxtend/archive/v0.9.1.tar.gz)

New Features

- Added `mlxtend.evaluate.bootstrap_point632_score` to evaluate the performance of estimators using the .632 bootstrap. ([283](https://github.com/rasbt/mlxtend/pull/283))
- New `max_len` parameter for the frequent itemset generation via the `apriori` function to allow for early stopping. ([270](https://github.com/rasbt/mlxtend/pull/270))

Changes

- All feature index tuples in `SequentialFeatureSelector` or now in sorted order. ([262](https://github.com/rasbt/mlxtend/pull/262))
- The `SequentialFeatureSelector` now runs the continuation of the floating inclusion/exclusion as described in Novovicova & Kittler (1994).
Note that this didn't cause any difference in performance on any of the test scenarios but could lead to better performance in certain edge cases.
([262](https://github.com/rasbt/mlxtend/pull/262))
- `utils.Counter` now accepts a name variable to help distinguish between multiple counters, time precision can be set with the 'precision' kwarg and the new attribute end_time holds the time the last iteration completed. ([278](https://github.com/rasbt/mlxtend/pull/278) via [Mathew Savage](https://github.com/matsavage))


Bug Fixes

- Fixed an deprecation error that occured with McNemar test when using SciPy 1.0. ([283](https://github.com/rasbt/mlxtend/pull/283))

New Features

- Added `evaluate.permutation_test`, a permutation test for hypothesis testing (or A/B testing) to test if two samples come from the same distribution. Or in other words, a procedure to test the null hypothesis that that two groups are not significantly different (e.g., a treatment and a control group). ([250](https://github.com/rasbt/mlxtend/pull/250))
- Added `'leverage'` and `'conviction` as evaluation metrics to the `frequent_patterns.association_rules` function. ([246](https://github.com/rasbt/mlxtend/pull/246) & [#247](https://github.com/rasbt/mlxtend/pull/247))
- Added a `loadings_` attribute to `PrincipalComponentAnalysis` to compute the factor loadings of the features on the principal components. ([251](https://github.com/rasbt/mlxtend/pull/251))
- Allow grid search over classifiers/regressors in ensemble and stacking estimators. ([259](https://github.com/rasbt/mlxtend/pull/259))
- New `make_multiplexer_dataset` function that creates a dataset generated by a n-bit Boolean multiplexer for evaluating supervised learning algorithms. ([263](https://github.com/rasbt/mlxtend/pull/263))
- Added a new `BootstrapOutOfBag` class, an implementation of the out-of-bag bootstrap to evaluate supervised learning algorithms. ([265](https://github.com/rasbt/mlxtend/pull/265))
- The parameters for `StackingClassifier`, `StackingCVClassifier`, `StackingRegressor`, `StackingCVRegressor`, and `EnsembleVoteClassifier` can now be tuned using scikit-learn's `GridSearchCV` ([254](https://github.com/rasbt/mlxtend/pull/254) via [James Bourbeau](https://github.com/jrbourbeau))


Changes

- The `'support'` column returned by `frequent_patterns.association_rules` was changed to compute the support of "antecedant union consequent", and new `antecedant support'` and `'consequent support'` column were added to avoid ambiguity. ([245](https://github.com/rasbt/mlxtend/pull/245))
- Allow the `OnehotTransactions` to be cloned via scikit-learn's `clone` function, which is required by e.g., scikit-learn's `FeatureUnion` or `GridSearchCV` (via [Iaroslav Shcherbatyi](https://github.com/iaroslav-ai)). ([#249](https://github.com/rasbt/mlxtend/pull/249))

Bug Fixes

- Fix issues with `self._init_time` parameter in `_IterativeModel` subclasses. ([256](https://github.com/rasbt/mlxtend/pull/256))
- Fix imprecision bug that occurred in `plot_ecdf` when run on Python 2.7. ([264](https://github.com/rasbt/mlxtend/pull/264))
- The vectors from SVD in `PrincipalComponentAnalysis` are no being scaled so that the eigenvalues via `solver='eigen'` and `solver='svd'` now store eigenvalues that have the same magnitudes. ([251](https://github.com/rasbt/mlxtend/pull/251))

Downloads

- [Source code (zip)](https://github.com/rasbt/mlxtend/archive/v0.8.0.zip)
- [Source code (tar.gz)](https://github.com/rasbt/mlxtend/archive/v0.8.0.tar.gz)

New Features

- Added a `mlxtend.evaluate.bootstrap` that implements the ordinary nonparametric bootstrap to bootstrap a single statistic (for example, the mean. median, R^2 of a regression fit, and so forth) [232](https://github.com/rasbt/mlxtend/pull/232)
- `SequentialFeatureSelecor`'s `k_features` now accepts a string argument "best" or "parsimonious" for more "automated" feature selection. For instance, if "best" is provided, the feature selector will return the feature subset with the best cross-validation performance. If "parsimonious" is provided as an argument, the smallest feature subset that is within one standard error of the cross-validation performance will be selected. [238](https://github.com/rasbt/mlxtend/pull/238)

Changes

- `SequentialFeatureSelector` now uses `np.nanmean` over normal mean to support scorers that may return `np.nan` [211](https://github.com/rasbt/mlxtend/pull/211) (via [mrkaiser](https://github.com/mrkaiser))
- The `skip_if_stuck` parameter was removed from `SequentialFeatureSelector` in favor of a more efficient implementation comparing the conditional inclusion/exclusion results (in the floating versions) to the performances of previously sampled feature sets that were cached [237](https://github.com/rasbt/mlxtend/pull/237)
- `ExhaustiveFeatureSelector` was modified to consume substantially less memory [195](https://github.com/rasbt/mlxtend/pull/195) (via [Adam Erickson](https://github.com/adam-erickson))

Bug Fixes

- Fixed a bug where the `SequentialFeatureSelector` selected a feature subset larger than then specified via the `k_features` tuple max-value [213](https://github.com/rasbt/mlxtend/pull/213)

New Features

- New [mlxtend.plotting.ecdf](http://rasbt.github.io/mlxtend/user_guide/plotting/ecdf/) function for plotting empirical cumulative distribution functions ([#196](https://github.com/rasbt/mlxtend/pull/196)).
- New [`StackingCVRegressor`](http://rasbt.github.io/mlxtend/user_guide/regressor/StackingCVRegressor/) for stacking regressors with out-of-fold predictions to prevent overfitting ([#201](https://github.com/rasbt/mlxtend/pull/201)via [Eike Dehling](https://github.com/EikeDehling)).

Changes

- The TensorFlow estimator have been removed from mlxtend, since TensorFlow has now very convenient ways to build on estimators, which render those implementations obsolete.
- `plot_decision_regions` now supports plotting decision regions for more than 2 training features [189](https://github.com/rasbt/mlxtend/pull/189), via [James Bourbeau](https://github.com/jrbourbeau)).
- Parallel execution in `mlxtend.feature_selection.SequentialFeatureSelector` and `mlxtend.feature_selection.ExhaustiveFeatureSelector` is now performed over different feature subsets instead of the different cross-validation folds to better utilize machines with multiple processors if the number of features is large ([193](https://github.com/rasbt/mlxtend/pull/193), via [whalebot-helmsman](https://github.com/whalebot-helmsman)).
- Raise meaningful error messages if pandas `DataFrame`s or Python lists of lists are fed into the `StackingCVClassifer` as a `fit` arguments ([198](https://github.com/rasbt/mlxtend/pull/198)).
- The `n_folds` parameter of the `StackingCVClassifier` was changed to `cv` and can now accept any kind of cross validation technique that is available from scikit-learn. For example, `StackingCVClassifier(..., cv=StratifiedKFold(n_splits=3))` or `StackingCVClassifier(..., cv=GroupKFold(n_splits=3))` ([203](https://github.com/rasbt/mlxtend/pull/203), via [Konstantinos Paliouras](https://github.com/sque)).

Bug Fixes

- `SequentialFeatureSelector` now correctly accepts a `None` argument for the `scoring` parameter to infer the default scoring metric from scikit-learn classifiers and regressors ([171](https://github.com/rasbt/mlxtend/pull/171)).
- The `plot_decision_regions` function now supports pre-existing axes objects generated via matplotlib's `plt.subplots`. ([184](https://github.com/rasbt/mlxtend/pull/184), [see example](http://rasbt.github.io/mlxtend/user_guide/plotting/plot_decision_regions/#example-6-working-with-existing-axes-objects-using-subplots))
- Made `math.num_combinations` and `math.num_permutations` numerically stable for large numbers of combinations and permutations ([200](https://github.com/rasbt/mlxtend/pull/200)).