- Add _serialize_json_safe to utils and add unit tests
- various memory optimizations in interpret-community done as part of integration with another product
1.) where possible use _local_importance_values since local_importance_values converts from numpy to list
2.) avoid typed_dataset in some paths since that creates a new pandas dataframe
3.) add clear_references parameter to DatasetWrapper such that it can be cleared after use, which should reduce memory usage - note this is only for users who know what they are doing
4.) added test for dense and wide data which can be used for future performance testing as well
- Pickling and Unpickling MimicExplainer and Surrogate models
- fix categorical handling for scikit-learn 0.24 during one hot encoding for failing test
- Add unit test for error_handling.py
- Use ndcg from metrics in test_validate_explanations.py
- Consolidate MimicExplainer serialization tests
- Add support for inverse soft logit for binary classification scenarios
- Add replication metric computation in MimicExplainer
- various memory optimizations to the explanation-related APIs, particularly explain_global, including:
1.) preventing matrix multiply if matrix is identity in engineered to raw mapping (which happens very often) - this significantly reduces memory usage
2.) differentiating between call from explain_global vs explain_local when entering _explain_local which allows us to skip some duplicate computations which lead to higher memory usage
3.) refactoring out into functions (eg _explain_local_helper) which allows GC of various temporary variables (I could have also used del but it looks much uglier)