Snowflake-ml-python

Bug Fixes

- Modeling: Fix an issue causing lineage information to be missing for
`Pipeline`, `GridSearchCV` , `SimpleImputer`, and `RandomizedSearchCV`
- Registry: Fix an issue that leads to incorrect result when using pandas Dataframe with over 100, 000 rows as the input
of `ModelVersion.run` method in Stored Procedure.

Behavior Changes

New Features

- Registry: Add support for TIMESTAMP_NTZ model signature data type, allowing timestamp input and output.
- Dataset: Add `DatasetVersion.label_cols` and `DatasetVersion.exclude_cols` properties.

Bug Fixes

- Registry: Fix an issue that leads to unable to log model in store procedure.
- Modeling: Quick fix `import snowflake.ml.modeling.parameters.enable_anonymous_sproc` cannot be imported due to package
dependency error.

Behavior Changes

New Features

Bug Fixes

- Dataset: Fix `snowflake.connector.errors.DataError: Query Result did not match expected number of rows` when accessing
DatasetVersion properties when case insensitive `SHOW VERSIONS IN DATASET` check matches multiple version names.
- Dataset: Fix bug in SnowFS bulk file read when used with DuckDB
- Registry: Fixed a bug when loading old models.
- Lineage: Fix Dataset source lineage propagation through `snowpark.DataFrame` transformations

Behavior Changes

- Feature Store: convert clear() into a private function. Also make it deletes feature views and entities only.
- Feature Store: Use NULL as default value for timestamp tag value.

New Features

- Feature Store: Added new `snowflake.ml.feature_store.setup_feature_store()` API to assist Feature Store RBAC setup.
- Feature Store: Add `output_type` argument to `FeatureStore.generate_dataset()` to allow generating data snapshots
as Datasets or Tables.
- Registry: `log_model`, `get_model`, `delete_model` now supports fully qualified name.
- Modeling: Supports anonymous stored procedure during fit calls so that modeling would not require sufficient
permissions to operate on schema. Please call
`import snowflake.ml.modeling.parameters.enable_anonymous_sproc noqa: F401`

Bug Fixes

- Registry: Fix invalid parameter 'SHOW_MODEL_DETAILS_IN_SHOW_VERSIONS_IN_MODEL' error.

Behavior Changes

- Model Development: The behavior of `fit_transform` for all estimators is changed.
Firstly, it will cover all the estimator that contains this function,
secondly, the output would be the union of pandas DataFrame and snowpark DataFrame.

Model Registry (PrPr)

`snowflake.ml.registry.artifact` and related `snowflake.ml.model_registry.ModelRegistry` APIs have been removed.

- Removed `snowflake.ml.registry.artifact` module.
- Removed `ModelRegistry.log_artifact()`, `ModelRegistry.list_artifacts()`, `ModelRegistry.get_artifact()`
- Removed `artifacts` argument from `ModelRegistry.log_model()`

Dataset (PrPr)

`snowflake.ml.dataset.Dataset` has been redesigned to be backed by Snowflake Dataset entities.

- New `Dataset`s can be created with `Dataset.create()` and existing `Dataset`s may be loaded
with `Dataset.load()`.
- `Dataset`s now maintain an immutable `selected_version` state. The `Dataset.create_version()` and
`Dataset.load_version()` APIs return new `Dataset` objects with the requested `selected_version` state.
- Added `dataset.create_from_dataframe()` and `dataset.load_dataset()` convenience APIs as a shortcut
to creating and loading `Dataset`s with a pre-selected version.
- `Dataset.materialized_table` and `Dataset.snapshot_table` no longer exist with `Dataset.fully_qualified_name`
as the closest equivalent.
- `Dataset.df` no longer exists. Instead, use `DatasetReader.read.to_snowpark_dataframe()`.
- `Dataset.owner` has been moved to `Dataset.selected_version.owner`
- `Dataset.desc` has been moved to `DatasetVersion.selected_version.comment`
- `Dataset.timestamp_col`, `Dataset.label_cols`, `Dataset.feature_store_metadata`, and
`Dataset.schema_version` have been removed.

Feature Store (PrPr)

- `FeatureStore.generate_dataset` argument list has been changed to match the new
`snowflake.ml.dataset.Dataset` definition

- `materialized_table` has been removed and replaced with `name` and `version`.
- `name` moved to first positional argument
- `save_mode` has been removed as `merge` behavior is no longer supported. The new behavior is always `errorifexists`.

- Change feature view version type from str to `FeatureViewVersion`. It is a restricted string literal.

- Remove as_dataframe arg from FeatureStore.list_feature_views(), now always returns result as DataFrame.

- Combines few metadata tags into a new tag: SNOWML_FEATURE_VIEW_METADATA. This will make previously created feature views
not readable by new SDK.

New Features

- Registry: Add `export` method to `ModelVersion` instance to export model files.
- Registry: Add `load` method to `ModelVersion` instance to load the underlying object from the model.
- Registry: Add `Model.rename` method to `Model` instance to rename or move a model.

Dataset (PrPr)

- Added Snowpark DataFrame integration using `Dataset.read.to_snowpark_dataframe()`
- Added Pandas DataFrame integration using `Dataset.read.to_pandas()`
- Added PyTorch and TensorFlow integrations using `Dataset.read.to_torch_datapipe()`
and `Dataset.read.to_tf_dataset()` respectively.
- Added `fsspec` style file integration using `Dataset.read.files()` and `Dataset.read.filesystem()`

Feature Store

- use new tag_reference_internal to speed up metadata lookup.

New Features

- Registry: Add support for `catboost` model (`catboost.CatBoostClassifier`, `catboost.CatBoostRegressor`).
- Registry: Add support for `lightgbm` model (`lightgbm.Booster`, `lightgbm.LightGBMClassifier`, `lightgbm.LightGBMRegressor`).

Bug Fixes

- Registry: Fix a bug that leads to relax_version option is not working.

Behavior changes

- Feature Store: update_feature_view takes refresh_freq and warehouse as argument.

Bug Fixes

- Registry: Fix a bug when multiple models are being called from the same query, models other than the first one will
have incorrect result. This fix only works for newly logged model.
- Modeling: When registering a model, only method(s) that is mentioned in `save_model` would be added to model signature
in SnowML models.
- Modeling: Fix a bug that when n_jobs is not 1, model cannot execute methods such as
predict, predict_log_proba, and other batch inference methods. The n_jobs would automatically
set to 1 because vectorized udf currently doesn't support joblib parallel backend.
- Modeling: Fix a bug that batch inference methods cannot infer the datatype when the first row of data contains NULL.
- Modeling: Matches Distributed HPO output column names with the snowflake identifier.
- Modeling: Relax package versions for all Distributed HPO methods if the installed version
is not available in the Snowflake conda channel
- Modeling: Add sklearn as required dependency for LightGBM package.

Behavior Changes

- Registry: `apply` method is no longer by default logged when logging a xgboost model. If that is required, it could
be specified manually when logging the model by `log_model(..., options={"target_methods": ["apply", ...]})`.
- Feature Store: register_entity returns an entity object.
- Feature Store: register_feature_view `block=true` becomes default.

New Features

- Registry: Add support for `sentence-transformers` model (`sentence_transformers.SentenceTransformer`).
- Registry: Now version name is no longer required when logging a model. If not provided, a random human readable ID
will be generated.