Featurebyte

🛑 Breaking Changes

+ `api` Support using observation tables in feature, target and featurelist preview
+ Parameter `observation_set` in `Feature.preview`, `Target.preview` and `FeatureList.preview` now accepts `ObservationTable` object or pandas dataframe
+ Breaking change: Parameter `observation_table` in `FeatureList.compute_historical_feature_table` is renamed to `observation_set`
+ `feature_list` Change feature list catalog output dataframe column name from `primary_entities` to `primary_entity`

💡 Enhancements

+ `databricks-unity` Add session for databricks unity cluster, and migrate one UDF to python for databricks unity cluster.
+ `target` Allow users to create observation table with just a target id, but no graph.
+ `service` Support latest aggregation for vector columns
+ `service` Update repeated columns validation logic to handle excluded columns.
+ `endpoints` Enable observation table to associate with multiple use cases from endpoints
+ `target` Derive window for lookup targets as well
+ `service` Add critical data info validation logic
+ `api` Implement remove observation table from context
+ `service` Support rename of context, use case, observation table and historical feature table
+ `target_table` Persist primary entity IDs for the target observation table
+ `observation_table` Update observation table creation check to make sure primary entity is set
+ `service` Implement service to materialize features to be published to external feature store
+ `service` Add feature definition hash to new feature model to allow duplicated features to be detected
+ `observation_table` Track uploaded file name when creating an observation table from an uploaded file.
+ `observation_table` Add way to update purpose for observation table.
+ `tests` Use published featurebyte library in notebook tests.
+ `service` Reduce complexity of describe query to avoid memory issue during query compilation
+ `session` Use DBFS for Databricks session storage to simplify setup
+ `target_namespace` Add support for target namespace deletion
+ `observation_table` add minimum interval between entities to observation table
+ `api` Implement delete observation table from use case
+ `api` Implement removal of default preview and eda table for context
+ `api` Enable observation table to associate with multiple use cases from api
+ `api` Implement removal of default preview and eda table for use case

🐛 Bug Fixes

+ `observation_table` fix validation around primary entity IDs when creating observation tables
+ `worker` Use cpu worker for feature job setting analysis to avoid blocking io worker async loop
+ `session` Make data warehouse session creation asynchronous with a timeout to avoid blocking the asyncio main thread. This prevents the API service from being unresponsive when certain compute clusters takes a long time to start up.
+ `service` Fix observation table sampling so that it is always uniform over the input
+ `worker` Fix feature job setting analysis fails for databricks feature store
+ `session` Fix spark session failing with spark version >= 3.4.1
+ `service` Fix observation table file upload error
+ `target` Support value_column=None for count in forward_aggregate/target operations.
+ `service` Fix division by zero error when calling describe on empty views
+ `worker` Fix bug where feature job setting analysis backtest fails when the analysis is missing an optional histogram
+ `service` Fixes a view join issue that causes the generated feature not savable due to graph inconsistency.
+ `use_case` Allow use cases to be created with descriptive only targets
+ `service` Fixes an error when rendering FeatureJobStatusResult in notebooks when matplotlib package is not available.
+ `feature` Fix feature saving bug when the feature contains timestamp filtering

🐛 Bug Fixes

+ `api` fixed async task return code

🛑 Breaking Changes

+ `observation_table` Validate that entities are present when creating an observation table.

💡 Enhancements

+ `target` Use window from target namespace instead of the target version.
+ `service` UseCase creation to accept TargetNameSpace id as a parameter
+ `historical_feature_table` Make FeatureClusters optional when creating historical feature table from UI.
+ `service` Move online serving code template generation to the online serving service
+ `model` Handle old Context records with entity_ids attribute in the database
+ `service` Add key_with_highest_value() and key_with_lowest_value() for cross aggregates
+ `api` Add consistent table feature job settings validation during feature creation.
+ `api` Change Context Entity attribute's name to Primary Entity
+ `api` Use primary entity parameter in Target and Context creation
+ `service` Add last_updated_at in FeatureModel to indicate when feature value is last updated
+ `api` Revise feature list create new version to avoid throwing error when the feature list is the same as the previous version
+ `service` Support rprefix parameter in View's join method
+ `observation_table` Add an optional purpose to observation table when creating a new observation table.
+ `docs` Documentation for Context and UseCase
+ `observation_table` Track earliest point in time, and unique entity col counts as part of metadata.
+ `service` Support extracting value counts and customised statistics in PreviewService
+ `api` Remove direct observation table reference from UseCase
+ `warehouse` improve data warehouse asset validation
+ `api` Use EntityBriefInfoList for entity info for both UseCase and Context
+ `api` Add trigo functions to series.
+ `api` Include observation table operation into Context API Object
+ `observation_table` Add route to allow users to upload CSV files to create observation tables.
+ `target` Tag entity_ids when creating an observation table from a target.
+ `api-client` improve api-client retry
+ `service` Entity Validation for Context, Target and UseCase
+ `service` Add Context Info method into both Context API Object and Route
+ `api` Add functionality to calculate haversine distance.
+ `service` Fix PreviewService describe() method when stats_names are provided

🐛 Bug Fixes

+ `service` Validate non-existent Target and Context when creating Use Case
+ `session` Fix execute query failing when variant columns contain null values
+ `service` Validate null target_id when adding obs table to use case
+ `service` Fix maximum recursion depth exceeded error in complex queries
+ `service` Fix race condition when accessing cached values in ApiObject's get_by_id()
+ `hive` fix hive connection error when spark_catalog is not the default
+ `api` Targetlist should include items in target namespace.
+ `target` Fix target definition SDK code generation by skipping project.
+ `service` Fix join validation logic to account for rprefix

💡 Enhancements

+ `service` Optimize feature readiness service update runtime.

🐛 Bug Fixes

+ `packaging` Restore cryptography package dependency [DEV-2233]

🛑 Breaking Changes

+ `Configurations` Configurations::use_profile() function is now a method rather than a classmethod
diff
- Configurations.use_profile("profile")
+ Configurations().use_profile("profile")


💡 Enhancements

+ `service` Cache view created from query in Spark for better performance
+ `vector-aggregation` Add java UDAFs for sum and max for use in spark.
+ `vector-operations` Add cosine_similarity to compare two vector columns.
+ `vector-aggregation` Add integration test to test end to end for VECTOR_AGGREGATE_MAX.
+ `vector-aggregations` Enable vector aggregations for tiling aggregate - max and sum - functions
+ `middleware` Organize exceptions to reduce verbosity in middleware
+ `api` Add support for updating description of table columns in the python API
+ `vector-aggregation` Update groupby logic for non tile based aggregates
+ `api` Implement API object for Use Case component
+ `api` Use Context name instead of Context id for the API signature
+ `api` Implement API object for Context
+ `vector_aggregation` Add UDTF for max, sum and avg for snowflake.
+ `api` Integrate Context API object for UseCase
+ `vector-aggregation` Snowflake return values for vector aggregations should be a list now, instead of a string.
+ `vector-aggregation` Add java UDAFs for average for use in spark.
+ `vector_aggregation` Only return one row in table vector aggregate function per partition
+ `service` Support conditionally updating a feature using a mask derived from other feature(s)
+ `vector-aggregation` Add guardrails to prevent array aggregations if agg func is not max or avg.
+ `service` Tag semantics for all special columns during table creation
+ `api` Implement UseCase Info
+ `service` Change join type to inner when joining event and item tables
+ `vector-aggregation` Register vector aggregate max, and update parent dtype inference logic.
+ `service` Implement scheduled task to clean up stale versions and drop online store tables when possible
+ `use-case` Implement guardrail for use case's observation table not to be deleted
+ `vector-aggregations` Enable vector aggregations for tiling aggregate avg function
+ `api` Rename description update functions for versioned assets
+ `vector-aggregation` Support integer values in vectors; add support integration test for simple aggregates
+ `vector-aggregation` Update groupby_helper to take in parent_dtype.
+ `httpClient` added a ssl_verify value in Configurations to allow disabling of ssl certificate verification
+ `online-serving` Split online store compute and insert query to minimize table locking
+ `tests` Use the notebook as the test id in the notebook tests.
+ `vector-aggregation` Add simple average spark udaf.
+ `vector-aggregation` Add average snowflake udtf.
+ `api` Associate Deployment with UseCase
+ `service` Skip creating a data warehouse session when online disabling a feature
+ `use-case` implement use case model and its associated routes
+ `service` Apply event timestamp filter on EventTable directly in scheduled tile jobs when possible

🐛 Bug Fixes

+ `worker` Block running multiple concurrent deployment create/update tasks for the same deployment
+ `service` Fix bug where feature job starts running while the feature is still being enabled
+ `dependencies` upgrading `scipy` dependency
+ `service` Fixes an invalid identifier error in sql when feature involves a mix of filtered and non-filtered versions of the same view.
+ `worker` Fixes a bug where scheduler does not work with certain mongodb uris.
+ `online-serving` Fix incompatible column types when inserting to online store tables
+ `service` Fix feature saving error due to tile generation bug
+ `service` Ensure row ordering of online serving output DataFrame matches input request data
+ `dependencies` Limiting python range to 3.8>=,<3.12 due to scipy constraint
+ `service` Use execute_query_long_running when inserting to online store tables to fix timeout errors
+ `model` Mongodb index on periodic task name conflicts with scheduler engine
+ `service` Fix conversion of date type to double in spark

🐛 Bug Fixes

+ `api` Fix logic for determining timezone offset column in datetime accessor
+ `service` Fix SDK code generation for conditional assignment when the assign value is a series
+ `service` Fix invalid identifier error for complex features with both item and window aggregates

💡 Enhancements

+ `profile` Allow creating of profile directly with fb.register_profile(name, url, token)