Featurebyte

🛑 Breaking Changes

+ `Configurations` Configurations::use_profile() function is now a method rather than a classmethod
diff
- Configurations.use_profile("profile")
+ Configurations().use_profile("profile")


💡 Enhancements

+ `service` Cache view created from query in Spark for better performance
+ `vector-aggregation` Add java UDAFs for sum and max for use in spark.
+ `vector-operations` Add cosine_similarity to compare two vector columns.
+ `vector-aggregation` Add integration test to test end to end for VECTOR_AGGREGATE_MAX.
+ `vector-aggregations` Enable vector aggregations for tiling aggregate - max and sum - functions
+ `middleware` Organize exceptions to reduce verbosity in middleware
+ `api` Add support for updating description of table columns in the python API
+ `vector-aggregation` Update groupby logic for non tile based aggregates
+ `api` Implement API object for Use Case component
+ `api` Use Context name instead of Context id for the API signature
+ `api` Implement API object for Context
+ `vector_aggregation` Add UDTF for max, sum and avg for snowflake.
+ `api` Integrate Context API object for UseCase
+ `vector-aggregation` Snowflake return values for vector aggregations should be a list now, instead of a string.
+ `vector-aggregation` Add java UDAFs for average for use in spark.
+ `vector_aggregation` Only return one row in table vector aggregate function per partition
+ `service` Support conditionally updating a feature using a mask derived from other feature(s)
+ `vector-aggregation` Add guardrails to prevent array aggregations if agg func is not max or avg.
+ `service` Tag semantics for all special columns during table creation
+ `api` Implement UseCase Info
+ `service` Change join type to inner when joining event and item tables
+ `vector-aggregation` Register vector aggregate max, and update parent dtype inference logic.
+ `service` Implement scheduled task to clean up stale versions and drop online store tables when possible
+ `use-case` Implement guardrail for use case's observation table not to be deleted
+ `vector-aggregations` Enable vector aggregations for tiling aggregate avg function
+ `api` Rename description update functions for versioned assets
+ `vector-aggregation` Support integer values in vectors; add support integration test for simple aggregates
+ `vector-aggregation` Update groupby_helper to take in parent_dtype.
+ `httpClient` added a ssl_verify value in Configurations to allow disabling of ssl certificate verification
+ `online-serving` Split online store compute and insert query to minimize table locking
+ `tests` Use the notebook as the test id in the notebook tests.
+ `vector-aggregation` Add simple average spark udaf.
+ `vector-aggregation` Add average snowflake udtf.
+ `api` Associate Deployment with UseCase
+ `service` Skip creating a data warehouse session when online disabling a feature
+ `use-case` implement use case model and its associated routes
+ `service` Apply event timestamp filter on EventTable directly in scheduled tile jobs when possible

🐛 Bug Fixes

+ `worker` Block running multiple concurrent deployment create/update tasks for the same deployment
+ `service` Fix bug where feature job starts running while the feature is still being enabled
+ `dependencies` upgrading `scipy` dependency
+ `service` Fixes an invalid identifier error in sql when feature involves a mix of filtered and non-filtered versions of the same view.
+ `worker` Fixes a bug where scheduler does not work with certain mongodb uris.
+ `online-serving` Fix incompatible column types when inserting to online store tables
+ `service` Fix feature saving error due to tile generation bug
+ `service` Ensure row ordering of online serving output DataFrame matches input request data
+ `dependencies` Limiting python range to 3.8>=,<3.12 due to scipy constraint
+ `service` Use execute_query_long_running when inserting to online store tables to fix timeout errors
+ `model` Mongodb index on periodic task name conflicts with scheduler engine
+ `service` Fix conversion of date type to double in spark

🐛 Bug Fixes

+ `api` Fix logic for determining timezone offset column in datetime accessor
+ `service` Fix SDK code generation for conditional assignment when the assign value is a series
+ `service` Fix invalid identifier error for complex features with both item and window aggregates

💡 Enhancements

+ `profile` Allow creating of profile directly with fb.register_profile(name, url, token)

🐛 Bug Fixes

+ `service` Fix feature materialization error due to ambiguous internal column names
+ `service` Fix error when generating info for features in some edge cases
+ `api` Fix item table default job settings not synchronized when job settings are updated in the event table, fix historical feature table listing failure

🛑 Breaking Changes

+ `target` Update compute_target to return observation table instead of target table
will make it easier to use with compute historical features
+ `target` Update target info to return a TableBriefInfoList instead of a custom struct
this will help keep it consistent with feature, and also fix a bug in info where we wrongly assumed there was only one input table.

💡 Enhancements

+ `target` Add as_target to SDK, and add node to graph when it is called
+ `target` Add fill_value and skip_fill_na to forward_aggregate, and update name
+ `target` Create lookup target graph node
+ `service` Speed up operation structure extraction by caching the result of _extract() in BaseGraphExtractor

🐛 Bug Fixes

+ `api` Fix api objects listing failure in some notebooks environments
+ `utils` Fix is_notebook check to support Google Colab [https://github.com/featurebyte/featurebyte/issues/1598]

🛑 Breaking Changes

+ `online-serving` Update online store table schema to use long table format
+ `dependencies` Limiting python version from >=3.8,<4.0 to >=3.8,<3.13 due to scipy version constraint

💡 Enhancements

+ `generic-function` add user-defined-function support
+ `target` add basic API object for Target
Initialize the basic API object for Target.
+ `feature-group` update the feature group save operation to use `/feature/batch` route
+ `service` Update describe query to be compatible with Spark 3.2
+ `service` Ensure FeatureModel's derived attributes are derived from pruned graph
+ `target` add basic info for Target
Adds some basic information about Target's. Additional information that contains more details about the actual data will be added in a follow-up.
+ `list_versions` update Feature's & FeatureList's `list_versions` method by adding `is_default` to the dataframe output
+ `service` Move TILE_JOB_MONITOR table from data warehouse to persistent
+ `service` Avoid using SHOW COLUMNS to support Spark 3.2
+ `table` skip calling data warehouse for table metadata during table construction
+ `target` add ForwardAggregate node to graph for ForwardAggregate
Implement ForwardAggregator - only adds node to graph. Node is still a no-op.
+ `service` Add option to disable audit logging for internal documents
+ `query-graph` optimize query graph pruning computation by combining multiple pruning tasks into one
+ `target` add input data and metadata for targets
Add more information about target metadata.
+ `target` Add primary_entity property to Target API object.
+ `service` Refactor FeatureManager and TileManager as services
+ `tests` Move tutorial notebooks into the FeatureByte repo
+ `service` Replace ONLINE_STORE_MAPPING data warehouse table by OnlineStoreComputeQueryService
+ `feature` block feature readiness & feature list status transition from DRAFT to DEPRECATED
+ `task_manager` refactor task manager to take celery object as a parameter, and refactor task executor to import tasks explicitly
+ `feature` fix bug with feature_list_ids not being updated after a feature list is deleted
+ `service` Replace TILE_FEATURE_MAPPING table in the data warehouse with mongo persistent
+ `target` perform SQL generation for forward aggregate node
+ `feature` fix primary entity identification bug for time aggregation over item aggregation features
+ `feature` limit manual default feature version selection to only the versions with highest readiness level
+ `feature-list` revise feature list saving to reduce api calls
+ `service` Refactor tile task to use dependency injection
+ `service` Fix error when disabling features created before OnlineStoreComputeQueryService is introduced
+ `deployment` Skip redundant updates of ONLINE_STORE_MAPPING table
+ `static-source-table` support materialization of static source table from source table or view
+ `catalog` Create target_table API object
Remove default catalog, require explicit activation of catalog before catalog operations.
+ `feature-list` update feature list to preserve feature order
+ `target` Add gates to prevent target from setting item to non-target series.
+ `target` Add TargetNamespacecreate
This will allow us to register spec-like versions of a Target, that don't have a recipe attached.
+ `deployment` Reduce unnecessary backfill computation when deploying features
+ `service` Refactor TileScheduler as a service
+ `target` stub out target namespace schema and models
+ `service` Add traceback to tile job log for troubleshooting
+ `target` add end-to-end integration test for target, and include preview endpoint in target
+ `feature` update feature & feature list save operation to use POST `/feature/batch` route
+ `service` Disable tile monitoring by default
+ `service` Fix listing of databases and schemas in Spark 3.2
+ `target` Refactor compute_target and compute_historical_feature
+ `feature` optimize time to deserialize feature model
+ `entity-relationship` remove POST /relationship_info, POST /entity/parent and DELETE /entity/parent/<parent_entity_id> endpoints
+ `service` Support description update and retrieval for all saved objects
+ `config` Add default_profile in config to allow for a default profile to be set, and require a profile to be set if default_profile is not set
+ `target` Create target_table API object
Create the TargetTable API object, and stub out the compute_target endpoint.
+ `target` Add datetime and string accessors into the Target API object.
+ `service` Fix unnecessary usage of SQL functions incompatible with Spark 3.2 (ILIKE and DATEADD)
+ `preview` Improve efficiency of feature and feature list preview by reducing unnecessary tile computation
+ `service` Fix DATEADD undefined function error in Spark 3.2 and re-enable tests
+ `service` Implement TileRegistryService to track tile information in mongo persistent
+ `spark-session` add kerberos authentication and webhdfs support for Spark session
+ `service` Fix compatibility of string contains operation with Spark 3.2
+ `target` add CRUD API endpoints for Target
First portion of the work to include the Target API object.
+ `target` Fully implement compute_target to materialize a dataframe
+ `service` Refactor info service by splitting out logic to their respective services.
Most of the info service logic was not being reused. It also feels cleaner for each service to be responsible for its own info logic. This way, dependencies are clearer. We also refactor service initialization such that we consistently use the dependency injection pattern.
+ `online-serving` Use INSERT operation to update online store tables to address concurrency issues
+ `target` create target namespace when we create a target
+ `service` Fix more datetime transform compatibility issues in Spark 3.2
+ `storage` Add support for using s3 as storage for featurebyte service
+ `target` Create target_table services, routes, models and schema
This will help us support materializing target tables in the warehouse.

⚠️ Deprecations

+ `target` remove blind_spot from target models as it is not used

🐛 Bug Fixes

+ `worker` fixed cpu threading model
+ `service` Fix feature definition for isin() operation
+ `online-serving` Fix the job_schedule_ts_str parameter when updating online store tables in scheduled tile tasks
+ `gh-actions` Add missing build dependencies for kerberos support.
+ `feature_readiness` fix feature readiness bug due to readiness is treated as string when finding default feature ID
+ `transforms` Update get_relative_frequency to return 0 when there is no matching label
+ `service` Fix OnlineStoreComputeQuery prematurely deleted when still in use by other features
+ `data-warehouse` Fix metadata schema update for Spark and Databricks and bump working version
+ `service` Fix TABLESAMPLE syntax error in Spark for very small sample percentage
+ `feature` fix view join operation bug which causes improper query graph pruning
+ `service` Fix a bug in add_feature() where entity_id was incorrectly attached to the derived column