This release brings new features and breaking API changes.
Read on for more details.
What's changed
Many updates to `StreamingDataFrame` and `StreamingSeries` (ex-`Column`) [[238](https://github.com/quixio/quix-streams/pull/238)]
We updated how streaming data is processed by `StreamingDataFrame` and `StreamingSeries` classes.
They now use the same engine to apply functions to the message values.
It introduces both new features and breaking changes:
1. [breaking] Changes to `StreamingDataFrame` and new methods `.update()` and `.filter()`
- **Functions passed to `StreamingDataFrame.apply()` should always return a new value.** The result of the function will be propagated downstream.
Previously, `.apply()` was used to both mutate values and generate new ones.
Now it's discouraged, and the `.update()` method should be used instead to mutate data and perform side effects (e.g. to print to the console).
[Docs](https://github.com/quixio/quix-streams/blob/v2.1alpha1/docs/streamingdataframe.md#streamingdataframeupdatefunction)
- **New method `StreamingDataFrame.update()` to mutate values in place.**
[Docs](https://github.com/quixio/quix-streams/blob/v2.1alpha1/docs/streamingdataframe.md#streamingdataframeupdatefunction)
- **New method `StreamingDataFrame.filter()` to filter values.**
[Docs](https://github.com/quixio/quix-streams/blob/v2.1alpha1/docs/streamingdataframe.md#streamingdataframeupdatefunction)
- **New syntax to filter values - `dataframe[dataframe.apply(<func>)]`**
`StreamingDataFrame` can now filter data using a similar API as `pandas.DataFrame`.
[Docs](https://github.com/quixio/quix-streams/blob/v2.1alpha1/docs/streamingdataframe.md#streamingdataframeapplyfunction)
- **New syntax to update values - `dataframe['field'] = dataframe.apply(<func>)`**
`StreamingDataFrame` can now assign new keys to the values using functions.
[Docs](https://github.com/quixio/quix-streams/blob/v2.1alpha1/docs/streamingdataframe.md#streamingdataframeapplyfunction)`
- **`StreamingDataFrame` doesn't require values to be dictionaries anymore.**
- **`StreamingDataFrame` methods now return a new `StreamingDataFrame`.**
Previously, many methods like `.apply()` and `.to_topic()` were mutating the state of the `StreamingDataFrame` instance.
Now all of them return a new instance, and the current instance remains intact.
- **Functions passed to `.apply()` now get only one argument - message value.**
Previously, functions passed to `.apply()` were provided with at least 2 positional arguments: message value and `MessageContext` with Kafka message metadata.
Now they don't need to accept `MessageContext` to access a current message key, and it is stored globally for each incoming message.
To get a current message key and or full message metadata use `quixstreams.message_key()` and `quixstreams.message_context()` in
your custom functions.
[Docs](https://github.com/quixio/quix-streams/blob/v2.1alpha1/docs/streamingdataframe.md#accessing-the-kafka-message-keys-and-metadata)
New method to check if the key exists in the message value - `StreamingDataFrame.contains()` [[235](https://github.com/quixio/quix-streams/pull/235)]
New method to clear the local state - `Application.clear_state()`[[239](https://github.com/quixio/quix-streams/pull/239)]
Use `Application.clear_state()` to clear the local state of the application.
Quix Streams keeps state data per consumer group, so the state will be cleared only for the
particular consumer group.
[breaking] Changes to logical operators `|`, `&` and `~` in `StreamingSeries` [[238](https://github.com/quixio/quix-streams/pull/238)]
The logical operators `|`, `&`, and `~` now do logical comparisons.
Previously it worked this way only with booleans, when for numbers it would do a bitwise operation.
Support for Python 3.12 [[244](https://github.com/quixio/quix-streams/pull/244)]
Quix Streams now works with Python 3.12
**Full Changelog**: https://github.com/quixio/quix-streams/compare/v2.0alpha2...v2.1alpha1