Bytewax

Latest version: v0.21.0

Safety actively analyzes 682387 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 3 of 4

0.16.0

- *Breaking change* Reworked the execution model. `run_main` and `cluster_main`
have been moved to `bytewax.testing` as they are only supposed to be used
when testing or prototyping.
Production dataflows should be ran by calling the `bytewax.run`
module with `python -m bytewax.run <dataflow-path>:<dataflow-name>`.
See `python -m bytewax.run -h` for all the possible options.
The functionality offered by `spawn_cluster` are now only offered by the
`bytewax.run` script, so `spawn_cluster` was removed.

- *Breaking change* `{Sliding,Tumbling}Window.start_at` has been
renamed to `align_to` and both now require that argument. It's not
possible to recover windowing operators without it.

- Fixes bugs with windows not closing properly.

- Fixes an issue with SQLite-based recovery. Previously you'd always
get an "interleaved executions" panic whenever you resumed a cluster
after the first time.

- Add `SessionWindow` for windowing operators.

- Add `SlidingWindow` for windowing operators.

- *Breaking change* Rename `TumblingWindowConfig` to `TumblingWindow`

- Add `filter_map` operator.

- *Breaking change* New partition-based input and output API. This
removes `ManualInputConfig` and `ManualOutputConfig`. See
`bytewax.inputs` and `bytewax.outputs` for more info.

- *Breaking change* `Dataflow.capture` operator is renamed to
`Dataflow.output`.

- *Breaking change* `KafkaInputConfig` and `KafkaOutputConfig` have
been moved to `bytewax.connectors.kafka.KafkaInput` and
`bytewax.connectors.kafka.KafkaOutput`.

- *Deprecation warning* The `KafkaRecovery` store is being deprecated
in favor of `SqliteRecoveryConfig`, and will be removed in a future
release.

0.15.0

- *Breaking change* Fixes issue with multi-worker recovery. If the
cluster crashed before all workers had completed their first epoch,
the cluster would resume from the incorrect position. This requires
a change to the recovery store. You cannot resume from recovery data
written with an older version.

0.14.0

- Dataflow continuation now works. If you run a dataflow over a finite
input, all state will be persisted via recovery so if you re-run the
same dataflow pointing at the same input, but with more data
appended at the end, it will correctly continue processing from the
previous end-of-stream.

- Fixes issue with multi-worker recovery. Previously resume data was
being routed to the wrong worker so state would be missing.

- *Breaking change* The above two changes require that the recovery
format has been changed for all recovery stores. You cannot resume
from recovery data written with an older version.

- Adds an introspection web server to dataflow workers.

- Adds `collect_window` operator.

0.13.1

- Added Google Colab support.

0.13.0

- Added tracing instrumentation and configurations for tracing backends.

0.12.0

- Fixes bug where window is never closed if recovery occurs after last
item but before window close.

- Recovery logging is reduced.

- *Breaking change* Recovery format has been changed for all recovery stores.
You cannot resume from recovery data written with an older version.

- Adds a `DynamoDB` and `Bigquery` output connector.

Page 3 of 4

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.