Streamparse

Latest version: v5.0.1

Safety actively analyzes 688554 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 6 of 8

3.0.0.dev0

This is the first developer preview release of streamparse 3.0. It has **not been tested extensively in production** yet, so we are looking for as much feedback as we can get from users who are willing to test it out.

You can install this release via pip with `pip install --pre streamparse==3.0.0.dev0`. It will not automatically install because it's a pre-release.

:warning: API Breaking Changes :warning:
- Topologies are now specified via a [Python Topology DSL](http://streamparse.readthedocs.org/en/master/topologies.html#topology-dsl) instead of the Clojure Topology DSL. This means you can/must now write your topologies in Python! Components can still be written in any language supported by Storm, of course. (Issues 84 and 136, PR 199, 226)
- The deprecated `Spout.emit_many` method has been removed. (pystorm/pystorm004dc27)
- As a consequence of using the new Python Topology DSL, all Bolts and Spouts that emit anything are expected to have the `outputs` attribute declared. It must either be a list of `str` or `Stream` objects, as described in the docs.
- We temporarily removed the `sparse run` command, as we've removed all of our Clojure code, and this was the only thing that had to still be done in Clojure. (Watch issue 213 for future developments)

Features
- Added `sparse slot_usage` command that can show you how balanced your topologies are across nodes. This is something that isn't currently possible with the Storm UI on its own. (PR 218)
- Can now specify `ssh_password` in `config.json` if you don't have SSH keys setup. Storing your password in plaintext is not recommended, but nice to have for local VMs. (PR 224, thanks motazreda)
- Now fully Python 3 compatible (and tested on up to 3.5), because we rely on [fabric3](https://pypi.python.org/pypi/Fabric3) instead of plain old [fabric](http://www.fabfile.org/) now. (4acfa2f)
- Now remove `_resources` directory after JAR has been created.

Other Changes
- Now rely on [pystorm](https://github.com/pystorm/pystorm) package for handling Multi-Lang IPC between Storm and Python. This library is essentially the same as our old `storm` subpackage with a few enhancements (e.g., the ability to use MessagePack instead of JSON to serialize messages). (Issue #174, Commits aaeb3e9 and 1347ded)
- All Bolt, Spout, and Topology-related classes are all available directly at the `streamparse` package level (i.e., you can just do `from streamparse import Bolt` now) (Commit b9bf4ae).
- `sparse kill` now will kill inactive topologies. (Issue 156)
- All examples now use the Python DSL
- The Kafka-JVM example has been cleaned up a bit, so now you can click on Storm UI log links and they'll work.

2.1.4

This minor release adds support for specifying `ui.port` in `config.json` to make the `sparse stats` and `sparse worker_uptime` commands work when `ui.port` is not set to the default 8080.

2.1.3

Fix a race condition in `TicklessBatchingBolt` that could cause a tuple to be part of more than one batch. (PR 193)

2.1.2

This release fixes an issue where `reraise` wasn't being imported from `six` in `bolt.py` (commit d743188).

2.1.1

This bugfix release just fixes an issue where `TicklessBatchingBolt` was crashing when trying to handle exceptions in `TicklessBatchingBolt.run()` (commit 48bace62da85000eb9e92b2dbe266b7229e0382e).

2.1.0

Features
- Added back an updated version of the pre-2.0 `BatchingBolt` that did not rely on tick tuples called `TicklessBatchingBolt`. This is useful in cases where you know your spout will not replay tuples after a topology shutdown. Because Storm is not guaranteed to continue to send tick tuples when the topology is shutting down, the standard `BatchingBolt` may have a batch of tuples waiting to be processed (that were never ACKed) sitting in it when the topology shuts down. When you resubmit and start it back up, those tuples will be lost unless the spout saves state between runs (which is pretty uncommon). With the `TicklessBatchingBolt` this is much less likely to happen because we use a timer thread which is independent of Storm, which will continue to execute even while the topology is shutting down. As long as the time you give Storm to shutdown is greater than the time it takes to process the last batch, your last batch will always be fully processed. (PR 191)
- Can now specify virtualenv command-line arguments in `config.json` via [`virtualenv_flags`](http://streamparse.readthedocs.org/en/master/quickstart.html?highlight=virtualenv_flags#disabling-configuring-virtualenv-creation) (issue 94, PR 159)
- Added support for pulling out `source->stream->fields` mapping with Storm 0.10.0+ (commit 61f163d)

Bug fixes
- Restored `--version` argument to `sparse` that was accidentally removed in previous release. (commit 48b6de7)
- Fixed missing comma in `setup.py` (issue 160, commit bde3cc3)
- Fixed issue where an empty `tasks.py` file (for invoke) was necessary to make fabric pre-submit hooks work. (issue 157, commit a10c478)
- Fixed issue where `run` and `submit` couldn't parse email addresses and git hashes properly (PR 189, thanks eric7j, commit 8670e3f)
- Fixed issue where fabric `env` wasn't being populated when `use_virtualenv` was False (commit a10c478)
- Fixed issue where updating virtualenvs would hang when VCS path changed. (commits e923a3c and 3e27cf0)

Documentation
- Added [documentation](http://streamparse.readthedocs.org/en/master/topologies.html#parallelism-and-workers) that explains how parallelism and workers work in Storm and streamparse. (issue 163, PR 165)
- Added documentation about [tuple routing and direct streams](http://streamparse.readthedocs.org/en/master/topologies.html#streams). (issue 162, commit fff05a0b3c8d91b2f255316e80315ebc6247c21e)
- Fixed some inconsistencies in the capitalization of `Bolt` and `Spout` in our docs (issue 164, PR 166)
- Embedded [PyCon 2015 presentation video](https://www.youtube.com/watch?t=94&v=ja4Qj9-l6WQ) in docs. (PR #161)
- Added some more [FAQs](http://streamparse.readthedocs.org/en/master/faq.html) to docs. (PR #88, thanks konarkmodi)

Depedencies
- `simplejson` is now a required dependency instead of just recommended (commit 08ef3d4)
- Updated `invoke` requirements to now require 0.8 or later (commit 360128c)
- Updated requirements to specify `six >= 1.5` specifically to avoid issues for OS X users (issues 113 and 190, commit a0c1309)

Page 6 of 8

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.