Spamscope

Highlights

- New Travis CI configuration. Added automatic deploy.
- Fixed Elasticsearch template (`index.mapping.ignore_malformed`).
- Added `Dockerfile` example: debug images.
- Upgraded `mail-parser` and `streamparse`.
- New field `to_domains` in main report: domains of `to` header.
- Bugfix

Highlights
- More stable version. Solved three bug: `heartbeat timeout` when using Thug, getting custom header now not raise `pystorm/serializers/json_serializer.py` and you can avoid to set blacklist or whitelist in configuration.
- Removed `waiting.sleep` in configuration spout file to avoid Apache Storm timeout.
- `Thug` section has more important options.
- Added `debug` environment in `config.json`.
- Added `docker-compose` example for debug use.
- Splitted `requirements` in `optional` and `not`.
- Fixed Thug analysis integration: replaced `os.kill` with `raise`. Added more options.

Highlights
- Upgraded requirements
- Upgraded `mail-parser`
- Upgraded `Apache Tika` support to `1.16` version
- New installer that manages all Python dependencies. SpamScope is on [PyPI](https://pypi.python.org/pypi/SpamScope)
- Command line `spamscope-elasticsearch` can be used in the cases where Elasticsearch is behind a reverse proxy. You can use RFC-1738 formatted URLs.

:warning: Breaking Changes :warning:
- The mail format is different. See [mail-parser](https://github.com/SpamScope/mail-parser/releases/tag/v3.0.0) for more details.

Highlights

- Support `Outlook` mail (msg format). Enable flag `outlook: True` in mailboxes main configuration for folders that have this mail format.
- Added [SpamAssassin](http://spamassassin.apache.org/) post processing. Every mails can be analyzed from SpamAssassin. Be careful the parsing can be very slow.
- Now you can extract custom headers from mail with `headers:` list in mailboxes main configuration (see configuration example in this repository).
- Merged urls bolts of body and attachments in one bolt.
- Fixed Elasticsearch template.
- Upgraded `mail-parser` and `streamparse`.
- Added headers `receiveds` in main output.
- The form in mails now are analyzed from `phishing` bolt.
- New code to analyze raw mail with third part tools.
- The `binary` attachments have a flag `binary`.
- SpamScope can extract samples from mailformed `zip` attachments.
- SpamScope can extract samples from `quoted-printable` headers mail.

Highlights

- Upgraded requirements: `Apache Tika`, `Streamparse` and `Shodan`
- Resolved bug _JsonSerializer_ in Shodan report. [Issue 368 Streamparse](https://github.com/Parsely/streamparse/issues/368issuecomment-305873056)
- Resolved bug in Elasticsearch template to manage Shodan report
- Added `.editorconfig` to setup editor for SpamScope

Highlights

- Auto build docker images with Travis.
- Changed function to make base64 decode.
- Manage incorrect padding errors for base64 attachments and report in results.