Threatingestor

Changelog

Bug Fixes
- Bug fix for GitHub configuration. Now allows the user to select a specific number of days since the creation date (`num_of_days` in `config.yml`) when searching for a repository. (https://github.com/InQuest/ThreatIngestor/issues/113)
- Bug fix for retweeted bodies not being properly ingested. Now when collecting artifacts, the retweet body should be included. (https://github.com/InQuest/ThreatIngestor/issues/114)

Features
- Added GitHub workflow for running tests when a new commit is pushed.
- Now includes a Docker build for running in a containerized environment.
- New ingestion stream included. Users can now run a search against GitHub gists, searching by username. (https://github.com/InQuest/ThreatIngestor/issues/88)

New build info and documentation can be found here: https://pypi.org/project/threatingestor/1.0.0b9/

* Added a whitelist feature for skipping common, non-malicious domains and hosts.

* Updated Twitter ingestion to support extended tweets.
* MISP upgrades.

Sixth beta release.

* Updated some ThreatKB plugin internals.

Fifth beta release.

**Note: This release has breaking changes!**

* Changed the names of Twitter credential keys for improved clarity (67).
* Added a warning to log output that mentions the above change, and points to these release notes. (Will be removed in a future release.)

Migration notes

If you used a Twitter plugin (source and/or operator) with a previous release (<1.0.0b5), you will need to update the Twitter credentials in your configuration YAML. The credential keys have been renamed as follows:

| Old name | Current name |
| ------------- | ------------------ |
| token | access_token |
| token_key | access_token_secret |
| con_secret_key | api_secret_key |
| con_secret | api_key |

Example old Twitter credentials :

yml
Don't do this!
- name: twitter-auth
token: MY_ACCESS_TOKEN
token_key: MY_ACCESS_TOKEN_SECRET
con_secret_key: MY_API_SECRET_KEY
con_secret: MY_API_KEY


Example new Twitter credentials:

yml
Do this instead!
- name: twitter-auth
api_key: MY_API_KEY
api_secret_key: MY_API_SECRET_KEY
access_token: MY_ACCESS_TOKEN
access_token_secret: MY_ACCESS_TOKEN_SECRET


The order in which you specify the keys in the configuration is not important. The order used in all new, updated examples is the same order in which the secrets are given in your Twitter developer account app details.

Fourth beta release.

* Fixed a bug in the git source where YARA rule files that were deleted from a repository caused an exception.