Pymisp

---------------------

New
~~~
- Add import script for dxy data. [Raphaël Vinot]
- Csse covid19 daily report importer. [Raphaël Vinot]

Changes
~~~~~~~
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]
- Bump changelog. [Raphaël Vinot]
- Bump dependencies. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- JSON files are UTF8. [Raphaël Vinot]

Bump dev deps, update comment
- Add tag, set distribution, add file and source (CSSE importer)
[Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]

---------------------

New
~~~
- Add uuid by default in MISPEvent, add F/L seen in feed output.
[Raphaël Vinot]
- Admin script to setup a sync server. [Raphaël Vinot]
- Add feed generation example in notebook. [Raphaël Vinot]

Changes
~~~~~~~
- Bump changelog. [Raphaël Vinot]
- Comments were still referencing pipenv. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Bump dependencies. [Raphaël Vinot]
- Bump dep. [Raphaël Vinot]
- Fix typo in readme. [Raphaël Vinot]
- Use bionic on travis. [Raphaël Vinot]
- Add poetry support. [Raphaël Vinot]
- Bump changelog. [Raphaël Vinot]
- Bump objects. [Raphaël Vinot]
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]

Fix
~~~
- Test cases & template version. [Raphaël Vinot]
- Mypy, more typing. [Raphaël Vinot]
- Do not skip data in add_attribute methods. [Raphaël Vinot]
- Remove references to the old API. [Raphaël Vinot]
- Make lief optional again. [Raphaël Vinot]

fix 538

Other
~~~~~
- Use poetry everywhere, fix readme. [Raphaël Vinot]

---------------------

New
~~~
- Add includeDecayScore to rest search. [VVX7]
- Support for first_seen/last_seen. [Raphaël Vinot]

Cleaner import of datetime
- [attributes] chrome-extension-id added. [Alexandre Dulaunoy]

Changes
~~~~~~~
- Bump version. [Raphaël Vinot]
- Do not install neo by default. [Raphaël Vinot]
- Bump objects. [Raphaël Vinot]
- More flexible when an event is in a weird state. [Raphaël Vinot]
- Str to int, properly load SharingGroup. [Raphaël Vinot]

Fix 535
- Bump deps, add pep8 test. [Raphaël Vinot]
- Bump objects. [Raphaël Vinot]
- Support dict in tag/untag. [Raphaël Vinot]
- Test update last seen. [Raphaël Vinot]
- Add test cases in feed. [Raphaël Vinot]
- Add test cases. [Raphaël Vinot]
- Normalize to_datetime conversion. [Raphaël Vinot]
- Trustar example uses objects. [Raphaël Vinot]
- Add lief in the generic requirements. [Raphaël Vinot]
- Refactorize typing, validate. [Raphaël Vinot]

Fix
~~~
- Bump objects. [Raphaël Vinot]
- Issue with readme. [Raphaël Vinot]
- Remove debugging. [Raphaël Vinot]
- [*-seen] Consider that `-` can also be in the date component while
parsing. [mokaddem]
- First seen was after last seen, trigerring the exception. [Raphaël
Vinot]
- Tests failing if local tz was not CET. [Raphaël Vinot]
- Syntax and typos. [Raphaël Vinot]
- Bugs introduced by last commit. [Raphaël Vinot]

Other
~~~~~
- Doc: fix Search-FullOverview.ipynb code example. [Bernhard E. Reiter]
- Chore: delete old examples. [Manabu Niseki]

Delete examples which use deprecated/deleted methods
- Scrape trustar intel platform reports and create misp events.
[th3jiv3r]
- Configuration for trustar integration. [th3jiv3r]
- Fixed trailing lines. [turtlefac3]
- Fixed trailing lines. [turtlefac3]
- Custom integration written in python to scrape Proofpoint VAP API for
metrics of top Very Attacked Persons and create MISP events.
[turtlefac3]
- Fix typos on FullOverview.ipynb. [Bernhard E. Reiter]

---------------------

New
~~~
- [attribute type] kusto-query attribute type. [Alexandre Dulaunoy]

Kusto query is the query language for the Kusto services in Azure used
to search large dataset. It's used in Windows Defender ATP Hunting-Queries
and also Azure Sentinel (Cloud-native SIEM).
- Remove python < 3.6 support. [Raphaël Vinot]
- URLObject (requires pyfaup) [Raphaël Vinot]

Changes
~~~~~~~
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]
- Bump Changelog. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Bump dependencies, add debug. [Raphaël Vinot]
- Upate dummy events creator. [Raphaël Vinot]
- Add tests on more version of Python. [Raphaël Vinot]
- Search with the STIX output returns a json STIX. [Raphaël Vinot]

Was XML before.
- Bump dependencies. [Raphaël Vinot]
- Add more typing information. [Raphaël Vinot]
- Add typing markup. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Bump Dependencies. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Bump changelog. [Raphaël Vinot]
- Version bump. [Raphaël Vinot]
- Bump test files. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Debug travis error message. [Raphaël Vinot]
- [types] eppn type added. [Alexandre Dulaunoy]
- Fix typo. [Raphaël Vinot]
- Move scrippsco2 feed generator to a sub directory. [Raphaël Vinot]
- Update documentation. [Raphaël Vinot]

Fix 396
- Bump objects. [Raphaël Vinot]

Fix
~~~
- Bump template_version in test cases. [Raphaël Vinot]
- Add missing variable in dummy creator. [Raphaël Vinot]
- Et2misp was python2 only. [Raphaël Vinot]
- Feed generator was broken. [Raphaël Vinot]

Fix 506
- Event without hashable attribute. [Raphaël Vinot]

Related 506
- Properly test custom objects. [Raphaël Vinot]
- Adding a sighting takes a little bit of time. [Raphaël Vinot]
- Test case on reference. [Raphaël Vinot]
- Add missing fields to event & attribute for the feed output. [Raphaël
Vinot]
- Make sure the publish timestamp is bumped on update. [Raphaël Vinot]

Other
~~~~~
- Update api.py. [AaronK]

minor typo, can;t help it noticing those. sorry,
- Fixed TODO, added quarantineFolder/quarantineRule from
messagesBlocked, added some error handling to prevent empty attributes
from trying to be added. [th3jiv3r]
- Scrape proofpoint tap api for messages blocked/delivered & clicks
blocked/permitted and create misp events. [th3jiv3r]
- Add variable for proofpoint tap api auth. [th3jiv3r]
- Update README.md. [AaronK]

minor typo
- Define the number of entries to output. [AndreC10002]

Allow for defining in the settings.py file the number of entries to output
- Update generate.py. [AndreC10002]
- Cleanup of code and 'quick-n-dirty' sanitizing of tags. [Koen Van
Impe]
- Sync. [Koen Van Impe]
- Update README.md. [Raphaël Vinot]

---------------------

New
~~~
- Script to generate the metadata of a feed out of a directory. [Raphaël
Vinot]
- Add to_feed export to MISPEvent. [Raphaël Vinot]
- Validate object templates. [Raphaël Vinot]

fix https://github.com/MISP/misp-objects/issues/199
- Test cases for restricted tags. [Raphaël Vinot]

Fix 483
- Get Database Schema Diagnostic. [Raphaël Vinot]

Fix 492
- Add support for UserSettings. [Raphaël Vinot]

Changes
~~~~~~~
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]
- Bump dependencies. [Raphaël Vinot]
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]
- Bump dependencies. [Raphaël Vinot]
- Require stable version of lief again. [Raphaël Vinot]
- Few more improvements on the feed export. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Make the feed generator more generic. [Raphaël Vinot]
- Use New version of PyMISP in the feed generator. [Raphaël Vinot]
- Bump misp-object. [Raphaël Vinot]
- Allow to sort and indent the json output for objects. [Raphaël Vinot]
- Bump objects. [Raphaël Vinot]
- Bump dependencies. [Raphaël Vinot]
- [test] feed test updated as botvrij is now TLS by default. [Alexandre
Dulaunoy]
- Bump changelog. [Raphaël Vinot]
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Use default category from template. [Raphaël Vinot]

Fix 477
- Skip usersettings tests when emails are disabled. [Raphaël Vinot]

Fix
~~~
- Bump lief to 0.10.1. [Raphaël Vinot]
- Update tests. [Raphaël Vinot]
- Raise PyMISPError instead of Exception. [Raphaël Vinot]
- Rename feed_meta_generator so it clearly fails with python<3.6.
[Raphaël Vinot]
- Improve stability of feed output. [Raphaël Vinot]
- Do not unitialize the uuid in MISPEvent. [Raphaël Vinot]
- Bump url template version in test cases. [Raphaël Vinot]
- Python 2.7 tests. [Raphaël Vinot]
- Print the full json blob in debug mode. [Raphaël Vinot]

Related https://github.com/MISP/PyMISP/issues/462
- Avoid exception on legacy MISP. [Raphaël Vinot]
- [examples] typo uuid. [Jean-Louis Huynen]

give me a hoodie.
- Prevents exception when lief is not installed. [Christophe Vandeplas]
- Python <3.4 should work again.... [Raphaël Vinot]

Fix 482
- Remote_describe_types response was invalid. [Raphaël Vinot]
- Missing file in last commit. [Raphaël Vinot]
- Remove overwrite of remote_describe_types. [Raphaël Vinot]

Other
~~~~~
- Cch: Bump misp-objects. [Raphaël Vinot]
- Added example for checking sync servers. [wotschel]
- Corrected docstring. [Shortfinga]
- Include to_ids and replace newlines in title. [Koen Van Impe]
- Update aping.py. [ater49]

Just fixing a typo
- Remove unused MISPFileCache from PyMISP class. [Marc Hoersken]

---------------------

New
~~~
- Better handling of delete(d) attributes. [Raphaël Vinot]

* Hard delete on attribute
* Get the deleted attributes within an event

Changes
~~~~~~~
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]
- Test if json exists in cached method. [Raphaël Vinot]
- Decode datetime without dateutils if possible. [Raphaël Vinot]
- Add support for rapidjson, refactoring and code cleanup. [Raphaël
Vinot]
- Cleanups. [Raphaël Vinot]
- Cleanups and improvements. [Raphaël Vinot]
- [types] updated to the latest version. [Christophe Vandeplas]

now using the gen_misp_types_categories using jq
- [describeTypes] updated to the latest version. [Alexandre Dulaunoy]
- Bump dependencies. [Raphaël Vinot]
- Add missing return formats in restsearch, bump objects. [Raphaël
Vinot]
- [misp-objects] updated to the latest version. [Alexandre Dulaunoy]
- Update search examples. [Raphaël Vinot]
- Update main notebook. [Raphaël Vinot]
- [test] remove attribute field which was not foreseen in 2.4 branch.
[Alexandre Dulaunoy]
- Fix travis tests due to sighting_timestamp. [Raphaël Vinot]
- Use default for warnings. [Raphaël Vinot]

fix: 453
- Dump dependencies, update tests. [Raphaël Vinot]
- Bump readme. [Raphaël Vinot]
- Update upload malware/attachment example script. [Raphaël Vinot]

Fix 447

Make data at attibute level more generic with getter/setter methods

Fix
~~~
- [Python2] Use LRU cache decorator, fix call to describe_types in
PyMISP. [Raphaël Vinot]
- Python2 SyntaxError... [Raphaël Vinot]
- Objects helpers were broken, do not overwrite describe_types. [Raphaël
Vinot]
- Support for legacy python versions. [Raphaël Vinot]

90 days and counting, folks.
- Cache object templates at AbstractMISP level. [Raphaël Vinot]

Related 468 and 471
- Cache describeTypes at AbstractMISP level. [Raphaël Vinot]
- Big speed improvment when loading MISPEvent. [Raphaël Vinot]

1. `properties` is a list comprehension
2. Massively reduce the amount of calls to `properties`
- Python 2.7 support. [Raphaël Vinot]

I want a cookie.

Other
~~~~~
- Use classmethod instead of staticmethod and avoid hard-coded
reference. [Marc Hoersken]
- Cache JSON definitions in memory LFU cache provided by cachetools.
[Marc Hoersken]

- Path and modified time of JSON file are used as the cache key
- Global state is hidden away inside a root-class for re-use
- Maximum size is 150 considering the number of JSON definitions

During my tests the memory usage of the test suites was halved.
- Fix mixed whitespace in the travis helper script files. [Marc
Hoersken]
- Remove explicit clonce as the viper-test-files are now a Git
submodule. [Marc Hoersken]
- Add viper-test-files repository as Git submodule. [Marc Hoersken]
- Update .gitignore to exclude files produced during tests. [Marc
Hoersken]
- Code cleanup. [Koen Van Impe]
- Update type and code cleanup. [Koen Van Impe]
- List all the sightings - show_sightings.py. [Koen Van Impe]
- Disable to_ids based on false positive sightings reporting. [Koen Van
Impe]
- Adds support to add local tags. [Antoine Cailliau]

Requires https://github.com/MISP/MISP/pull/5215 to be merged first.
- Minor grammar errors. [Miroslav Stampar]
- Make client_certs out of the box friendly. [Campbell McKenzie]