Synapse

--------------------

New Features
- 395 - Add formTufosByProps() API to the Cortex() class to do bulk tufo creation. This has significant performance for doing bulk tufo creation by utilizing a single storage layer transaction.
- 359, 405 - All splices now represent atomic actions. This is a breaking change from v0.0.21, since the node:set events no longer multiple props, but instead a single prop per event. This requires an ecosystem wide upgrade for users utilizing the splice subsystem.
- 408 - Provisional task() operator added to the storm runtime. This fires events in the form of "task:<name>" on the storm core eventbus. This behavior may change in the future.
- 409, 414, 416 - Synapse model properties which had req=1 set on them now actually require that property to be present when model enforcement is in place. This check occurs after node:form events, allowing code to hook the node formation process and provide required properties or set them as needed.
- 417 - Added a tree() operator to storm for doing recursive pivot operations on a set of nodes.
- 419 - Added a delprop() operator to storm to remove properties from nodes.

Enhancements
- 410 - pivot() operator syntax updated to match that of the macro syntax operator. This may be a breaking change for any programmatic use of the pivot operator syntax.
- 411 - Added pycodestyle checks to CI builds to identify code style issues.
- 412 - Fix resourcewarnings with unclosed file handles.

Bugs
- 399 - Fix Cortex.__init__() on Python 2.7 where a list comprehension smashed function locals. This prevented configable options from being set at initialization.
- 401 - Fix setTufoProps() to add nodes created from secondary properties when appropriate.

Documentation
- 403 - Initial docs for Ingest subsystem,
- 408, 420 - Updates to Storm runtime documentation.
- 418 - Added markdown template to release documentation.

--------------------

The 0.0.20 release is not small - so here are some notes for it!

Since there are significant changes to how we handle models and storage layers in this version, we recommend that you make a backup of any production Cortexes you have before deploying this, and test your systems prior to deployment.

-- vEpiphyte

New Features

- 277 - Add the ability to load python modules via dmon.
- 275 - All Synapse core models have been cutover to using CoreModule implementations for doing model revisioning.
- 274 - Optional Time boundaries added to tags for doing timeboxing of tags.
- 279 - Added columns support and additional help framework for cmdr CLI
- 285 - Easycert tool now makes server certificates with the subjectAltName (SAN) field populated. Tell your fiends - this a way easier tool to use to make SAN certs than using the openssl binary itself. This allows certificate pairs to be used in chrome 58+.
- 177 - Added a ndef() function to synapse.lib.tufo to get a tufo type and primary prop.
- 286 - Easycert tool can make PKCS12 client certs.
- 333 - Comp types now support optional kw fields, allowing recording of varying levels of knowledge for a given type.
- 321 - SSL Support added to the webapp
- 289, 290, 301 - Add support for a blob key/value store to the Cortex which exists separately from the Row layer storage.
- 291, 292, 301 - Add support for storage layers to be revisioned independently of models.
- 300, 342 - Add support for sending BODY content via remcycle
- 348, 350 - Add a guid() operator to storm for lifting a node by iden.
- 358 - Add a delTufoProp() API to cortex for deleting a tufo property. This changed splice contention and event handlers, node:set is no longer fired; node:prop:set and node:prop:del are now fired.
- 374, 378 - Add a new limit() operator for Storm.
- 320, 362 - Storage layers are now separated from the Cortex class by an API boundary. This allows for future custom storage layers to be implemented easily.
- 319, 262 - Add dumprows and loadrows tools to dump a cortex to a savefile or create a new storage object from a existing savefile.

Improvements
- 282 - Code style cleanup
- 293 - Prevent reference databases used in tests from being modified accidently.
- 288, 296, 332 - Storm setprop operator (and prop edit mode) now respect relative prop values. Addnode also uses relative props now.
- 272, 342 - Rewrote remcycle tests to no longer require external resources. They run consistently now.
- 302, 342 - Remcycle now uses configable definitions in a consistent manner.
- 364 - Restored pre v0.0.15 axon path behaviors.
- 378 - Add a plan step to storm query parsing - allows for future optimizations.
- 338, 362 - Configable objects may now use a decorator to declare all of their options, which will be loaded at runtime.

Bugfixes
- 276 - Fixes for eq/lt/le/gt/ge operators in storm
- 236, 295 - Allow cache disable on a cortex to actually clear the cache.
- 287, 294 - Fix delnode storm operator and delTufo() API
- 347, 349 - File:bytes nodes created from seed ctors (file:bytes:md5, file:bytes:sha1, etc) now have stable guids. Previously guids were case sensitive.
- 335, 352 - Fix cmdr quit function on Python 2.7
- 345, 351 - Fix inet:cidr range lookups.
- 367, 370 - Regex macro filter "~=" accidently ate whitespace. This is fixed.
- 380, 382 - Fix a issue with tag caches upon tag deletion.
- 378 - Fix for comp type arg parsing in long form storm operators.
- 339, 346 - Fix for inet:url:ipv4 and inet:url:fqdn parsing.
- 354, 355 - Fix for inet:tcp4 / inet:udp4 :ipv4 and :port parsing. Also make ipv6 repr's consistent.

Documentation
- 273, 278 - Initial performance benchmarks for Synapse
- 281, 283 - Initial User Guide for Synapse
- 284 - Change docs to using the easier to read RTD theme.
- 368 - ADditional User Guide documentation.
- 338, 362 - Automatic documentation is now generator for configable objects, detailing their options.
- 323, 324, 362 - Synapse devops documentation moved together.