Synapse

====================

Features and Enhancements
-------------------------
- Update the allowed versions of the ``pyopenssl`` and ``pytz`` libraries.
(`2657 <https://github.com/vertexproject/synapse/pull/2657>`_)
(`2658 <https://github.com/vertexproject/synapse/pull/2658>`_)

Bugfixes
--------
- When setting ival properties, they are now properly merged with existing
values. This only affected multi-layer views.
(`2655 <https://github.com/vertexproject/synapse/pull/2655>`_)

====================

Bugfixes
--------
- Fix a parsing regression in inet:url nodes related to unencoded "" symbols
in URLs.
(`2653 <https://github.com/vertexproject/synapse/pull/2653>`_)

====================

Features and Enhancements
-------------------------
- Updates to the ``inet`` and ``infotech`` models.
(`2634 <https://github.com/vertexproject/synapse/pull/2634>`_)
(`2644 <https://github.com/vertexproject/synapse/pull/2644>`_)
(`2652 <https://github.com/vertexproject/synapse/pull/2652>`_)

``inet:url``
The ``inet:url`` type now recognizes various ``file:///`` values from
RFC 8089.

``it:sec:cve``
The ``it:sec:cve`` type now replaces various Unicode dashes with hyphen
characters when norming. This allows a wider range of inputs to be
accepted for the type. Scrape related APIs have also been updated to
match on this wider range of inputs.

- The Cell now uses ``./backup`` as a default path for storing backups in, if
the ``backup:dir`` path is not set.
(`2648 <https://github.com/vertexproject/synapse/pull/2648>`_)
- Add POSIX advisory locking around the Cell ``cell.guid`` file, to prevent
multiple processes from attempting to start a Cell from the same directory.
(`2642 <https://github.com/vertexproject/synapse/pull/2642>`_)
- Change the default ``SLAB_COMMIT_WARN`` time from 5 seconds to 1 second, in
order to quickly identify slow storage performance.
(`2630 <https://github.com/vertexproject/synapse/pull/2630>`_)
- Change the Cell ``iterBackupArchive`` and ``iterNewBackupArchive`` routines
to always log exceptions they encounter, and report the final log message
at the appropriate log level for success and failure.
(`2629 <https://github.com/vertexproject/synapse/pull/2629>`_)
- When normalizing the ``str`` types, when ``onespace`` is specified, we skip
the ``strip`` behavior since it is redundant.
(`2635 <https://github.com/vertexproject/synapse/pull/2635>`_)
- Log exceptions raised by Cell creation in ``initFromArgv``. Catch
``lmdb.LockError`` when opening a LMDB database and re-raise an exception
with a clear error message.
(`2638 <https://github.com/vertexproject/synapse/pull/2638>`_)
- Update schema validation for Storm packages to ensure that cmd arguments do
not have excess fields in them.
(`2650 <https://github.com/vertexproject/synapse/pull/2650>`_)

Bugfixes
--------
- Adjust comma requirements for the JSON style list and dictionary expressions
in Storm.
(`2636 <https://github.com/vertexproject/synapse/pull/2636>`_)
- Add Storm query logging in a code execution path where it was missing.
(`2647 <https://github.com/vertexproject/synapse/pull/2647>`_)
- Tuplify the output of ``synapse.tools.genpkg.loadPkgProto`` to ensure that
Python list constructs ``[...]`` do not make it into Power-Up documentation.
(`2646 <https://github.com/vertexproject/synapse/pull/2646>`_)
- Fix an issue with heavy Stormtypes objects where caching was preventing
some objects from behaving in a dynamic fashion as they were intended to.
(`2640 <https://github.com/vertexproject/synapse/pull/2640>`_)
- In norming ``int`` values, when something is outside of the minimum or
maximum size of the type, we now include the string representation of the
valu instead of the raw value.
(`2643 <https://github.com/vertexproject/synapse/pull/2643>`_)
- Raise a ``NotReady`` exception when a client attempts to resolve an
``aha://`` URL and there have not been any ``aha`` servers registered.
(`2645 <https://github.com/vertexproject/synapse/pull/2645>`_)

Improved Documentation
----------------------
- Update Storm command reference to add additional commands.
(`2633 <https://github.com/vertexproject/synapse/pull/2633>`_)
- Expand Stormtypes API documentation.
(`2637 <https://github.com/vertexproject/synapse/pull/2637>`_)
(`2639 <https://github.com/vertexproject/synapse/pull/2639>`_)

====================

Features and Enhancements
-------------------------
- Updates to the ``meta`` and ``infotech`` models.
(`2624 <https://github.com/vertexproject/synapse/pull/2624>`_)

``meta:rule``
Add a new form for generic rules, which should be linked to
the nodes they match with a ``matches`` light edge.

``meta:ruleset``
Add ``:author``, ``:created``, and ``:updated`` secondary properties.

``it:app:yara:rule``
Add ``:created`` and ``:updated`` secondary properties.

- Add a new Docker image ``vertexproject/synapse-jsonstor``.
(`2627 <https://github.com/vertexproject/synapse/pull/2627>`_)

- Allow passing a version requirement string to ``$lib.import()``.
(`2626 <https://github.com/vertexproject/synapse/pull/2626>`_)

Bugfixes
--------
- Fix an issue where using a regex lift on an array property could
incorrectly yield the same node multiple times.
(`2625 <https://github.com/vertexproject/synapse/pull/2625>`_)

Improved Documentation
----------------------
- Update documentation regarding mirroring to be clearer about
whether a given cell supports it.
(`2619 <https://github.com/vertexproject/synapse/pull/2619>`_)

====================

Features and Enhancements
-------------------------
- Update the ``meta`` model.
(`2621 <https://github.com/vertexproject/synapse/pull/2621>`_)

``meta:ruleset``
Add a new form to denote the collection of a set of nodes representing
rules, which should be linked together with a ``has`` light edge.

- Add additional filter options for the Storm ``merge`` command.
(`2615 <https://github.com/vertexproject/synapse/pull/2615>`_)
- Update the ``BadSyntaxError`` exception thrown when parsing Storm queries to
additionally include line and column when available. Fix an issue
where a ``!`` character being present in the exception text could truncate
the output.
(`2618 <https://github.com/vertexproject/synapse/pull/2618>`_)

====================

Automatic Migrations
--------------------
- Re-normalize the ``geo:place:name``, ``crypto:currency:block:hash``, and
``crypto:currency:transaction:hash`` values to account for their modeling
changes. Migrate ``crypto:currency:transaction:input`` and
``crypto:currency:transaction:output`` values to the secondary properties
on the respective ``crypto:payment:input`` and ``crypto:payment:output``
nodes to account for the modeling changes. Make ``geo:name`` nodes for
``geo:place:name`` secondary properties to account for the modeling changes.
See :ref:`datamigration` for more information about automatic
migrations.

Features and Enhancements
-------------------------
- Several updates for the ``crypto``, ``geospace``, ``inet``, and ``meta``
models.
(`2594 <https://github.com/vertexproject/synapse/pull/2594>`_)
(`2608 <https://github.com/vertexproject/synapse/pull/2608>`_)
(`2611 <https://github.com/vertexproject/synapse/pull/2611>`_)
(`2616 <https://github.com/vertexproject/synapse/pull/2616>`_)

``crypto:payment:input``
Add a secondary property ``:transaction`` to denote the transaction
for the payment.

``crypto:payment:output``
Add a secondary property ``:transaction`` to denote the transaction
for the payment.

``crypto:currency:block``
Change the type of the ``:hash`` property from a ``0x`` prefixed ``str``
to a ``hex`` type.

``crypto:currency:transaction``
Change the type of the ``:hash`` property from a ``0x`` prefixed ``str``
to a ``hex`` type.
Deprecate the ``:inputs`` and ``:outputs`` secondary properties.

``geo:place``
Change the type of the ``:name`` secondary property to ``geo:name``.

``inet:web:channel``
Add a new form to denote a channel within a web service or instance.

``inet:web:instance``
Add a new form to track an instance of a web service, such as a channel
based messaging platform.

``inet:web:mesg``
Add ``:channel``, ``:place``, and ``:place:name`` secondary properties.

``inet:web:post``
Add ``:channel`` and ``:place:name`` secondary properties.

``meta:event``
Add a new form to denote an analytically relevant event in a curated
timeline.

``meta:event:taxonomy``
Add a new form to represent a taxonomy of ``meta:event:type`` values.

``meta:timeline``
Add a new form to denote a curated timeline of analytically relevant
events.

``meta:timeline:taxonomy``
Add a new form to represent a taxonomy of ``meta:timeline:type`` values.

- Add support for ``$lib.len()`` to count the length of emitter or generator
functions.
(`2603 <https://github.com/vertexproject/synapse/pull/2603>`_)
- Add support for scrape APIs to handle text that has been defanged with
``\\.`` characters.
(`2605 <https://github.com/vertexproject/synapse/pull/2605>`_)
- Add a ``nomerge`` option to View objects that can be set to prevent merging
a long lived fork.
(`2614 <https://github.com/vertexproject/synapse/pull/2614>`_)
- Add ``liftByProp()`` and ``liftByTag()`` methods to the Stormtypes
``layer`` objects. These allow lifting of nodes based on data stored
in a specific layer.
(`2613 <https://github.com/vertexproject/synapse/pull/2613>`_)
- Expand Synapse requirements to include updated versions of the ``pygments``
library.
(`2602 <https://github.com/vertexproject/synapse/pull/2602>`_)

Improved Documentation
----------------------
- Fix the example regular expressions used in the ``$lib.scrape.genMatches()``
Storm library API examples.
(`2606 <https://github.com/vertexproject/synapse/pull/2606>`_)