Synapse

=====================

Automatic Migrations
--------------------
- Re-normalize ``risk:mitigation:name``, ``it:mitre:attack:technique:name``,
and ``it:mitre:attack:mitigation:name`` secondary properties.
(`3585 <https://github.com/vertexproject/synapse/pull/3585>`_)
- Re-normalize ``velocity`` properties which are float values.
(`3616 <https://github.com/vertexproject/synapse/pull/3616>`_)
- See :ref:`datamigration` for more information about automatic migrations.

Model Changes
-------------
- Add a new model, ``sci``, for modeling elements of the scientific method. Updates to
the ``econ``, ``file``, ``infotech``, ``inet``, ``ou``, ``ps``, and ``risk``
models.
(`3559 <https://github.com/vertexproject/synapse/pull/3559>`_)
(`3585 <https://github.com/vertexproject/synapse/pull/3585>`_)
(`3595 <https://github.com/vertexproject/synapse/pull/3595>`_)
(`3604 <https://github.com/vertexproject/synapse/pull/3604>`_)
(`3606 <https://github.com/vertexproject/synapse/pull/3606>`_)
(`3622 <https://github.com/vertexproject/synapse/pull/3622>`_)
(`3635 <https://github.com/vertexproject/synapse/pull/3635>`_)
- See :ref:`userguide_model_v2_165_0` for more detailed model changes.

Features and Enhancements
-------------------------
- Change the compression mode used when streaming Cell backups to speed up
the backup process.
(`3608 <https://github.com/vertexproject/synapse/pull/3608>`_)
- When a Cell is mirroring, gracefully go into read-only mode if the leader is
a greater version than the mirror.
(`3581 <https://github.com/vertexproject/synapse/pull/3581>`_)
(`3631 <https://github.com/vertexproject/synapse/pull/3631>`_)
- Add ``null`` as a constant that can be used in Storm expression syntax.
(`3600 <https://github.com/vertexproject/synapse/pull/3600>`_)
- Add ``cortex.storm.pool.get``, ``cortex.storm.pool.set``, and
``cortex.storm.pool.del`` commands to manage the Storm query pool which may
be used by the Cortex. This replaces the experimental support added in
``v2.160.0`` for Storm query pool configuration. The experimental Cortex
configurations options ``storm:pool``, ``storm:pool:timeout:sync``, and
``storm:pool:timeout:connection`` have been removed.
(`3602 <https://github.com/vertexproject/synapse/pull/3602>`_)
- Add ``$lib.regex.escape()`` API for escaping strings which may be used as
regular expression patterns.
(`3605 <https://github.com/vertexproject/synapse/pull/3605>`_)
- Add ``View.setMergeComment()`` and ``View.setMergeVoteComment()`` Storm APIs
for setting comments on merge requests and merge votes.
(`3597 <https://github.com/vertexproject/synapse/pull/3597>`_)
- Add handlers to the ``float``, ``int``, and ``str`` types to handle norming
Storm ``Number`` objects.
(`3601 <https://github.com/vertexproject/synapse/pull/3601>`_)
- Add a new Storm command, ``gen.geo.place``, to generate a ``geo:place`` node
by name.
(`3620 <https://github.com/vertexproject/synapse/pull/3620>`_)
- Add an optional reporter name argument to the Storm command
``gen.risk.vuln``.
(`3628 <https://github.com/vertexproject/synapse/pull/3628>`_)
- Add a ``norm`` option to the ``$node.difftags()`` command.
(`3612 <https://github.com/vertexproject/synapse/pull/3612>`_)
- Add logging around the leader promotion and handoff actions.
(`3615 <https://github.com/vertexproject/synapse/pull/3615>`_)
- Add Telepath APIs to AHA for clearing unused provisioning information.
(`3607 <https://github.com/vertexproject/synapse/pull/3607>`_)

Bugfixes
--------
- Fix a bug where Cortex Cron jobs could start prior to data migrations
having completed running.
(`3610 <https://github.com/vertexproject/synapse/pull/3610>`_)
- Fix an issue where ``node.prop.set`` and ``node.prop.del`` permissions were
not being properly checked.
(`3627 <https://github.com/vertexproject/synapse/pull/3627>`_)
- Fix a bug in the Storm ``merge`` command where the destination layer was
not being properly checked for property set and deletion permissions.
(`3627 <https://github.com/vertexproject/synapse/pull/3627>`_)
- Fix a bug in the Storm ``copyto`` command where the destination layer was
not being properly checked for property set permissions.
(`3641 <https://github.com/vertexproject/synapse/pull/3641>`_)
- Fix an error when granting a role admin permissions on a vault.
(`3603 <https://github.com/vertexproject/synapse/pull/3603>`_)
- Prevent the ``synapse.tools.easycert`` tool from making certificates with
names greater than 64 characters in length. Prevent AHA provisioning from
creating provisioning requests which would exceed that length.
(`3609 <https://github.com/vertexproject/synapse/pull/3609>`_)
- Fix an issue with the ``velocity`` base type returning a float instead
of an integer when handling a string value without a unit.
(`3616 <https://github.com/vertexproject/synapse/pull/3616>`_)
- Fix an issue that could occur when pivoting from a secondary property to
a form when using variables for the source and target values.
(`3618 <https://github.com/vertexproject/synapse/pull/3618>`_)
- Fix a syntax parsing issue when using the try-set-plus or try-set-minus
operator to update an array property on a node using a variable for the
property name.
(`3630 <https://github.com/vertexproject/synapse/pull/3630>`_)
- Fix an issue with AHA service pools where their Telepath Clients were
not configured for use as ``aha://`` clients.
(`3643 <https://github.com/vertexproject/synapse/pull/3643>`_)
- Fix an issue with AHA service pools where a fini'd Proxy was not properly
cleaned up.
(`3645 <https://github.com/vertexproject/synapse/pull/3645>`_)

Improved Documentation
----------------------
- Update Storm pivot documentation to add additional examples.
(`3599 <https://github.com/vertexproject/synapse/pull/3599>`_)
- Update the Cortex deployment guide to include a step to configure a
Storm query pool.
(`3602 <https://github.com/vertexproject/synapse/pull/3602>`_)

Deprecations
------------
- The tool ``synapse.tools.cellauth`` has been marked as deprecated and will
be removed in ``v3.0.0``.
(`3587 <https://github.com/vertexproject/synapse/pull/3587>`_)
- The tool ``synapse.tools.cmdr`` has been marked as deprecated and will
be removed in ``v3.0.0``.
(`3589 <https://github.com/vertexproject/synapse/pull/3589>`_)
- The Storm ``$lib.model.edge`` APIs have been marked as deprecated and will
be removed in ``v3.0.0``.
(`3623 <https://github.com/vertexproject/synapse/pull/3623>`_)
- The ``CoreAPI.enableMigrationMode()`` and ``CoreAPI.disableMigrationMode()``
Telepath methods have been marked as deprecated and will be removed after
2024-05-05.
(`3610 <https://github.com/vertexproject/synapse/pull/3610>`_)
- The Cortex configuration options ``cron:enable`` and ``trigger:enable`` have
been marked as deprecated and will be removed in ``v3.0.0``. These
configuration options no longer control cron or trigger behavior.
(`3610 <https://github.com/vertexproject/synapse/pull/3610>`_)
- The Storm Package ``synapse_minversion`` key has been deprecated and will
be removed in ``v3.0.0``. Package authors should use the ``synapse_version``
key to specify a version range for Synapse they support. An example is
the string ``>=2.165.0,<3.0.0``.
(`3593 <https://github.com/vertexproject/synapse/pull/3593>`_)

=====================

Features and Enhancements
-------------------------
- Update the Beholder messages ``view:merge:init``, ``view:merge:prog``, and
``view:merge:fini`` to add ``merge`` and ``vote`` information.
(`3580 <https://github.com/vertexproject/synapse/pull/3580>`_)
- When optimizing Storm lift operations, skip lifts that would be fully
filtered out.
(`3582 <https://github.com/vertexproject/synapse/pull/3582>`_)
- Add ``tmpdir`` information to the ``getSystemInfo()`` APIs. This is the
directory that the service would use for creating any temporary files.
(`3583 <https://github.com/vertexproject/synapse/pull/3583>`_)
- Update the ``synapse.tools.modrole`` tool to add a ``--del`` option to
delete a role.
(`3586 <https://github.com/vertexproject/synapse/pull/3586>`_)
- Add the ``reporter`` ``ou:org`` to ``ou:campaign`` nodes generated with
``gen.ou.campaign``
(`3594 <https://github.com/vertexproject/synapse/pull/3594>`_)
- The ``synapse.lib.certdir.CertDir`` class has been updated to use the
``cryptography`` APIs instead of the ``PyOpenSSL`` APIs where possible.
The ``CertDir`` APIs no longer return ``PyOpenSSL`` objects, and now
return ``cryptography`` related objects.
(`3568 <https://github.com/vertexproject/synapse/pull/3568>`_)
- Update the ``cryptography`` and ``PyOpenSSL`` libraries to require their
latest versions.
(`3568 <https://github.com/vertexproject/synapse/pull/3568>`_)

Bugfixes
--------
- Model interfaces now populate properties for the sub-interfaces.
(`3582 <https://github.com/vertexproject/synapse/pull/3582>`_)
- Use ``tostr`` on property and form names when computing lifts and pivots
to avoid a Python ``AttributeError`` exception. Invalid types will now
raise a ``StormRuntimeException``.
(`3584 <https://github.com/vertexproject/synapse/pull/3584>`_)

Deprecations
------------
- The tool ``synapse.tools.cellauth`` has been marked as deprecated and will
be removed in ``v3.0.0``.
(`3587 <https://github.com/vertexproject/synapse/pull/3587>`_)
- The tool ``synapse.tools.cmdr`` has been marked as deprecated and will
be removed in ``v3.0.0``.
(`3589 <https://github.com/vertexproject/synapse/pull/3589>`_)

=====================

Features and Enhancements
-------------------------
- Add Storm API methods to ``$lib.axon`` which share the functionality of
``$lib.bytes`` APIs. These include ``$lib.axon.has``, ``$lib.axon.hashset``,
``$lib.axon.put``, ``$lib.axon.size``, and ``$lib.axon.upload``.
(`3570 <https://github.com/vertexproject/synapse/pull/3570>`_)
(`3576 <https://github.com/vertexproject/synapse/pull/3576>`_)
- Add support for user provided certificates for doing mTLS in Storm HTTP
requests.
(`3566 <https://github.com/vertexproject/synapse/pull/3566>`_)
- Enable constructing a guid in Storm from a single value with
``$lib.guid(valu=$item)``.
(`3575 <https://github.com/vertexproject/synapse/pull/3575>`_)

=====================

Model Changes
-------------
- Updates to the ``inet``, ``infotech``, ``ou``, ``proj``, and ``risk`` models.
(`3549 <https://github.com/vertexproject/synapse/pull/3549>`_)
(`3551 <https://github.com/vertexproject/synapse/pull/3551>`_)
(`3564 <https://github.com/vertexproject/synapse/pull/3564>`_)
- See :ref:`userguide_model_v2_162_0` for more detailed model changes.

Features and Enhancements
-------------------------
- Add Storm API methods for inspecting and manipulating dictionary objects
in Storm. These are ``$lib.dict.has()``, ``$lib.dict.keys()``,
``$lib.dict.pop()``, ``$lib.dict.update()``, and ``$lib.dict.values()``.
(`3548 <https://github.com/vertexproject/synapse/pull/3548>`_)
- Add a ``json()`` method to the ``str`` type in Storm to deserialize a string
as JSON data.
(`3555 <https://github.com/vertexproject/synapse/pull/3555>`_)
- Add an ``_ahainfo`` attribute to the ``Telepath.Proxy``, containing AHA
service name information if that is provided to the Dmon.
(`3552 <https://github.com/vertexproject/synapse/pull/3552>`_)
- Add permissions checks to ``$lib.bytes`` APIs using ``axon.has`` for APIs
that check for information about the Axon or metrics; and ``axon.upload``
for APIs which put bytes in the Axon. These are checked with
``default=True`` for backward compatibility.
(`3563 <https://github.com/vertexproject/synapse/pull/3563>`_)
- The rstorm ``storm-svc`` and ``storm-pkg`` directives now wait for any
``onload`` handlers to complete.
(`3567 <https://github.com/vertexproject/synapse/pull/3567>`_)
- Update the Synapse Python package trove classifiers to list the platforms
we support using Synapse with.
(`3557 <https://github.com/vertexproject/synapse/pull/3557>`_)

Bugfixes
--------
- Fix a bug in the ``Cell.updateHttpSessInfo()`` API when the Cell does not
have the session in memory.
(`3556 <https://github.com/vertexproject/synapse/pull/3556>`_)
- Fix a bug where a user was allowed to vote for their own View merge request.
(`3565 <https://github.com/vertexproject/synapse/pull/3565>`_)
- Include Storm variables from the current and parent scopes when resolving
STIX properties and relationships.
(`3571 <https://github.com/vertexproject/synapse/pull/3571>`_)

Improved Documentation
----------------------
- Update the Storm automation documentation. Added additional information
about permissions used to manage automations. Added examples for
``edge:add`` and ``edge:del`` triggers. Added examples for managing Macro
permissions.
(`3547 <https://github.com/vertexproject/synapse/pull/3547>`_)
- Update the Storm filtering and lifting documentation to add information
about using interfaces and wildcard values with those operations.
(`3560 <https://github.com/vertexproject/synapse/pull/3560>`_)
- Update the Synapse introduction to note that Synapse is not intended to
replace big-data or data-lake solutions.
(`3553 <https://github.com/vertexproject/synapse/pull/3553>`_)

Deprecations
------------
- The Storm function ``$lib.dict()`` has been deprecated, in favor of using
the ``({"key": "value"})`` style syntax for directly declaring a dictionary
in Storm.
(`3548 <https://github.com/vertexproject/synapse/pull/3548>`_)
- Writeback layer mirrors and upstream layer mirrors have been marked as
deprecated configuration options.
(`3562 <https://github.com/vertexproject/synapse/pull/3562>`_)

=====================

Features and Enhancements
-------------------------
- Add a Storm command ``gen.it.av.scan.result`` to help generate
``it:av:scan:result`` nodes.
(`3516 <https://github.com/vertexproject/synapse/pull/3516>`_)
- Add item specific error message when users do not have sufficient permissions
on an object which is using easyperms.
(`3532 <https://github.com/vertexproject/synapse/pull/3532>`_)
- Ensure that Nexus events which are written to the log are always applied and
cannot be cancelled while the Nexus handler is running.
(`3518 <https://github.com/vertexproject/synapse/pull/3518>`_)
- Add ``getMergeRequest()`` and ``getMergeRequestSummary()`` Storm APIs to the
``View`` object, in order to get information about View merges via Storm.
(`3541 <https://github.com/vertexproject/synapse/pull/3541>`_)
- Add AHA information to the output of the ``Cell.getCellInfo()`` API. This
includes the service name, leader, and network.
(`3519 <https://github.com/vertexproject/synapse/pull/3519>`_)
- Logs related to AHA service registration and setting services as offline are
now logged at the ``INFO`` level.
(`3534 <https://github.com/vertexproject/synapse/pull/3534>`_)
- When creating Cron jobs and Triggers, record their creation time.
(`3521 <https://github.com/vertexproject/synapse/pull/3521>`_)
(`3538 <https://github.com/vertexproject/synapse/pull/3538>`_)
- Add a ``Cell.updateHttpSessInfo()`` API to set multiple keys at once on a
HTTP session.
(`3544 <https://github.com/vertexproject/synapse/pull/3544>`_)
- Update the allowed versions of the ``cbor2`` and `` pycryptodome``
libraries.
(`3540 <https://github.com/vertexproject/synapse/pull/3540>`_)

Bugfixes
--------
- The Storm API for creating websockets, ``$lib.inet.http.connect()``, did not
properly handle the ``ssl_verify`` argument, causing SSL verification of
Websocket requests to default to being disabled. This argument is now
handled correctly, with SSL verification being enabled by default.
(`3527 <https://github.com/vertexproject/synapse/pull/3527>`_)
- Fix a bug in embedded Storm queries where they failed to grab their variables
properly.
(`3531 <https://github.com/vertexproject/synapse/pull/3531>`_)
- Fix a bad variable reference in the Storm ``graph`` implementation.
(`3531 <https://github.com/vertexproject/synapse/pull/3531>`_)
- Fix a bug where modifying nodes in a Storm Dmon did not properly update the
in-flight node.
(`3520 <https://github.com/vertexproject/synapse/pull/3520>`_)

Improved Documentation
----------------------
- Update the Cortex admin guide with additional information about removing
extended forms and properties.
(`3510 <https://github.com/vertexproject/synapse/pull/3510>`_)
- Update the Data Model documentation to include additional information about
extended forms and properties.
(`3523 <https://github.com/vertexproject/synapse/pull/3523>`_)
- Update the Data Model documentation to include information about property
interfaces.
(`3523 <https://github.com/vertexproject/synapse/pull/3523>`_)

=====================

Automatic Migrations
--------------------
- Update ``inet:ipv6`` nodes to set their ``:type`` and ``:scope`` properties.
(`3498 <https://github.com/vertexproject/synapse/pull/3498>`_)
- Update existing layer push and layer pull configurations to set the default
chunk size and queue size values on them.
(`3480 <https://github.com/vertexproject/synapse/pull/3480>`_)
- See :ref:`datamigration` for more information about automatic migrations.

Model Changes
-------------
- Updates to the ``infotech``, ``ou``, and ``risk`` models.
(`3501 <https://github.com/vertexproject/synapse/pull/3501>`_)
(`3504 <https://github.com/vertexproject/synapse/pull/3504>`_)
(`3498 <https://github.com/vertexproject/synapse/pull/3498>`_)
- See :ref:`userguide_model_v2_160_0` for more detailed model changes.

Features and Enhancements
-------------------------
- Add tab completion of commands, forms, properties, tags, and ``$lib.``
functions the Storm CLI tool.
(`3493 <https://github.com/vertexproject/synapse/pull/3493>`_)
(`3507 <https://github.com/vertexproject/synapse/pull/3507>`_)
- Add ``node.set.<form>.<prop>`` and ``node.del.<form>.<prop>`` permissions
conventions to the Cortex for property sets and deletes.
(`3505 <https://github.com/vertexproject/synapse/pull/3505>`_)
- Add experimental support for Storm query offloading to the Cortex. This can
be used to offload Storm queries to an AHA service pool. This can be
configured with the ``storm:pool`` option on the Cortex.
(`3452 <https://github.com/vertexproject/synapse/pull/3452>`_)
(`3513 <https://github.com/vertexproject/synapse/pull/3513>`_)
- Add a ``--deledges`` option to the ``delnode`` command. This deletes the N2
edges for a node before deleting the node.
(`3503 <https://github.com/vertexproject/synapse/pull/3503>`_)
- When creating layer push or pull configurations, the chunk size and queue
size can now be set.
(`3480 <https://github.com/vertexproject/synapse/pull/3480>`_)
- Add a ``cell.hasHttpSess()`` API to check if a given Cell has a known HTTP
session.
(`3485 <https://github.com/vertexproject/synapse/pull/3485>`_)
- Fire a ``core:pkg:onload:complete`` event when a Storm package ``onload``
handler is completed. This can be used when writing unit tests for Rapid
Power-ups.
(`3497 <https://github.com/vertexproject/synapse/pull/3497>`_)

Bugfixes
--------
- Remove dataname index entries when removing all nodedata from a node.
(`3499 <https://github.com/vertexproject/synapse/pull/3499>`_)
- Fix an issue with ``tagprops`` not being correctly returned in
``$node.getByLayer()``.
(`3500 <https://github.com/vertexproject/synapse/pull/3500>`_)
- Fix an issue with the ``edges.del`` command when using the ``--n2`` option.
This now behaves correctly when the N1 node does not exist.
(`3506 <https://github.com/vertexproject/synapse/pull/3506>`_)
- Fix an issue with duplicate properties being tracked in the property type
map of the data model. This could have resulted in multiple nodes being
lifted with interface properties.
(`3512 <https://github.com/vertexproject/synapse/pull/3512>`_)

Improved Documentation
----------------------
- Update Storm filter documentation. Additional information about tag globbing
and interval filtering has been included.
(`3489 <https://github.com/vertexproject/synapse/pull/3489>`_)