Synapse

=====================

Automatic Migrations
--------------------
- Update indexing for light edges to index the N1 and N2 node identifiers
together.
(`3302 <https://github.com/vertexproject/synapse/pull/3302>`_)
- See :ref:`datamigration` for more information about automatic migrations.

Model Changes
-------------
- Update to the ``inet``, ``it``, and ``meta`` models.
(`3285 <https://github.com/vertexproject/synapse/pull/3285>`_)
(`3298 <https://github.com/vertexproject/synapse/pull/3298>`_)
(`3301 <https://github.com/vertexproject/synapse/pull/3301>`_)
(`3310 <https://github.com/vertexproject/synapse/pull/3310>`_)
- See :ref:`userguide_model_v2_145_0` for more detailed model changes.

Features and Enhancements
-------------------------
- Add a new Storm keyword, ``reverse( ... )``, which can be used to run a lift
operation in reverse order.
(`3266 <https://github.com/vertexproject/synapse/pull/3266>`_)
- Update indexing for light edges to index the N1 and N2 node identifiers
together.
(`3302 <https://github.com/vertexproject/synapse/pull/3302>`_)
- Update the Storm ``once`` command behavior and documentation to be more
intuitive when setting its timestamp and allowing nodes through it.
(`3282 <https://github.com/vertexproject/synapse/pull/3282>`_)
- Add a ``synapse_version`` key to the Storm Package schema. This can be used
to provide a string version indentifier with a minimum and maximum version,
such as ``>=2.145.0,<3.0.0``.
(`3304 <https://github.com/vertexproject/synapse/pull/3304>`_)
- Update the Storm runtime to respect permissions declared with a ``default``
value of ``true``. This allows Storm packages to define permissions which
are defaulted to ``true``.
(`3287 <https://github.com/vertexproject/synapse/pull/3287>`_)
- Add a ``SIGHUP`` handler to the base Cell which can be used to reload HTTPS
certificate files from disk. The ``synapse.tools.reload`` tool can also be
used to trigger this behavior.
(`3293 <https://github.com/vertexproject/synapse/pull/3293>`_)
- The optional ``max:users`` feature no longer counts ``locked`` or
``archived`` users when adding users.
(`3295 <https://github.com/vertexproject/synapse/pull/3295>`_)
- Update the YAML functions to use the ``yaml.CSafeLoader`` and
``yaml.CSafeDumper``.
(`3289 <https://github.com/vertexproject/synapse/pull/3289>`_)

Bugfixes
--------
- Replace ``asyncio.wait_for()`` use with a copy of the Python 3.12
implementation to avoid a race condition when cancelling tasks.
(`3299 <https://github.com/vertexproject/synapse/pull/3299>`_)
(`3307 <https://github.com/vertexproject/synapse/pull/3307>`_)
- Fix an issue with the Storm trigger ``set()`` method not properly checking
the values that it allows to be set.
(`3290 <https://github.com/vertexproject/synapse/pull/3290>`_)
- Fix an off-by-one bug in the ``SlabSeqn.aiter()`` method.
(`3300 <https://github.com/vertexproject/synapse/pull/3300>`_)
- Fix a performance issue with the IPv6 regular expression used in the scrape
APIs.
(`3311 <https://github.com/vertexproject/synapse/pull/3311>`_)

Improved Documentation
----------------------
- Revise the Storm User Guide to consolidate the background information
and data modeling sections. Add a user focused section on Views and Layers.
(`3303 <https://github.com/vertexproject/synapse/pull/3303>`_)
- Add ``int`` type specific information to the Storm documentation.
(`3288 <https://github.com/vertexproject/synapse/pull/3288>`_)
- The Storm ``movetag`` command now moves the ``doc:url`` property from the
old ``syn:tag`` node to the new ``syn:tag`` node.
(`3294 <https://github.com/vertexproject/synapse/pull/3294>`_)
- Storm Library and Type documentation no longer renders function signatures
with Python style defaults.
(`3296 <https://github.com/vertexproject/synapse/pull/3296>`_)

Deprecations
------------
- Many deprecated Cortex and splice related APIs have been marked for removal
after 2023-10-01. The full list of APIs which will be removed can be found
at :ref:`changelog-depr-20231001`.
(`3292 <https://github.com/vertexproject/synapse/pull/3292>`_)
- The use of ``synapse.common.aclosing()`` has been replaced with
``contextlib.aclosing()``. The vendored ``aclosing()`` implementation will
be removed in ``v2.250.0``.
(`3206 <https://github.com/vertexproject/synapse/pull/3206>`_)

=====================

Model Changes
-------------
- Updates to the ``inet:dns`` and ``it`` model.
(`3257 <https://github.com/vertexproject/synapse/pull/3257>`_)
(`3276 <https://github.com/vertexproject/synapse/pull/3276>`_)
- See :ref:`userguide_model_v2_144_0` for more detailed model changes.

Features and Enhancements
-------------------------
- The iden of the Cron job is now added to the Storm query log made with
the ``synapse.storm`` logger when using structured logging.
(`3235 <https://github.com/vertexproject/synapse/pull/3235>`_)
- Add a ``keepalive`` option to the Storm query ``opts``. This may be used
with long-running Storm queries when behind a network proxy or load balancer
which may terminate idle connections.
(`3272 <https://github.com/vertexproject/synapse/pull/3272>`_)
- Update the allowed versions of the ``cryptography`` library.
(`3281 <https://github.com/vertexproject/synapse/pull/3281>`_)

Bugfixes
--------
- Fix an issue where Storm Dmons could start prior to data model migrations.
(`3279 <https://github.com/vertexproject/synapse/pull/3279>`_)
- Adjust the storage convention for ``once`` state data to fix an edge case
and clarify documentation.
(`3282 <https://github.com/vertexproject/synapse/pull/3282>`_)
- Fix an issue with missing keys in storage nodes during migrations.
(`3284 <https://github.com/vertexproject/synapse/pull/3284>`_)

=====================

Model Changes
-------------
- Update to the ``crypto`` model.
(`3256 <https://github.com/vertexproject/synapse/pull/3256>`_)
- See :ref:`userguide_model_v2_143_0` for more detailed model changes.

Features and Enhancements
-------------------------
- Add ``$lib.model.ext.getExtModel()`` and
``$lib.model.ext.addExtModel()`` Storm APIs to get all the extended model
definitions in a Cortex and to add extended model definitions to
a Cortex in bulk.
(`3252 <https://github.com/vertexproject/synapse/pull/3252>`_)
- Add ``inet:ipv6`` to the list of types identified with scrape APIs. The
``inet:server`` form identified by scrape APIs now also identifies IPv6
server addresses.
(`3259 <https://github.com/vertexproject/synapse/pull/3259>`_)
- Add a check to the Cortex startup to identify and log the presence of
deprecated model elements and direct users to check and lock them
at :ref:`storm-model-deprecated-check`.
(`3253 <https://github.com/vertexproject/synapse/pull/3253>`_)
(`3264 <https://github.com/vertexproject/synapse/pull/3264>`_)
- Add a new Storm function, ``$lib.vars.type()``, to get the type
value of an object.
(`3100 <https://github.com/vertexproject/synapse/pull/3100>`_)
- Add a Storm library, ``$lib.pack``, for packing and unpacking structured
byte values.
(`3261 <https://github.com/vertexproject/synapse/pull/3261>`_)
- The Storm ``$lib.gen()`` functions and associated commands now generate
stable guid values based on their inputs when making nodes.
(`3268 <https://github.com/vertexproject/synapse/pull/3268>`_)
- Add the ``.bazar`` TLD to the list of TLDs identified by the Synapse scrape
functionality.
(`3271 <https://github.com/vertexproject/synapse/pull/3271>`_)
- Add the View iden to the task identifier for running Storm tasks.
(`3247 <https://github.com/vertexproject/synapse/pull/3247>`_)
- Add performance related sysctl values to the output of the Storm
``Cell.getSystemInfo()`` and ``$lib.cell.getSystemInfo()`` APIs.
(`3236 <https://github.com/vertexproject/synapse/pull/3236>`_)
- Update the allowed versions of the ``vcrpy`` library. Thank you
``captainGeech42`` for the contribution.
(`3204 <https://github.com/vertexproject/synapse/pull/3204>`_)

Bugfixes
--------
- Ensure the input to the ``CoreAPI.storm()`` ( and related APIs ) is a
string.
(`3255 <https://github.com/vertexproject/synapse/pull/3255>`_)
(`3269 <https://github.com/vertexproject/synapse/pull/3269>`_)
- Fix a bug in ``synapse.tools.aha.enroll`` where a user with a
``telepath.yaml`` file containing an ``aha:servers`` key with a list of
lists failed to enroll a local user.
(`3260 <https://github.com/vertexproject/synapse/pull/3260>`_)
- Fix an issue where Storm functions using ``emit`` failed to cleanup their
sub-runtimes.
(`3250 <https://github.com/vertexproject/synapse/pull/3250>`_)
- Add verification that a Storm function call is being made on a callable
object and raise a ``StormRuntimeError`` if the object cannot be called.
Previously invalid calls could raise a ``TypeError``.
(`3243 <https://github.com/vertexproject/synapse/pull/3243>`_)
- Fix the order of the Beholder ``cron:stop`` message firing when a Cron job
is stopped.
(`3265 <https://github.com/vertexproject/synapse/pull/3265>`_)

Improved Documentation
----------------------
- Add a section to the Storm reference for user defined functions in Storm.
That can be found at :ref:`storm-adv-functions`.
(`3245 <https://github.com/vertexproject/synapse/pull/3245>`_)
- Update the devops documentation to add a note about the Telepath ``aha://``
protocol using a ``mirror=true`` parameter to connect to a service mirror
instead of a leader.
(`3267 <https://github.com/vertexproject/synapse/pull/3267>`_)
- Update the ``preboot.sh`` example script to account for Docker changes
introduced in ``v2.133.0``.

=====================

Bugfixes
--------
- Fix an issue which caused the Docker image tags for
``vertexproject/synapse-cryotank:v2.141.1``,
``vertexproject/synapse-jsonstor:v2.141.1``, and
``vertexproject/synapse-stemcell:v2.141.1``, to refer to same image.
(`3249 <https://github.com/vertexproject/synapse/pull/3249>`_)

=====================

Bugfixes
--------
- Fix an issue which prevented the publication of the Synapse containers with
``v2.x.x`` tags.
(`3248 <https://github.com/vertexproject/synapse/pull/3248>`_)

=====================

Automatic Migrations
--------------------
- Renormalize the ``risk:vuln:cvss:v2`` and ``risk:vuln:cvss:v3`` properties.
(`3224 <https://github.com/vertexproject/synapse/pull/3224>`_)
- Migrate the ``risk:vuln:name`` type from a ``str`` to a ``risk:vulnname``
form.
(`3227 <https://github.com/vertexproject/synapse/pull/3227>`_)
- See :ref:`datamigration` for more information about automatic migrations.

Model Changes
-------------
- Update to the ``it``, ``ou``, and ``risk`` models.
(`3224 <https://github.com/vertexproject/synapse/pull/3224>`_)
(`3227 <https://github.com/vertexproject/synapse/pull/3227>`_)
(`3237 <https://github.com/vertexproject/synapse/pull/3237>`_)
- See :ref:`userguide_model_v2_142_0` for more detailed model changes.

Features and Enhancements
-------------------------
- Always convert dictionary keys to their primitive values when working with
dictionary objects in Storm. Dictionary objects can no longer have keys
set which are mutable objects, such as Nodes.
(`3233 <https://github.com/vertexproject/synapse/pull/3233>`_)
- Add support for octal constants, such as ``0o755``, in Storm expressions.
(`3231 <https://github.com/vertexproject/synapse/pull/3231>`_)
- Add additional events to the Behold API message stream for the addition
and removal of extended model elements.
(`3228 <https://github.com/vertexproject/synapse/pull/3228>`_)
- Update the ``$lib.dmon.add()`` variable capture to record variables
from embedded query objects.
(`3230 <https://github.com/vertexproject/synapse/pull/3230>`_)
- Add a ``.title()`` method on Storm strings to get title case formatted
strings.
(`3242 <https://github.com/vertexproject/synapse/pull/3242>`_)
- Add a general purpose process pool using forked workers in order to speed
up certain processing operations. This includes the Storm operations for
JSONSchema parsing, HTML parsing, STIX validation, and XML parsing.
(`3033 <https://github.com/vertexproject/synapse/pull/3033>`_)
(`3229 <https://github.com/vertexproject/synapse/pull/3229>`_)
- Add a new Cell configuration option, ``max:users``. This can be set to limit
the maximum number of non-``root`` users on Cell.
(`3244 <https://github.com/vertexproject/synapse/pull/3244>`_)
- Add an ``/api/v1/aha/services`` HTTP API to the Aha service. This
can be used to get a list of the services registered with Aha.
(`3238 <https://github.com/vertexproject/synapse/pull/3238>`_)
- Add support for Cosign signatures of tagged Synapse containers. See
additional information at :ref:`dev_docker_verification`.
(`3196 <https://github.com/vertexproject/synapse/pull/3196>`_)
- Adjust internal names for Storm objects.
(`3229 <https://github.com/vertexproject/synapse/pull/3229>`_)

Bugfixes
--------
- Fix a bug in the scrape for ``inet:ipv4`` where IP addresses were found
when there was leading or trailing numbers around the IP addresses.
(`3234 <https://github.com/vertexproject/synapse/pull/3234>`_)
- Fix a bug where ``$lib.model.ext.delForm()`` did not check for extended
property definitions before deletion. Extended properties on a custom form
must be deleted prior to deleting the form.
(`3223 <https://github.com/vertexproject/synapse/pull/3223>`_)
- Always remove the ``mirror`` configuration option from ``cell.yaml`` file
when provisioning a service via Aha. The previous behavior prevented the
correct restoration of a service from a backup which was previously
provisioned as a mirror and is being restored as a leader.
(`3240 <https://github.com/vertexproject/synapse/pull/3240>`_)
- Add additional type checking when adding extended model forms and properties
to the Cortex. Previously invalid types could raise an ``AttributeError``.
(`3243 <https://github.com/vertexproject/synapse/pull/3243>`_)

Improved Documentation
----------------------
- Update the Storm lift reference to add an example of lifting nodes by the
universal ``.created`` property.
(`3245 <https://github.com/vertexproject/synapse/pull/3245>`_)