Synapse

=====================

Automatic Migrations
--------------------
- Update the ``inet:ipv4:type`` value for RFC6598 addresses to ``shared``.
(`3410 <https://github.com/vertexproject/synapse/pull/3410>`_)
- See :ref:`datamigration` for more information about automatic migrations.

Model Changes
-------------
- Update to the ``inet`` and ``ou`` models.

(`3406 <https://github.com/vertexproject/synapse/pull/3406>`_)
(`3407 <https://github.com/vertexproject/synapse/pull/3407>`_)
(`3410 <https://github.com/vertexproject/synapse/pull/3410>`_)
(`3416 <https://github.com/vertexproject/synapse/pull/3416>`_)

**Updated Types**

``inet:ipv4``
RFC6598 addresses now have a ``:type`` property value of ``shared``.

``inet:url``
Accept Microsoft URLPrefix strings with a strong wildcard host value.

Add a check to prevent creating ``inet:url`` nodes with an empty host
and path part, such as ``inet:url=file://''``.

**New Properties**

``ou:org``
The form had the following property added to it:

``tag``
A base tag used to encode assessments made by the organization.

``risk:compromise``
The form had the following properties added to it:

``ext:id``
An external unique ID for the compromise.

``url``
A URL which documents the compromise.

``risk:alert``
The form had the following property added to it:

``host``
The host which generated the alert.

**New Forms**

``ou:requirement``
A specific requirement.

``risk:leak``
An event where information was disclosed without permission.

``risk:leak:type:taxonomy``
A taxonomy of leak event types

``risk:extortion``
An event where an attacker attempted to extort a victim.

``risk:extortion:type:taxonomy``
A taxonomy of extortion event types.

**Light Edges**

``leaked``
When used with a ``risk:leak`` node, the edge indicates the leak included
the disclosure of the target node.

``leveraged``
When used with a ``risk:extortion`` node, the edge indicates the extortion
event was based on attacker access to the target node.

``meets``
When used with a ``ou:requirement`` node, the edge indicates the
requirement was met by the source node.

Features and Enhancements
-------------------------
- Add ``edge:add`` and ``edge:del`` as trigger conditions. These trigger when
light edges are added or removed from a node.
(`3389 <https://github.com/vertexproject/synapse/pull/3389>`_)
- Storm lift and filter operations using regular expressions (``~=``) are now
case insensitive by default.
(`3403 <https://github.com/vertexproject/synapse/pull/3403>`_)
- Add a ``unique()`` method to the Storm ``list`` object. This returns a new
list with only unique elements in it.
(`3415 <https://github.com/vertexproject/synapse/pull/3415>`_)
- Add support for ``synapse.tools.autodoc`` to generate documentation for
API definitions declared in Storm packages.
(`3382 <https://github.com/vertexproject/synapse/pull/3382>`_)
- A review of Storm library functions was performed and all ``readonly`` safe
functions have been marked for execution in a ``readonly`` Storm runtime.
(`3402 <https://github.com/vertexproject/synapse/pull/3402>`_)
- Allow setting the layers on a root View with forks.
(`3413 <https://github.com/vertexproject/synapse/pull/3413>`_)

Bugfixes
--------
- Per-node Storm variables are now passed into subquery assignment
expressions.
(`3405 <https://github.com/vertexproject/synapse/pull/3405>`_)
- Fix an issue with Storm Dmon hive storage being opened too late in the
Cortex startup sequence.
(`3411 <https://github.com/vertexproject/synapse/pull/3411>`_)
- Remove a check when deleting tags from a node which prevented tag deletion
from a node when the root tag was deleted in a parent view.
(`3408 <https://github.com/vertexproject/synapse/pull/3408>`_)

=====================

Model Changes
-------------
- Update to the ``inet`` and ``ou`` models.

(`3393 <https://github.com/vertexproject/synapse/pull/3393>`_)
(`3396 <https://github.com/vertexproject/synapse/pull/3396>`_)

**Deprecated Properties**

``inet:web:acct``
The ``inet:web:acct`` form had the following properties marked as deprecated:

* ``name:en``
* ``realname:en``

``inet:web:group``
The ``inet:web:group`` form had the following property marked as deprecated:

* ``name:en``

``ou:industry``
The ``ou:industry`` form had the following property marked as deprecated:

* ``subs``

Features and Enhancements
-------------------------
- Add a new Storm API, ``$lib.cortex.httpapi``, for creating and managing
Extended HTTP API endpoints. These Cortex HTTP API endpoints allow a user to
create custom responses via Storm. Documentation for this feature can be
found at :ref:`devops-svc-cortex-ext-http`.
(`3366 <https://github.com/vertexproject/synapse/pull/3366>`_)
- Add a new Storm API, ``$lib.iters.zip()``, to iterate over sequences of
items together.
(`3392 <https://github.com/vertexproject/synapse/pull/3392>`_)
(`3398 <https://github.com/vertexproject/synapse/pull/3398>`_)
- Add a Storm command ``stats.countby`` to tally occurrences of values and
display a barchart representing the values.
(`3385 <https://github.com/vertexproject/synapse/pull/3385>`_)
- Update the Storm command ``auth.user.mod`` to allow setting a user as admin
on a specific auth gate.
(`3391 <https://github.com/vertexproject/synapse/pull/3391>`_)
- The ``proxy`` argument to ``$lib.inet.http.*``, ``$lib.axon.wget()``,
``$lib.axon.urlfile()``, and ``$lib.axon.wput()`` APIs is now gated behind
the permission ``storm.lib.inet.http.proxy``. Previously this required
admin permission to utilize.
(`3397 <https://github.com/vertexproject/synapse/pull/3397>`_)
- Add an ``errors`` parameter to ``$lib.axon.readlines()``,
``$lib.axon.csvrows()``, and ``$lib.axon.jsonlines()``. This parameter
defaults to ``ignore`` to ignore any decoding errors that are encountered
when decoding text.
(`3395 <https://github.com/vertexproject/synapse/pull/3395>`_)
- Lower the maximum allowed version of the ``pyopenssl`` library.
(`3399 <https://github.com/vertexproject/synapse/pull/3399>`_)

Bugfixes
--------
- Fix a bug in the ``Cortex.syncLayersEvents()`` and
``Cortex.syncIndexEvents()`` APIs which caused layers to stop sending their
node edits under certain conditions.
(`3394 <https://github.com/vertexproject/synapse/pull/3394>`_)
- Storm now raises a ``BadSyntaxError`` when attempting to filter by wildcard
tags or tagprops when a value is specified for the filter.
(`3373 <https://github.com/vertexproject/synapse/pull/3373>`_)

=====================

Model Changes
-------------
- Update to the ``biz``, ``crypto``, ``geo``, ``it``, ``mat``, ``media``,
and ``risk`` models.

(`3341 <https://github.com/vertexproject/synapse/pull/3341>`_)
(`3377 <https://github.com/vertexproject/synapse/pull/3377>`_)
(`3376 <https://github.com/vertexproject/synapse/pull/3376>`_)
(`3381 <https://github.com/vertexproject/synapse/pull/3381>`_)

**Updated Interfaces**

``crypto:smart:effect``
Add a ``doc`` value to the interface.

``it:host:activity``
Add a ``doc`` value to the interface.

``taxonomy``
Add a ``doc`` value to the interface.

**Updated Types**

``time``
The ``time`` type now recognizes RFC822 formatted time strings.

``biz:service:type:taxonomy``
The ``taxonomy`` interface has been added to the type.

``geo:place:taxonomy``
The ``taxonomy`` interface has been added to the type.

``it:log:event:type:taxonomy``
The ``taxonomy`` interface has been added to the type.

``it:prod:soft:taxonomy``
The ``taxonomy`` interface has been added to the type.

``mat:type``
The ``taxonomy`` interface has been added to the type.

``media:news:taxonomy``
The ``taxonomy`` interface has been added to the type.

``risk:alert:taxonomy``
The ``taxonomy`` interface has been added to the type.

``risk:alert:verdict:taxonomy``
The ``taxonomy`` interface has been added to the type.

``risk:threat:type:taxonomy``
The ``taxonomy`` interface has been added to the type.

**New Forms**

``it:dev:repo:label``
A developer selected label.

``it:dev:repo:issue:label``
A label applied to a repository issue.

Features and Enhancements
-------------------------
- Update the Storm string repr for ``$lib.null`` and ``$lib.undef`` values to
``$lib.null`` and ``$lib.undef``. Previously these printed ``None`` and an
opaque Python object repr.
(`3361 <https://github.com/vertexproject/synapse/pull/3361>`_)
- The ``synapse.tools.aha.list`` CLI tool now checks if it is connected to an
Aha server prior to enumerating Aha services.
(`3371 <https://github.com/vertexproject/synapse/pull/3371>`_)

Bugfixes
--------
- Update the ``file:path`` support for scrape related APIs to address an
issue when matching against Linux style paths.
(`3378 <https://github.com/vertexproject/synapse/pull/3378>`_)
- Update the ``hex`` type to ``zeropad`` strings prior to checking their
validity.
(`3387 <https://github.com/vertexproject/synapse/pull/3387>`_)
- Update the ``yaml.CSafeLoader`` check to not require the class to be
available.
(`3386 <https://github.com/vertexproject/synapse/pull/3386>`_)

Improved Documentation
----------------------
- Update the documentation for the Storm ``view.exec`` command to explain the
separation of events and nodes between the parent and sub-runtimes.
(`3379 <https://github.com/vertexproject/synapse/pull/3379>`_)

=====================

Model Changes
-------------
- Update to the ``it`` model.
(`3361 <https://github.com/vertexproject/synapse/pull/3361>`_)

**New Forms**

``it:mitre:attack:flow``
A MITRE ATT&CK Flow diagram.

Features and Enhancements
-------------------------
- Add a new Storm library ``$lib.infosec.mitre.attack.flow``. This can be used
to normalize and create ``it:mitre:attack:flow`` nodes from MITRE ATT&CK
Flow Diagrams.
(`3361 <https://github.com/vertexproject/synapse/pull/3361>`_)
(`3372 <https://github.com/vertexproject/synapse/pull/3372>`_)
- Update the Storm ``note.add`` command to set the ``meta:note:created``
property on the note.
(`3569 <https://github.com/vertexproject/synapse/pull/3569>`_)
- Add the Axon HTTP APIs to the Cortex. These API endpoints use the Axon that
the Cortex is configured to use.
(`3550 <https://github.com/vertexproject/synapse/pull/3550>`_)
- Allow user defined functions in Storm to execute in a ``readonly`` Storm
runtime.
(`3552 <https://github.com/vertexproject/synapse/pull/3552>`_)
- Clarify the Nexus ``IsReadOnly`` exception to include the common cause for
the error, which is normally insufficent space on disk.
(`3359 <https://github.com/vertexproject/synapse/pull/3359>`_)
- Add a ``SYN_LOG_DATEFORMAT`` environment variable to allow specifying custom
timestamp formats for Synapse services.
(`3362 <https://github.com/vertexproject/synapse/pull/3362>`_)
- Add a ``status`` attribute to structured log events for user and role
related log events. This attribute indicates if the event was a ``CREATE``,
``DELETE``, or ``MODIFY`` operation.
(`3363 <https://github.com/vertexproject/synapse/pull/3363>`_)
- Update ``Cell.getLogExtra()`` to prefer using the ``user`` key from the task
scope before using the ``sess`` key from the task scope. Cortex APIs which
execute Storm queries now set the ``user`` scope to the user the query is
running as. This increases the accuracy of log events caused by Storm
queries when the ``user`` is specified in the ``opts``.
(`3356 <https://github.com/vertexproject/synapse/pull/3356>`_)
- Update Storm setitem AST operator to check the readonly flag on functions
when operating in a ``readonly`` Storm runtime.
(`3364 <https://github.com/vertexproject/synapse/pull/3364>`_)
- Update the minimum required version of the ``fastjsonschema`` library.
(`3358 <https://github.com/vertexproject/synapse/pull/3358>`_)
- Update tests and remove the use of deprecated functions for improved
Python 3.12 compatibility.
(`3355 <https://github.com/vertexproject/synapse/pull/3355>`_)
(`3567 <https://github.com/vertexproject/synapse/pull/3567>`_)

Bugfixes
--------
- Fixed a bug when parenting a View to another View where the bottom view has
more than one layer in it omitted non-write layers. The set of layers is now
properly computed.
(`3354 <https://github.com/vertexproject/synapse/pull/3354>`_)

Improved Documentation
----------------------
- Update the list of Cortex permissions in the Admin Guide.
(`3331 <https://github.com/vertexproject/synapse/pull/3331>`_)
- The Form documentation has been updated to project the secondary properties
and associated light edges as tables.
(`3348 <https://github.com/vertexproject/synapse/pull/3348>`_)

=====================

Model Changes
-------------
- Updates to the ``inet`` model.
(`3347 <https://github.com/vertexproject/synapse/pull/3347>`_)

**Updated Types**

``inet:url``
The ``inet:url`` type now recognizes UNC network paths and converts
them into ``smb://`` URLs.

Features and Enhancements
-------------------------
- Allow Storm trigger APIs to reference triggers from other views.
(`3342 <https://github.com/vertexproject/synapse/pull/3342>`_)
- Update the ``synapse.lib.scrape`` and associated APIs to capture
additional data:
(`3223 <https://github.com/vertexproject/synapse/pull/3223>`_)
(`3347 <https://github.com/vertexproject/synapse/pull/3347>`_)

``it:sec:cpe``
CPE 2.3 strings are now identified.

``inet:url``
UNC based paths are now identified.

- Update the ``synapse.lib.scrape`` and associated APIs to use subprocesses
when scraping large volumes of text.
(`3344 <https://github.com/vertexproject/synapse/pull/3344>`_)
- Add additional logging for HTTP API endpoints when a request has invalid
login information.
(`3345 <https://github.com/vertexproject/synapse/pull/3345>`_)
- The CryoTank service has had permissions added to it.
(`3328 <https://github.com/vertexproject/synapse/pull/3328>`_)

Bugfixes
--------
- Stormtypes ``stor`` functions were not previously checked during
``readonly`` runtime execution. These are now validated and ``stor``
functions which would result in changing data in the Cortex will now
raise an exception when used with a ``readonly`` Storm runtime.
(`3349 <https://github.com/vertexproject/synapse/pull/3349>`_)

Improved Documentation
----------------------
- Update the list of Cortex permissions in the Admin Guide.
(`3331 <https://github.com/vertexproject/synapse/pull/3331>`_)
- The Form documentation has been updated to project the secondary properties
and associated light edges as tables.
(`3348 <https://github.com/vertexproject/synapse/pull/3348>`_)

=====================

Model Changes
-------------
- Updates to the ``it``, ``meta``, and ``ou`` models.
(`3338 <https://github.com/vertexproject/synapse/pull/3338>`_)

**New Properties**

``taxonomy``
The interface had the following property added to it:

``description``
A definition of the taxonomy entry.

``inet:email:message``
The form had the following property added to it:

``cc``
Email addresses parsed from the "cc" header.

``meta:source``
The form had the following property added to it:

``url``
A URL which documents the meta source.

``ou:campaign``
The form had the following property added to it:

``timeline``
A timeline of significant events related to the campaign.

**Deprecated Properties**

``taxonomy``
The ``taxonomy`` interface had the following property marked as deprecated:

* ``summary``

Features and Enhancements
-------------------------
- Add best-effort support to scrape APIs to identify Windows and Linux file
paths.
(`3343 <https://github.com/vertexproject/synapse/pull/3343>`_)
- Update the Storm ``view.add`` command to add a ``--worldreadable`` flag to
create a view which is readable by the ``all`` role. The ``$lib.view.add()``
Storm API now also accepts an optional ``worldreadable`` argument as well.
(`3333 <https://github.com/vertexproject/synapse/pull/3333>`_)
- Update the Storm ``note.add`` command to add a ``--yield`` flag which yields
the newly created note.
(`3337 <https://github.com/vertexproject/synapse/pull/3337>`_)
- Add Storm commands ``gen.ou.id.number`` and ``gen.ou.id.type`` to help
generate ``ou:id:number`` and ``ou:id:type`` nodes.
(`3339 <https://github.com/vertexproject/synapse/pull/3339>`_)
- Support dynamically setting a Layer to ``readonly`` using the Storm
``$layer.set()`` API.
(`3332 <https://github.com/vertexproject/synapse/pull/3332>`_)
- Update the Storm command ``help`` to display information about Storm types,
Storm Libraries and functions.
(`3335 <https://github.com/vertexproject/synapse/pull/3335>`_)

Bugfixes
--------
- Ensure that the Cell ``tmp`` directory is on the same volume as the Cell
storage directory prior to attempting to run the onboot optimization
process. If the volumes are different this now issues a warning message and
skips the optimization process.
(`3336 <https://github.com/vertexproject/synapse/pull/3336>`_)
- Protect the Cortex Cron scheduling loop from errors that could happen when
starting an agenda item.
(`3340 <https://github.com/vertexproject/synapse/pull/3340>`_)