Synapse

====================

Features and Enhancements
-------------------------
- Add an ``/api/v1/aha/provision/service`` HTTP API to the Aha service. This
can be used to generate ``aha:provision`` URLs.
(`2707 <https://github.com/vertexproject/synapse/pull/2707>`_)
- Add ``proxy`` options to ``$lib.inet.http`` Storm APIs, to allow an admin
user to specify an alternative (or to disable) proxy setting.
(`2706 <https://github.com/vertexproject/synapse/pull/2706>`_)
- Add a ``--tag`` and ``--prop`` option to the Storm ``diff`` command. Update
the Storm ``merge`` command examples to show more real-world use cases.
(`2710 <https://github.com/vertexproject/synapse/pull/2710>`_)
- Add the ability to set the layers in a non-forked view with the
``$view.set(layers, $iden)`` API on the Storm view object.
(`2711 <https://github.com/vertexproject/synapse/pull/2711>`_)
- Improve Storm parser logic for handling list and expression syntax.
(`2698 <https://github.com/vertexproject/synapse/pull/2698>`_)
(`2708 <https://github.com/vertexproject/synapse/pull/2708>`_)

Bugfixes
--------
- Improve error handling of double quoted strings in Storm when null
characters are present in the raw query string. This situation now raises a
BadSyntax error instead of an opaque Python ValueError.
(`2709 <https://github.com/vertexproject/synapse/pull/2709>`_)
- Fix unquoted JSON keys which were incorrectly allowed in Storm JSON style
expression syntax.
(`2698 <https://github.com/vertexproject/synapse/pull/2698>`_)
- When merging layer data, add missing permission checks for light edge and
node data changes.
(`2671 <https://github.com/vertexproject/synapse/pull/2671>`_)

====================

Features and Enhancements
-------------------------
- Updates to the ``transport`` model.
(`2697 <https://github.com/vertexproject/synapse/pull/2697>`_)

``velocity``
Add a new base type to record velocities in millimeters/second.

``transport:direction``
Add a new type to indicate a direction of movement with respect to true
North.

``transport:air:telem``
Add ``:course`` and ``:heading`` properties to record the direction of travel.
Add ``:speed``, ``:airspeed`` and ``:verticalspeed`` properties to record
the speed of travel.

``transport:sea:telem``
Add ``:course`` and ``:heading`` properties to record the direction of travel.
Add a ``:speed`` property to record the speed of travel.
Add ``:destination``, ``:destination:name`` and ``:destination:eta`` to record
information about the destination.

- Restore the precedence of environment variables over ``cell.yaml`` options
during Cell startup. API driven overrides are now stored in the
``cell.mods.yaml`` file.
(`2699 <https://github.com/vertexproject/synapse/pull/2699>`_)
- Add ``--dmon-port`` and ``--https-port`` options to the
``synapse.tools.aha.provision.service`` tool in order to specify fixed
listening ports during provisioning.
(`2703 <https://github.com/vertexproject/synapse/pull/2703>`_)
- Add the ability of ``synapse.tools.moduser`` to set user passwords.
(`2695 <https://github.com/vertexproject/synapse/pull/2695>`_)
- Restore the call to the ``recover()`` method on the Nexus during Cell
startup.
(`2701 <https://github.com/vertexproject/synapse/pull/2701>`_)
- Add ``mesg`` arguments to ``NoSuchLayer`` exceptions.
(`2696 <https://github.com/vertexproject/synapse/pull/2696>`_)
- Make the LMDB slab startup more resilient to a corrupted ``cell.opts.yaml``
file.
(`2694 <https://github.com/vertexproject/synapse/pull/2694>`_)

Bugfixes
--------
- Fix missing variable checks in Storm.
(`2702 <https://github.com/vertexproject/synapse/pull/2702>`_)

Improved Documentation
----------------------
- Add a warning to the deployment guide about using Docker on Mac OS.
(`2700 <https://github.com/vertexproject/synapse/pull/2700>`_)

====================

Bugfixes
--------
- Fix a regression in the Telepath ``aha://`` update from ``v2.95.0``.
(`2693 <https://github.com/vertexproject/synapse/pull/2693>`_)

====================

Features and Enhancements
-------------------------
- Add a ``search`` mode to Storm. The ``search`` mode utilizes the Storm
search interface to lift nodes. The ``lookup`` mode no longer uses the
search interface.
(`2689 <https://github.com/vertexproject/synapse/pull/2689>`_)
- Add a ``?mirror=true`` flag to ``aha://`` Telepath URLs which will cause
the Aha service lookups to prefer using a mirror of the service rather than
the leader.
(`2681 <https://github.com/vertexproject/synapse/pull/2681>`_)
- Add ``$lib.inet.http.urlencode()`` and ``$lib.inet.http.urldecode()`` Storm
APIs for handling URL encoding.
(`2688 <https://github.com/vertexproject/synapse/pull/2688>`_)
- Add type validation for all Cell configuration options throughout the
lifetime of the Cell and all operations which modify its configuration
values. This prevents invalid values from being persisted on disk.
(`2687 <https://github.com/vertexproject/synapse/pull/2687>`_)
(`2691 <https://github.com/vertexproject/synapse/pull/2691>`_)

Bugfixes
--------
- Fix an issue where the ``=`` sign in the Storm grammar was assigned an
anonymous terminal name by the grammar parser. This caused an issue with
interpreting various syntax errors.
(`2690 <https://github.com/vertexproject/synapse/pull/2690>`_)

====================

Automatic Migrations
--------------------
- Re-normalize the migrated properties noted in the data model updates listed
below. See :ref:`datamigration` for more information about automatic
migrations.

Features and Enhancements
-------------------------
- Updates to the ``crypto``, ``infotech``, ``ou``, and ``person`` models.
(`2620 <https://github.com/vertexproject/synapse/pull/2620>`_)
(`2684 <https://github.com/vertexproject/synapse/pull/2684>`_)

``crypto:algorithm``
Add a form to represent a named cryptography algorithm.

``crypto:key``
Add a form to represent a cryptographic key and algorithm.

``crypto:smart:effect:transfertoken``
Add a form to represent the effect of transferring ownership of a
non-fungible token.

``crypto:smart:effect:transfertokens``
Add a form to represent the effect of transferring multiple fungible
tokens.

``crypto:smart:effect:edittokensupply``
Add a form to represent the increase or decrease in the supply of
fungible tokens.

``it:prod:softname``
Add a form to represent a software name.

``it:host``
Add a ``:os:name`` secondary property.

``it:mitre:attack:software``
Migrate the ``:name`` and ``:names`` properties to ``it:prod:softname``
type.

``it:prod:soft``
Migrate the ``:name`` and ``:names`` properties to ``it:prod:softname``
type.

``it:prod:softver``
Deprecate the ``:software:name`` property.
Migrate the ``:name`` and ``:names`` properties to ``it:prod:softname``
type.

``it:app:yara:rule``
Add a ``:family`` property to represent the software family the rule is
designed to detect.

``it:sec:c2:config``
Add a form to represent C2 configuration data.

``ou:campaign``
Add a ``:org:name`` property to represent the name of the organization
responsible the campaign.
Add a ``:org:fqdn`` property to represent the fqdn of the organization
responsible the campaign.
Add a ``:team`` property to represent the team responsible for the
campaign.

``ou:team``
Add a form to represent a team within an organization.

``ou:industry``
Migrate the ``:name`` property to ``ou:industryname`` type.
Add a ``:names`` property for alternative names.

``ou:industryname``
Add a form to represent the name of an industry.

``ou:position``
Add a ``:team`` property to represent the team associated with a given
position.

``ps:contact``
Add a ``:crypto:address`` property to represent the crypto currency
address associated with the contact.

- Add ``$lib.copy()`` to Storm. This allows making copies of objects which
are compatible with being serialized with msgpack.
(`2678 <https://github.com/vertexproject/synapse/pull/2678>`_)
- Remove `print` events from the Storm `limit` command.
(`2674 <https://github.com/vertexproject/synapse/pull/2674>`_)

Bugfixes
--------
- Fix an issue where client certificates presented in Telepath ``ssl``
connections could fallback to resolving users by a prefix. This was not
intended to be allowed when client certificates are used with Telepath.
(`2675 <https://github.com/vertexproject/synapse/pull/2675>`_)
- Fix an issue where ``node:del`` triggers could fail to fire when adding
nodeedits directly to a view or snap.
(`2654 <https://github.com/vertexproject/synapse/pull/2654>`_)
- Fix header escaping when generating autodoc content for Synapse Cells.
(`2677 <https://github.com/vertexproject/synapse/pull/2677>`_)
- Assorted unit tests fixes to make tests more stable.
(`2680 <https://github.com/vertexproject/synapse/pull/2680>`_)
- Fix an issue with Storm function argument parsing.
(`2685 <https://github.com/vertexproject/synapse/pull/2685>`_)

Improved Documentation
----------------------
- Add an introduction to Storm libraries and types.
(`2670 <https://github.com/vertexproject/synapse/pull/2670>`_)
(`2683 <https://github.com/vertexproject/synapse/pull/2683>`_)
- Fix small typos and corrections in the devops documentation.
(`2673 <https://github.com/vertexproject/synapse/pull/2673>`_)

====================

Features and Enhancements
-------------------------
- Updates to the ``inet`` and ``infotech`` models.
(`2666 <https://github.com/vertexproject/synapse/pull/2666>`_)

``:sandbox:file``
Add a ``sandbox:file`` property to record an initial sample from a
sandbox environment to the following forms:

``it:exec:proc``
``it:exec:thread``
``it:exec:loadlib``
``it:exec:mmap``
``it:exec:mutex``
``it:exec:pipe``
``it:exec:url``
``it:exec:bind``
``it:exec:file:add``
``it:exec:file:del``
``it:exec:file:read``
``it:exec:file:write``
``it:exec:reg:del``
``it:exec:reg:get``
``it:exec:reg:set``


``it:host:activity``
Update the interface to add a ``sandbox:file`` property to record an
initial sample from a sandbox environment.

- Changed primary Storm parser to a LALR compatible syntax to gain 80x speed
up in parsing Storm queries
(`2649 <https://github.com/vertexproject/synapse/pull/2649>`_)
- Added service provisioning API to AHA service and associated tool
``synapse.tools.aha.provision.service`` and documentation to make
it easy to bootstrap Synapse services using service discovery and
SSL client-side certificates to identify service accounts.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Added user provisioning API to AHA service and associated tools
``synapse.tools.aha.provision.user`` and ``synapse.tools.aha.enroll``
to make it easy to bootstrap new users with SSL client-side certificates
and AHA service discovery configuration.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Added automatic mirror initialization logic to Synapse services to
enable new mirrors to be initilized dynamically via AHA provisioning
rather than from a pre-existing backup.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Added ``handoff()`` API to Synapse services to allow mirrors to be
gracefully promoted to leader.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Added ``synapse.tools.promote`` to allow easy promotion of mirror to
leader using the new ``handoff()`` API.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Added ``aha:provision`` configuration to Synapse services to allow
them to automatically provision and self-configure using AHA.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Adjusted Synapse service configuration preference to allow runtime settings
to be stored in ``cell.yaml``.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Added optional ``certhash`` parameter to telepath ``ssl://`` URLs to
allow cert-pinning behavior and automatic trust of provisioning URLs.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Added ``synapse.tools.moduser`` and ``synapse.tools.modrole`` commands
to modernize and ease user/role management from within Synapse service
docker containers.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Add ``$lib.jsonstor.cacheget()`` and ``lib.jsonstor.cacheset()`` functions
in Storm to easily implement data caching in the JSONStor.
(`2662 <https://github.com/vertexproject/synapse/pull/2662>`_)
- Add a ``params`` option to ``$lib.inet.http.connect()`` to pass parameters
when creating Websocket connections in Storm.
(`2664 <https://github.com/vertexproject/synapse/pull/2664>`_)

Bugfixes
--------
- Added ``getCellRunId()`` API to Synapse services to allow them to detect
incorrect mirror configurations where they refer to themselves.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Ensure that CLI history files can be read and written upon
starting interactive CLI tools.
(`2660 <https://github.com/vertexproject/synapse/pull/2660>`_)
- Assorted unit tests fixes to make tests more stable.
(`2656 <https://github.com/vertexproject/synapse/pull/2656>`_)
(`2665 <https://github.com/vertexproject/synapse/pull/2665>`_)
- Fix several uses of Python features which are formally deprecated
and may be removed in future Python versions.
(`2668 <https://github.com/vertexproject/synapse/pull/2668>`_)

Improved Documentation
----------------------
- Added new Deployment Guide with step-by-step production ready deployment
instructions
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Refactored Devops Guide to give task-oriented instructions on performing
common devops tasks.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Added new minimal Admin Guide as a place for documenting Cortex admin tasks.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Updated Getting Started to direct users to synapse-quickstart instructions.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Added ``easycert`` tool documentation.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Removed ``cmdr`` tool documentation to emphasize newer tools such as
``storm``.
(`2641 <https://github.com/vertexproject/synapse/pull/2641>`_)
- Update the list of available Advanced and Rapid Power-Ups.
(`2667 <https://github.com/vertexproject/synapse/pull/2667>`_)