Openwisp-controller

--------------------------

Work in progress.

--------------------------

Features
~~~~~~~~

- Added `support for ZeroTier
<https://openwisp.io/docs/stable/controller/user/zerotier.html>`_.
- Added support for adding `configuration templates
<https://openwisp.io/docs/stable/controller/user/device-groups.html#group-templates>`_
and `configuration variables
<https://openwisp.io/docs/stable/controller/user/device-groups.html#group-configuration-variables>`_
to device groups.
- Implemented support for `SNMP credentials
<https://openwisp.io/docs/stable/controller/user/intro.html#snmp>`_.
- Added support for device deactivation and reactivation.
- Introduced an admin action to quickly move devices between groups.
- Added autocomplete support for filters in the admin interface.
- Added support to display configuration errors reported by the OpenWISP
config agent.
- Enabled filtering of subnets by Subnet Division Rule type.
- Allowed `limiting the number of devices per organization
<https://openwisp.io/docs/stable/controller/user/organization-limits.html>`_.
- Added support for defining organization-level variables.
- Enabled `bulk import and export of devices
<https://openwisp.io/docs/stable/controller/user/import-export.html>`_.
- Added `functionality to configure available commands for each
organization
<https://openwisp.io/docs/stable/controller/user/settings.html#openwisp-controller-organization-enabled-commands>`_.
- Added a dashboard chart showing the distribution of devices across
different groups.
- Added filters to the REST API.

Changes
~~~~~~~

- Removing a VPN template from a device will not deleted the certificates
associated with the ``VPNClient``. Instead, the certificates will be
marked as revoked.
- Allowed reusing ``VNI`` across all the tunnels created using an instance
VXLAN over WireGuard VPN.
- Added a data migration for updating ``hwmode`` configuration option of
radio to "band". The ``hwmode`` option is deprecated on OpenWrt > 21.
- The configuration push update will not flag the configuration status as
"applied". The status will be update once the OpenWISP Config agent on
the device reports the status.
- Updated ``DeviceConnection.connect`` to attempt all available
credentials.
- Changed the ``on_cascade`` property of ``BaseDeviceLocation.location``
and ``BaseDeviceLocation.floorplan`` from ``PROTECTED`` to ``CASCADE``.
- Controller views will return ``HTTP 404`` response for devices belonging
to disabled organizations.
- Show a loading indicator for commands in progress.
- Added VPN subnet CIDR to device's system-defined variables.
- Allowed defining subnet and IP address for VPNs with OpenVPN backend.
- Changed the target link for configuration error notifications to the
"Configuration" tab of the device.
- JSONSchema Editor widget allows to define extra CSS classes. It will
ignore fields with ``manual`` class.
- Increased the soft time limit for celery task that operates of bulk
objects.
- Added notifications for background subnet division rule errors.
- Added `name` and `mac_address` to device list filters in the API.
- Use autocomplete fields for related fields.

**Dependencies**:

- Bumped ``django-sortedm2m~=4.0.0``.
- Bumped ``django-reversion~=5.1.0``.
- Bumped ``django-taggit~=4.0.0``.
- Bumped ``netjsonconfig~=1.1.0``.
- Bumped ``django-x509~=1.2.0``.
- Bumped ``django-loci~=1.1.0``.
- Bumped ``django-flat-json-widget~=0.3.0``.
- Bumped ``openwisp-users~=1.1.0``.
- Bumped ``openwisp-utils[celery]~=1.1.1``.
- Bumped ``openwisp-notifications~=1.1.0``.
- Bumped ``openwisp-ipam~=1.1.0``.
- Bumped ``djangorestframework-gis~=1.1``.
- Bumped ``paramiko[ed25519]~=3.5.0``.
- Bumped ``scp~=0.15.0``.
- Bumped ``django-cache-memoize~=0.2.0``.
- Bumped ``shortuuid~=1.0.13``.
- Bumped ``netaddr~=1.3.0``.
- Bumped ``django-import-export~=3.3.0``.
- Added support for Django ``4.1.x`` and ``4.2.x``.
- Added support for Python ``3.10``.
- Dropped support for Python ``3.7``.
- Dropped support for Django ``3.0.x`` and ``3.1.x``.

Bugfixes
~~~~~~~~

- Fixed `bug for displaying relevant templates where the backend of the
device's configuration template does not match
<https://github.com/openwisp/openwisp-controller/pull/771>`_.
- Fixed `displaying the command widget when the user has write permissions
<https://github.com/openwisp/openwisp-controller/pull/854>`_.
- Fixed `DeviceConnection.get_working_connection to handle scenario where
the device has no credentials
<https://github.com/openwisp/openwisp-controller/pull/720>`_.
- `User need to have required model permissions to perform admin actions
<https://github.com/openwisp/openwisp-controller/pull/873>`_.
- Fixed `import device preview table when browser is set to dark mode
<https://github.com/openwisp/openwisp-controller/issues/851>`_.
- Fixed `subnet division rule does not allow assigning only 1 ip in /32
<https://github.com/openwisp/openwisp-controller/issues/842>`_.
- Fixed `Device._has_group() wrongly returning True
<https://github.com/openwisp/openwisp-controller/pull/804>`_.
- Fixed `JS of history pages on config app
<https://github.com/openwisp/openwisp-controller/issues/681>`_.
- Fixed `REST API can creates device configs inadvertently
<https://github.com/openwisp/openwisp-controller/issues/699>`_.
- Fixed `broken shared template preview
<https://github.com/openwisp/openwisp-controller/issues/742>`_.
- Fixed `command APIs permissions
<https://github.com/openwisp/openwisp-controller/issues/754>`_.
- Fixed `deleting VpnClient result in deleting all provisioned subnets of
the device <https://github.com/openwisp/openwisp-controller/pull/805>`_.
- Fixed `group templates re-creating VPN clients
<https://github.com/openwisp/openwisp-controller/issues/703>`_.
- Fixed `DeviceListSerializer returning HTTP 500 instead of HTTP 400
reponse on invalid data
<https://github.com/openwisp/openwisp-controller/issues/695>`_.
- Fixed `re-ordering applied templates in device config
<https://github.com/openwisp/openwisp-controller/pull/830>`_.
- Fixed `re-ordering templates on device add page
<https://github.com/openwisp/openwisp-controller/issues/434>`_.
- Fixed `several issues with clone template feature
<https://github.com/openwisp/openwisp-controller/pull/838>`_.
- Fixed `updating template organization puts devices in perennial
"modified" state
<https://github.com/openwisp/openwisp-controller/issues/213>`_.
- Fixed `user defined commands that do not require input
<https://github.com/openwisp/openwisp-controller/pull/871>`_.
- Fixed `validation for SubnetDivisionRule
<https://github.com/openwisp/openwisp-controller/issues/706>`_.
- Fixed `subnet division rule validation when master subnet is empty
<https://github.com/openwisp/openwisp-controller/issues/866>`_.
- Fixed `validation in change device group admin action
<https://github.com/openwisp/openwisp-controller/issues/762>`_.
- `Increased "timeoutInterval" for ReconnectingWebSocket
<https://github.com/openwisp/openwisp-controller/issues/772>`_.
- Added `JS workaround for using mac address variable
<https://github.com/openwisp/openwisp-controller/pull/876>`_.
- Fixed `same credential can be added to a device twice
<https://github.com/openwisp/openwisp-controller/issues/795>`_.
- `Show reversion button to operators too
<https://github.com/openwisp/openwisp-controller/pull/652>`_.
- Fixed `unsaved changes alert triggering on previewing configuration
<https://github.com/openwisp/openwisp-controller/pull/857>`_.
- Fixed `dependency on the creation date in get_max_subnet method
<https://github.com/openwisp/openwisp-controller/issues/728>`_.
- Fixed `Vpn.webhook_endpoint accepting invalid URL
<https://github.com/openwisp/openwisp-controller/issues/689>`_.
- Fixed `object and config menu not opening in Device config editor
<https://github.com/openwisp/openwisp-controller/pull/913>`.

--------------------------

Bugfixes
~~~~~~~~

- `Fixed tests failing due to openwisp-notification>=1.0.2
<https://github.com/openwisp/openwisp-controller/pull/670>`_
- `Fixed checksum cache is not invalidated on VPN server change
<https://github.com/openwisp/openwisp-controller/issues/667>`_

--------------------------

Bugfixes
~~~~~~~~

- Fixed `device's "changed" signals emitting on the creation of new device
<https://github.com/openwisp/openwisp-controller/issues/649>`_
- Fixed *django-reversion's* recovery buttons were hidden from users of
the "Operator" group in the admin dashboard of ``Certificate`` and
``CA`` models
- Removed `hardcoded static URLs
<https://github.com/openwisp/openwisp-controller/issues/660>`_ which
created issues when static files are served using an external service
(e.g. S3 storage buckets)
- Fixed `permissions for "Operator" and "Administrator" groups to access
"OrganizationConfigSettings" objects
<https://github.com/openwisp/openwisp-controller/issues/664>`_
- Fixed `support for multiple wireguard tunnels on the same devices
<https://github.com/openwisp/openwisp-controller/issues/657>`_
- Fixed `"/api/v1/controller/device/{id}/" REST API endpoint not updating
the device's configuration backend
<https://github.com/openwisp/openwisp-controller/issues/658>`_

--------------------------

Bugfixes
~~~~~~~~

- Admin: show main group information in ``DeviceGroupAdmin`` list: - name
- organization - modified - created
- Fixed uncaught exception triggered on the deletion of VPN client
certificates
- SSH connection: fixed OpenWrt <= 19 authentication failure
- The SSH connection is now explicitly closed when the authentication
fails to avoid leaving lingering SSH connection objects open

--------------------------

Features
~~~~~~~~

- Added support for `remotely executing shell commands on device
<https://github.com/openwisp/openwisp-controller#sending-commands-to-devices>`_
- Added `automatic provisioning of Subnets and IPs
<https://github.com/openwisp/openwisp-controller#subnet-division-app>`_
- Added `support for WireGuard and VXLAN tunnels
<https://github.com/openwisp/openwisp-controller#how-to-setup-wireguard-tunnels>`_
- Added `required templates
<https://github.com/openwisp/openwisp-controller#required-templates>`_
- Added support for generating configurations for OpenWrt 21
- Added `REST API
<https://github.com/openwisp/openwisp-controller#rest-api-reference>`_
- Added dashboard charts for *config status*, *model*, *OS*, *hardware*
and *location type*
- Added `management_ip_changed
<https://github.com/openwisp/openwisp-controller#management_ip_changed>`_
and `device_name_changed
<https://github.com/openwisp/openwisp-controller#device_name_changed>`_
signals
- Added `OPENWISP_CONTROLLER_DEVICE_NAME_UNIQUE setting
<https://github.com/openwisp/openwisp-controller#openwisp_controller_device_name_unique>`_
to conditionally enforce unique device names in an organization
- Added caching for ``DeviceChecksumView``
- Added support for ED25519 SSH keys in ``Credentials``
- Added `Device Groups
<https://github.com/openwisp/openwisp-controller#device-groups>`_ to
organize devices of a particular organization
- Configuration push updates now use the SIGUSR1 signal to reload
openwisp-config
- The device list admin page now allows to search for location address

Changes
~~~~~~~

Backward incompatible changes
+++++++++++++++++++++++++++++

- Since django-sortedm2m, the widget we use to implement ordered
templates, clears all the many to many relationships every time it has
to make changes, we had to stop deleting ``VpnClient`` instances related
to VPN templates on ``post_clear`` m2m signals If you wrote any custom
derivative which relies on calls like
``device.config.templates.clear()`` to delete related ``VpnClient``
instances and their x509 certificates, you will have to update your code
to remove all the templates using their primary keys, instead of using
``clear()``
- The default behavior for the resolution of conflicting management IPs
between devices of different organizations has been changed; by default,
in this new version, the system assumes it's using only 1 management
tunnel for all the organizations, so different devices from any
organization will not have the same management IP to avoid conflicts.
The old behaviour can be restored by setting
`OPENWISP_CONTROLLER_SHARED_MANAGEMENT_IP_ADDRESS_SPACE
<https://github.com/openwisp/openwisp-controller#openwisp_controller_shared_management_ip_address_space>`_
to ``False``
- ``OPENWISP_CONTROLLER_BACKEND_DEVICE_LIST`` has been renamed to
``OPENWISP_CONTROLLER_CONFIG_BACKEND_FIELD_SHOWN``
- ``Device.check_management_ip_changed`` has been changed to private API
``Device._check_management_ip_changed``

Dependencies
++++++++++++

- Dropped support for Python 3.6
- Dropped support for Django 2.2
- Added support for Python 3.8 and 3.9
- Added support for Django 3.2 and 4.0
- Upgraded django-sortedm2m to 3.1.x
- Upgraded django-reversion to 4.0.x
- Upgraded django-taggit to 2.1.x
- Upgraded djangorestframework-gis to 0.18.0
- Upgraded paramiko[ed25519] to 2.10.3
- Upgraded scp to 0.14.2
- Upgraded django-flat-json-widget to 0.2.x
- Upgraded celery to 5.2.x
- Upgraded channels to 3.0.x
- Upgraded django-x509 to 1.1.x
- Upgraded django-loci to 1.0.x
- Upgraded netjsonconfig to 1.0.x
- Upgraded openwisp-utils to 1.0.x
- Upgraded openwisp-users to 1.0.x
- Upgraded openwisp-notifications to 1.0.x
- Upgraded openwisp-ipam to 1.0.x
- Added shortuuid 1.0.x
- Added netaddr 0.8.x
- Added django-cache-memoize to 0.1

Other changes
+++++++++++++

- `Reworked implementation of config_modified signal
<https://github.com/openwisp/openwisp-controller#config_modified>`_:

- the signal is now always emitted on templates changes m2m events, also
if ``config.status`` is modified, with the differences that only
post_add and post_remove m2m events are used, while ``post_clear`` is
ignored, which fixes the duplicate signal emission caused by the
implementation of sortedm2m;
- added ``action`` and ``previous_status`` arguments, which allow to
understand where the ``config_modified`` signal is being emitted from,
this allows more advanced usage of the signal by custom
implementations

- Context variable follows template order: If two or more applied
templates have "default_values" with the same keys, then the context
variables of the template which comes later in the order will be used
- New credentials created with ``auto_add`` set to ``True`` will get added
to the existing devices in a background task. This improves the
responsiveness of the web application
- Decoupled admin LogEntry from Template model
- Device admin only lists relevant templates, i.e. templates that are
shared or belong to the device's organization
- Improved UX of `system-defined variables
<https://github.com/openwisp/openwisp-controller/issues/344>`_
- Name of ``Vpn``, ``Template`` and ``Credentials`` objects is unique only
within the same organization and within the shared objects
- Added functionality to configure connection failure reasons for which
the system should not send notifications. Added ``old_failure_reason``
parameter in
``openwisp_controller.connection.signals.is_working_changed`` signal
- Allowed searching devices using their location address in Device admin.
- Removed deprecated ``api/device-location/<pk>`` endpoint
- Made device name unique per organization instead of unique system wide
- Added time limits to background celery tasks

Bugfixes
~~~~~~~~

- Fixed a bug which caused ``VpnClient`` instances to be recreated every
time the configuration templates of a device were changed, which caused
x590 certificates to be destroyed and recreated as well
- Hardened config validation of OpenVPN backend. The validation fails if
the ``openvpn`` key is missing from the configuration
- Fixed a bug that caused issues in updating related ``Config`` whenever a
template's ``default_values`` were changed
- Fixed pop-up view of CA and Cert not displaying data
- Fixed config status stays ``applied`` after clearing all device
templates
- Fixed ``VpnClient`` not created when multiple VPN templates are added
- Fixed configuration editor raising validation error when using variables
in fields with ``maxLength`` set
- Fixed connection notifications reporting outdated status
- Fixed migrations referencing non-swappable OpenWISP modules that broke
OpenWISP's extensibility
- Fixed bugs in restoring deleted devices using ``django-reversion``
- Fixed cloning of shared templates
- Disallowed blank values for ``key_length`` or ``digest`` fields for
``CA`` and ``Cert`` objects
- Fixed template ordering bug in the configuration preview on Device admin
The order of templates was not always retained when generating the
preview of a config object