Patroni

-------------

Released 2024-06-17

**Stability improvements**

- Compatibility with Python 3.12 (Alexander Kukushkin)

Handle a new attribute added to ``logging.LogRecord``.


**Bugfixes**

- Fix infinite recursion in ``replicatefrom`` tags handling (Alexander Kukushkin)

As a part of this fix, also improve ``is_physical_slot()`` check and adjust documentation.

- Fix wrong role reporting in standby clusters (Alexander Kukushkin)

``synchronous_standby_names`` and synchronous replication only work on a real primary node and in the case of cascading replication are simply ignored by Postgres. Before this fix, ``patronictl list`` and ``GET /cluster`` were falsely reporting some nodes as synchronous.

- Fix availability of the ``allow_in_place_tablespaces`` GUC (Polina Bungina)

``allow_in_place_tablespaces`` was not only added to PostgreSQL 15 but also backpatched to PostgreSQL 10-14.

-------------

Released 2024-04-04

.. warning::
All older Partoni versions are not compatible with ``ydiff>=1.3``.

There are the following options available to "fix" the problem:

1. upgrade Patroni to the latest version
2. install ``ydiff<1.3`` after installing Patroni
3. install ``cdiff`` module


**New features**

- Add ability to pass ``auth_data`` to Zookeeper client (Aras Mumcuyan)

It allows to specify the authentication credentials to use for the connection.

- Add a contrib script for ``Barman`` integration (Israel Barth Rubio)

Provide an application ``patroni_barman`` that allows to perform ``Barman`` operations remotely and can be used as a custom bootstrap/custom replica method or as an ``on_role_change`` callback. Please check :ref:`here <tools_integration>` for more information.

- Support ``JSON`` log format (alisalemmi)

Apart from ``plain`` (default), Patroni now also supports ``json`` log format. Requires ``python-json-logger>=2.0.2`` library to be installed.

- Show ``pending_restart_reason`` information (Polina Bungina)

Provide extended information about the PostgreSQL parameters that caused ``pending_restart`` flag to be set. Both ``patronictl list`` and ``/patroni`` REST API endpoint now show the parameters names and their "diff" as ``pending_restart_reason``.

- Implement ``nostream`` tag (Grigory Smolkin)

If ``nostream`` tag is set to ``true``, the node will not use replication protocol to stream WAL but instead rely on archive recovery (if ``restore_command`` is configured). It also disables copying and synchronization of permanent logical replication slots on the node itself and all its cascading replicas.


**Improvements**

- Implement validation of the ``log`` section (Alexander Kukushkin)

Until now validator was not checking the correctness of the logging configuration provided.

- Improve logging for PostgreSQL parameters change (Polina Bungina)

Convert old values to a human-readable format and log information about the ``pg_controldata`` vs Patroni global configuration mismatch.


**Bugfixes**

- Properly filter out not allowed ``pg_basebackup`` options (Israel Barth Rubio)

Due to a bug, Patroni was not properly filtering out the not allowed options configured for the ``basebackup`` replica bootstrap method, when provided in the ``- setting: value`` format.

- Fix ``etcd3`` authentication error handling (Alexander Kukushkin)

Always retry one time on ``etcd3`` authentication error if authentication was not done right before executing the request. Also, do not restart watchers on reauthentication.

- Improve logic of the validator files discovery (Waynerv)

Use ``importlib`` library to discover the files with available configuration parameters when possible (for Python 3.9+). This implementation is more stable and doesn't break the Patroni distributions based on ``zip`` archives.

- Use ``target_session_attrs`` only when multiple hosts are specified in the ``standby_cluster`` section (Alexander Kukushkin)

``target_session_attrs=read-write`` is now added to the ``primary_conninfo`` on the standby leader node only when ``standby_cluster.host`` section contains multiple hosts separated by commas.

- Add compatibility code for ``ydiff`` library version 1.3+ (Alexander Kukushkin)

Patroni is relying on some API from ``ydiff`` that is not public because it is supposed to be just a terminal tool rather than a python module. Unfortunately, the API change in 1.3 broke old Patroni versions.

-------------

Released 2024-01-17

**Bugfixes**

- Don't let replica restore initialize key when DCS was wiped (Alexander Kukushkin)

It was happening in the method where Patroni was supposed to take over a standalone PG cluster.

- Use consistent read when fetching just updated sync key from Consul (Alexander Kukushkin)

Consul doesn't provide any interface to immediately get ``ModifyIndex`` for the key that we just updated, therefore we have to perform an explicit read operation. Since stale reads are allowed by default, we sometimes used to get an outdated version of the key.

- Reload Postgres config if a parameter that requires restart was reset to the original value (Polina Bungina)

Previously Patroni wasn't updating the config, but only resetting the ``pending_restart``.

- Fix erroneous inverted logic of the confirmation prompt message when doing a failover to an async candidate in synchronous mode (Polina Bungina)

The problem existed only in ``patronictl``.

- Exclude leader from failover candidates in ``patronictl`` (Polina Bungina)

If the cluster is healthy, failing over to an existing leader is no-op.

- Create Citus database and extension idempotently (Alexander Kukushkin, Zhao Junwang)

It will allow to create them in the ``post_bootstrap`` script in case if there is a need to add some more dependencies to the Citus database.

- Don't filter our contradictory ``nofailover`` tag (Polina Bungina)

The configuration ``{nofailover: false, failover_priority: 0}`` set on a node didn't allow it to participate in the race, while it should, because ``nofailover`` tag should take precedence.

- Fixed PyInstaller frozen issue (Sophia Ruan)

The ``freeze_support()`` was called after ``argparse`` and as a result, Patroni wasn't able to start Postgres.

- Fixed bug in the config generator for ``patronictl`` and ``Citus`` configuration (Israel Barth Rubio)

It prevented ``patronictl`` and ``Citus`` configuration parameters set via environment variables from being written into the generated config.

- Restore recovery GUCs and some Patroni-managed parameters when joining a running standby (Alexander Kukushkin)

Patroni was failing to restart Postgres v12 onwards with an error about missing ``port`` in one of the internal structures.

- Fixes around ``pending_restart`` flag (Polina Bungina)

Don't expose ``pending_restart`` when in custom bootstrap with ``recovery_target_action = promote`` or when someone changed ``hot_standby`` or ``wal_log_hints`` using for example ``ALTER SYSTEM``.

-------------

Released 2023-11-30

**Bugfixes**

- Limit accepted values for ``--format`` argument in ``patronictl`` (Alexander Kukushkin)

It used to accept any arbitrary string and produce no output if the value wasn't recognized.

- Verify that replica nodes received checkpoint LSN on shutdown before releasing the leader key (Alexander Kukushkin)

Previously in some cases, we were using LSN of the SWITCH record that is followed by CHECKPOINT (if archiving mode is enabled). As a result the former primary sometimes had to do ``pg_rewind``, but there would be no data loss involved.

- Do a real HTTP request when performing node name uniqueness check (Alexander Kukushkin)

When running Patroni in containers it is possible that the traffic is routed using ``docker-proxy``, which listens on the port and accepts incoming connections. It was causing false positives.

- Fixed Citus support with Etcd v2 (Alexander Kukushkin)

Patroni was failing to deploy a new Citus cluster with Etcd v2.

- Fixed ``pg_rewind`` behavior with Postgres v16+ (Alexander Kukushkin)

The error message format of ``pg_waldump`` changed in v16 which caused ``pg_rewind`` to be called by Patroni even when it was not necessary.

- Fixed bug with custom bootstrap (Alexander Kukushkin)

Patroni was falsely applying ``--command`` argument, which is a bootstrap command itself.

- Fixed the issue with REST API health check endpoints (Sophia Ruan)

There were chances that after Postgres restart it could return ``unknown`` state for Postgres because connections were not properly closed.

- Cache ``postgres --describe-config`` output results (Waynerv)

They are used to figure out which GUCs are available to validate PostgreSQL configuration and we don't expect this list to change while Patroni is running.

-------------

Released 2023-10-25

**Deprecation notice**

- The ``bootstrap.users`` support will be removed in version 4.0.0. If you need to create users after deploying a new cluster please use the ``bootstrap.post_bootstrap`` hook for that.


**Breaking changes**

- Enforce ``loop_wait + 2*retry_timeout <= ttl`` rule and hard-code minimal possible values (Alexander Kukushkin)

Minimal values: ``loop_wait=2``, ``retry_timeout=3``, ``ttl=20``. In case values are smaller or violate the rule they are adjusted and a warning is written to Patroni logs.


**New features**

- Failover priority (Mark Pekala)

With the help of ``tags.failover_priority`` it's now possible to make a node more preferred during the leader race. More details in the documentation (ref tags).

- Implemented ``patroni --generate-config [--dsn DSN]`` and ``patroni --generate-sample-config`` (Polina Bungina)

It allows to generate a config file for the running PostgreSQL cluster or a sample config file for the new Patroni cluster.

- Use a dedicated connection to Postgres for Patroni REST API (Alexander Kukushkin)

It helps to avoid blocking the main heartbeat loop if the system is under stress.

- Enrich some endpoints with the ``name`` of the node (sskserk)

For the monitoring endpoint ``name`` is added next to the ``scope`` and for metrics endpoint the ``name`` is added to tags.

- Ensure strict failover/switchover difference (Polina Bungina)

Be more precise in log messages and allow failing over to an asynchronous node in a healthy synchronous cluster.

- Make permanent physical replication slots behave similarly to permanent logical slots (Alexander Kukushkin)

Create permanent physical replication slots on all nodes that are allowed to become the leader and use ``pg_replication_slot_advance()`` function to advance ``restart_lsn`` for slots on standby nodes.

- Add capability of specifying namespace through ``--dcs`` argument in ``patronictl`` (Israel Barth Rubio)

It could be handy if ``patronictl`` is used without a configuration file.

- Add support for additional parameters in custom bootstrap configuration (Israel Barth Rubio)

Previously it was only possible to add custom arguments to the ``command`` and now one could list them as a mapping.


**Improvements**

- Set ``citus.local_hostname`` GUC to the same value which is used by Patroni to connect to the Postgres (Alexander Kukushkin)

There are cases when Citus wants to have a connection to the local Postgres. By default it uses ``localhost``, which is not always available.


**Bugfixes**

- Ignore ``synchronous_mode`` setting in a standby cluster (Polina Bungina)

Postgres doesn't support cascading synchronous replication and not ignoring ``synchronous_mode`` was breaking a switchover in a standby cluster.

- Handle SIGCHLD for ``on_reload`` callback (Alexander Kukushkin)

Not doing so results in a zombie process, which is reaped only when the next ``on_reload`` is executed.

- Handle ``AuthOldRevision`` error when working with Etcd v3 (Alexander Kukushkin, Kenny Do)

The error is raised if Etcd is configured to use JWT and when the user database in Etcd is updated.

-------------

Released 2023-09-26

**Bugfixes**

- Fixed bug with ``wal_keep_size`` checks (Alexander Kukushkin)

The ``wal_keep_size`` is a GUC that normally has a unit and Patroni was failing to cast its value to ``int``. As a result the value of ``bootstrap.dcs`` was not written to the ``/config`` key afterwards.

- Detect and resolve inconsistencies between ``/sync`` key and ``synchronous_standby_names`` (Alexander Kukushkin)

Normally, Patroni updates ``/sync`` and ``synchronous_standby_names`` in a very specific order, but in case of a bug or when someone manually reset ``synchronous_standby_names``, Patroni was getting into an inconsistent state. As a result it was possible that the failover happens to an asynchronous node.

- Read GUC's values when joining running Postgres (Alexander Kukushkin)

When restarted in ``pause``, Patroni was discarding the ``synchronous_standby_names`` GUC from the ``postgresql.conf``. To solve it and avoid similar issues, Patroni will read GUC's value if it is joining an already running Postgres.

- Silenced annoying warnings when checking for node uniqueness (Alexander Kukushkin)

``WARNING`` messages are produced by ``urllib3`` if Patroni is quickly restarted.