Patroni

-------------

Released 2023-09-20

**Bugfixes**

- Reset failsafe state on promote (ChenChangAo)

If switchover/failover happened shortly after failsafe mode had been activated, the newly promoted primary was demoting itself after failsafe becomes inactive.

- Silence useless warnings in ``patronictl`` (Alexander Kukushkin)

If ``patronictl`` uses the same patroni.yaml file as Patroni and can access ``PGDATA`` directory it might have been showing annoying warnings about incorrect values in the global configuration.

- Explicitly enable synchronous mode for a corner case (Alexander Kukushkin)

Synchronous mode effectively was never activated if there are no replicas streaming from the primary.

- Fixed bug with ``0`` integer values validation (Israel Barth Rubio)

In most cases, it didn't cause any issues, just warnings.

- Don't return logical slots for standby cluster (Alexander Kukushkin)

Patroni can't create logical replication slots in the standby cluster, thus they should be ignored if they are defined in the global configuration.

- Avoid showing docstring in ``patronictl --help`` output (Israel Barth Rubio)

The ``click`` module needs to get a special hint for that.

- Fixed bug with ``kubernetes.standby_leader_label_value`` (Alexander Kukushkin)

This feature effectively never worked.

- Returned cluster system identifier to the ``patronictl list`` output (Polina Bungina)

The problem was introduced while implementing the support for Citus, where we need to hide the identifier because it is different for coordinator and all workers.

- Override ``write_leader_optime`` method in Kubernetes implementation (Alexander Kukushkin)

The method is supposed to write shutdown LSN to the leader Endpoint/ConfigMap when there are no healthy replicas available to become the new primary.

- Don't start stopped postgres in pause (Alexander Kukushkin)

Due to a race condition, Patroni was falsely assuming that the standby should be restarted because some recovery parameters (``primary_conninfo`` or similar) were changed.

- Fixed bug in ``patronictl query`` command (Israel Barth Rubio)

It didn't work when only ``-m`` argument was provided or when none of ``-r`` or ``-m`` were provided.

- Properly treat integer parameters that are used in the command line to start postgres (Polina Bungina)

If values are supplied as strings and not casted to integer it was resulting in an incorrect calculation of ``max_prepared_transactions`` based on ``max_connections`` for Citus clusters.

- Don't rely on ``pg_stat_wal_receiver`` when deciding on ``pg_rewind`` (Alexander Kukushkin)

It could happen that ``received_tli`` reported by ``pg_stat_wal_recevier`` is ahead of the actual replayed timeline, while the timeline reported by ``DENTIFY_SYSTEM`` via replication connection is always correct.

-------------

Released 2023-08-03

**Breaking changes**

- Changed semantic of ``restapi.keyfile`` and ``restapi.certfile`` (Alexander Kukushkin)

Previously Patroni was using ``restapi.keyfile`` and ``restapi.certfile`` as client certificates as a fallback if there were no respective configuration parameters in the ``ctl`` section.

.. warning::
If you enabled client certificates validation (``restapi.verify_client`` is set to ``required``), you also **must** provide **valid client certificates** in the ``ctl.certfile``, ``ctl.keyfile``, ``ctl.keyfile_password``. If not provided, Patroni will not work correctly.


**New features**

- Make Pod role label configurable (Waynerv)

Values could be customized using ``kubernetes.leader_label_value``, ``kubernetes.follower_label_value`` and ``kubernetes.standby_leader_label_value`` parameters. This feature will be very useful when we change the ``master`` role to the ``primary``. You can read more about the feature and migration steps :ref:`here <kubernetes_role_values>`.


**Improvements**

- Various improvements of ``patroni --validate-config`` (Alexander Kukushkin)

Improved parameter validation for different DCS, ``bootstrap.dcs`` , ``ctl``, ``restapi``, and ``watchdog`` sections.

- Start Postgres not in recovery if it crashed during recovery while Patroni is running (Alexander Kukushkin)

It may reduce recovery time and will help to prevent unnecessary timeline increments.

- Avoid unnecessary updates of ``/status`` key (Alexander Kukushkin)

When there are no permanent logical slots Patroni was updating the ``/status`` on every heartbeat loop even when LSN on the primary didn't move forward.

- Don't allow stale primary to win the leader race (Alexander Kukushkin)

If Patroni was hanging during a significant time due to lack of resources it will additionally check that no other nodes promoted Postgres before acquiring the leader lock.

- Implemented visibility of certain PostgreSQL parameters validation (Alexander Kukushkin, Feike Steenbergen)

If validation of ``max_connections``, ``max_wal_senders``, ``max_prepared_transactions``, ``max_locks_per_transaction``, ``max_replication_slots``, or ``max_worker_processes`` failed Patroni was using some sane default value. Now in addition to that it will also show a warning.

- Set permissions for files and directories created in ``PGDATA`` (Alexander Kukushkin)

All files created by Patroni had only owner read/write permissions. This behaviour was breaking backup tools that run under a different user and relying on group read permissions. Now Patroni honors permissions on ``PGDATA`` and correctly sets permissions on all directories and files it creates inside ``PGDATA``.


**Bugfixes**

- Run ``archive_command`` through shell (Waynerv)

Patroni might archive some WAL segments before doing crash recovery in a single-user mode or before ``pg_rewind``. If the archive_command contains some shell operators, like ``&&`` it didn't work with Patroni.

- Fixed "on switchover" shutdown checks (Polina Bungina)

It was possible that specified candidate is still streaming and didn't received shut down checking but the leader key was removed because some other nodes were healthy.

- Fixed "is primary" check (Alexander Kukushkin)

During the leader race replicas were not able to recognize that Postgres on the old leader is still running as a primary.

- Fixed ``patronictl list`` (Alexander Kukushkin)

The Cluster name field was missing in ``tsv``, ``json``, and ``yaml`` output formats.

- Fixed ``pg_rewind`` behaviour after pause (Alexander Kukushkin)

Under certain conditions, Patroni wasn't able to join the false primary back to the cluster with ``pg_rewind`` after coming out of maintenance mode.

- Fixed bug in Etcd v3 implementation (Alexander Kukushkin)

Invalidate internal KV cache if key update performed using ``create_revision``/``mod_revision`` field due to revision mismatch.

- Fixed behaviour of replicas in standby cluster in pause (Alexander Kukushkin)

When the leader key expires replicas in standby cluster will not follow the remote node but keep ``primary_conninfo`` as it is.

-------------

Released 2023-07-13

**New features**

- Make the replication status of standby nodes visible (Alexander Kukushkin)

For PostgreSQL 9.6+ Patroni will report the replication state as ``streaming`` when the standby is streaming from the other node or ``in archive recovery`` when there is no replication connection and ``restore_command`` is set. The state is visible in ``member`` keys in DCS, in the REST API, and in ``patronictl list`` output.


**Improvements**

- Improved error messages with Etcd v3 (Alexander Kukushkin)

When Etcd v3 cluster isn't accessible Patroni was reporting that it can't access ``/v2`` endpoints.

- Use quorum read in ``patronictl`` if it is possible (Alexander Kukushkin)

Etcd or Consul clusters could be degraded to read-only, but from the ``patronictl`` view everything was fine. Now it will fail with the error.

- Prevent splitbrain from duplicate names in configuration (Mark Pekala)

When starting Patroni will check if node with the same name is registered in DCS, and try to query its REST API. If REST API is accessible Patroni exits with an error. It will help to protect from the human error.

- Start Postgres not in recovery if it crashed while Patroni is running (Alexander Kukushkin)

It may reduce recovery time and will help from unnecessary timeline increments.


**Bugfixes**

- REST API SSL certificate were not reloaded upon receiving a SIGHUP (Israel Barth Rubio)

Regression was introduced in 3.0.3.

- Fixed integer GUCs validation for parameters like ``max_connections`` (Feike Steenbergen)

Patroni didn't like quoted numeric values. Regression was introduced in 3.0.3.

- Fix issue with ``synchronous_mode`` (Alexander Kukushkin)

Execute ``txid_current()`` with ``synchronous_commit=off`` so it doesn't accidentally wait for absent synchronous standbys when ``synchronous_mode_strict`` is enabled.

-------------

Released 2023-06-22

**New features**

- Compatibility with PostgreSQL 16 beta1 (Alexander Kukushkin)

Extended GUC's validator rules.

- Make PostgreSQL GUC's validator extensible (Israel Barth Rubio)

Validator rules are loaded from YAML files located in ``patroni/postgresql/available_parameters/`` directory. Files are ordered in alphabetical order and applied one after another. It makes possible to have custom validators for non-standard Postgres distributions.

- Added ``restapi.request_queue_size`` option (Andrey Zhidenkov, Aleksei Sukhov)

Sets request queue size for TCP socket used by Patroni REST API. Once the queue is full, further requests get a "Connection denied" error. The default value is 5.

- Call ``initdb`` directly when initializing a new cluster (Matt Baker)

Previously it was called via ``pg_ctl``, what required a special quoting of parameters passed to ``initdb``.

- Added before stop hook (Le Duane)

The hook could be configured via ``postgresql.before_stop`` and is executed right before ``pg_ctl stop``. The exit code doesn't impact shutdown process.

- Added support for custom Postgres binary names (Israel Barth Rubio, Polina Bungina)

When using a custom Postgres distribution it may be the case that the Postgres binaries are compiled with different names other than the ones used by the community Postgres distribution. Custom binary names could be configured using ``postgresql.bin_name.*`` and ``PATRONI_POSTGRESQL_BIN_*`` environment variables.


**Improvements**

- Various improvements of ``patroni --validate-config`` (Polina Bungina)

- Make ``bootstrap.initdb`` optional. It is only required for new clusters, but ``patroni --validate-config`` was complaining if it was missing in the config.
- Don't error out when ``postgresql.bin_dir`` is empty or not set. Try to first find Postgres binaries in the default PATH instead.
- Make ``postgresql.authentication.rewind`` section optional. If it is missing, Patroni is using the superuser.

- Improved error reporting in ``patronictl`` (Israel Barth Rubio)

The ``\n`` symbol was rendered as it is, instead of the actual newline symbol.


**Bugfixes**

- Fixed issue in Citus support (Alexander Kukushkin)

If the REST API call from the promoted worker to the coordinator failed during switchover it was leaving the given Citus group blocked during indefinite time.

- Allow `etcd3` URL in `--dcs-url` option of `patronictl` (Israel Barth Rubio)

If users attempted to pass a `etcd3` URL through `--dcs-url` option of `patronictl` they would face an exception.

-------------

Released 2023-03-24

.. warning::
Version 3.0.2 dropped support of Python older than 3.6.


**New features**

- Added sync standby replica status to ``/metrics`` endpoint (Thomas von Dein, Alexander Kukushkin)

Before were only reporting ``primary``/``standby_leader``/``replica``.

- User-friendly handling of ``PAGER`` in ``patronictl`` (Israel Barth Rubio)

It makes pager configurable via ``PAGER`` environment variable, which overrides default ``less`` and ``more``.

- Make K8s retriable HTTP status code configurable (Alexander Kukushkin)

On some managed platforms it is possible to get status code ``401 Unauthorized``, which sometimes gets resolved after a few retries.


**Improvements**

- Set ``hot_standby`` to ``off`` during custom bootstrap only if ``recovery_target_action`` is set to ``promote`` (Alexander Kukushkin)

It was necessary to make ``recovery_target_action=pause`` work correctly.

- Don't allow ``on_reload`` callback to kill other callbacks (Alexander Kukushkin)

``on_start``/``on_stop``/``on_role_change`` are usually used to add/remove Virtual IP and ``on_reload`` should not interfere with them.

- Switched to ``IMDSFetcher`` in aws callback example script (Polina Bungina)

The ``IMDSv2`` requires a token to work with and the ``IMDSFetcher`` handles it transparently.


**Bugfixes**

- Fixed ``patronictl switchover`` on Citus cluster running on Kubernetes (Lukáš Lalinský)

It didn't work for namespaces different from ``default``.

- Don't write to ``PGDATA`` if major version is not known (Alexander Kukushkin)

If right after the start ``PGDATA`` was empty (maybe wasn't yet mounted), Patroni was making a false assumption about PostgreSQL version and falsely creating ``recovery.conf`` file even if the actual major version is v10+.

- Fixed bug with Citus metadata after coordinator failover (Alexander Kukushkin)

The ``citus_set_coordinator_host()`` call doesn't cause metadata sync and the change was invisible on worker nodes. The issue is solved by switching to ``citus_update_node()``.

- Use etcd hosts listed in the config file as a fallback when all etcd nodes "failed" (Alexander Kukushkin)

The etcd cluster may change topology over time and Patroni tries to follow it. If at some point all nodes became unreachable Patroni will use a combination of nodes from the config plus the last known topology when trying to reconnect.

-------------

Released 2023-02-16

**Bugfixes**

- Pass proper role name to an ``on_role_change`` callback script'. (Alexander Kukushkin, Polina Bungina)

Patroni used to erroneously pass ``promoted`` role to an ``on_role_change`` callback script on promotion. The passed role name changed back to ``master``. This regression was introduced in 3.0.0.