-------------------
This release includes Ansible 2.8 and SELinux support, fixes for two deadlocks,
and major internal design overhauls in preparation for future functionality.
Enhancements
~~~~~~~~~~~~
* :gh:issue:`556`,
:gh:issue:`587`: Ansible 2.8 is supported.
`Become plugins <https://docs.ansible.com/ansible/latest/plugins/become.html>`_ (:gh:issue:`631`) and
`interpreter discovery <https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html>`_ (:gh:issue:`630`)
are not yet handled.
* :gh:issue:`419`, :gh:issue:`470`: file descriptor usage is approximately
halved, as it is no longer necessary to separately manage read and write
sides to work around a design problem.
* :gh:issue:`419`: setup for all connections happens almost entirely on one
thread, reducing contention and context switching early in a run.
* :gh:issue:`419`: Connection setup is better pipelined, eliminating some
network round-trips. Most infrastructure is in place to support future
removal of the final round-trips between a target booting and receiving
function calls.
* :gh:pull:`595`: the :meth:`~mitogen.parent.Router.buildah` connection method
is available to manipulate `Buildah <https://buildah.io/>`_ containers, and
is exposed to Ansible as the :ans:conn:`buildah`.
* :gh:issue:`615`: a modified :ans:mod:`fetch` implements streaming transfer
even when ``become`` is active, avoiding excess CPU and memory spikes, and
improving performance. A representative copy of two 512 MiB files drops from
55.7 seconds to 6.3 seconds, with peak memory usage dropping from 10.7 GiB to
64.8 MiB. [i615]_
* `Operon <https://networkgenomics.com/operon/>`_ no longer requires a custom
library installation, both Ansible and Operon are supported by a single
Mitogen release.
* The ``MITOGEN_CPU_COUNT`` variable shards the connection multiplexer into
per-CPU workers. This may improve throughput for large runs involving file
transfer, and is required for future functionality. One multiplexer starts by
default, to match existing behaviour.
* :gh:commit:`d6faff06`, :gh:commit:`807cbef9`, :gh:commit:`e93762b3`,
:gh:commit:`50bfe4c7`: locking is avoided on hot paths, and some locks are
released before waking a thread that must immediately acquire the same lock.
Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~
* :gh:issue:`363`: fix an obscure race matching *Permission denied* errors from
some versions of :linux:man1:`su` running on heavily loaded machines.
* :gh:issue:`410`: Uses of :linux:man7:`unix` sockets are replaced with
traditional :linux:man7:`pipe` pairs when SELinux is detected, to work around
a broken heuristic in common SELinux policies that prevents inheriting
:linux:man7:`unix` sockets across privilege domains.
* :gh:issue:`467`: an incompatibility running Mitogen under `Molecule
<https://ansible.readthedocs.io/projects/molecule/>`_ was resolved.
* :gh:issue:`547`, :gh:issue:`598`: fix a deadlock during initialization of
connections, ``async`` tasks, tasks using custom :mod:`module_utils`,
``mitogen_task_isolation: fork`` modules, and modules present on an internal
blacklist. This would manifest as a timeout or hang, was easily hit, had been
present since 0.2.0, and likely impacted many users.
* :gh:issue:`549`: the open file limit is increased to the permitted hard
limit. It is common for distributions to ship with a higher hard limit than
the default soft limit, allowing *"too many open files"* errors to be avoided
more often in large runs without user intervention.
* :gh:issue:`558`, :gh:issue:`582`: on Ansible 2.3 a directory was
unconditionally deleted after the first module belonging to an action plug-in
had executed, causing the :ans:mod:`unarchive` to fail.
* :gh:issue:`578`: the extension could crash while rendering an error due to an
incorrect format string.
* :gh:issue:`590`: the importer can handle modules that replace themselves in
:data:`sys.modules` with completely unrelated modules during import, as in
the case of Ansible 2.8 :mod:`ansible.module_utils.distro`.
* :gh:issue:`591`: the working directory is reset between tasks to ensure
:func:`os.getcwd` cannot fail, in the same way :class:`AnsibleModule`
resets it during initialization. However this restore happens before the
module executes, ensuring code that calls :func:`os.getcwd` prior to
:class:`AnsibleModule` initialization, such as the Ansible 2.7
:ans:mod:`pip`, cannot fail due to the actions of a prior task.
* :gh:issue:`593`: the SSH connection method exposes
``mitogen_ssh_keepalive_interval`` and ``mitogen_ssh_keepalive_count``
variables, and the default timeout for an SSH server has been increased from
`15*3` seconds to `30*10` seconds.
* :gh:issue:`600`: functionality to reflect changes to ``/etc/environment`` did
not account for Unicode file contents. The file may now use any single byte
encoding.
* :gh:issue:`602`: connection configuration is more accurately inferred for
:ans:mod:`meta: reset_connection <meta>`, the :ans:mod:`synchronize`, and for
any action plug-ins that establish additional connections.
* :gh:issue:`598`, :gh:issue:`605`: fix a deadlock managing a shared counter
used for load balancing, present since 0.2.4.
* :gh:issue:`615`: streaming is implemented for the :ans:mod:`fetch` and other
actions that transfer files from targets to the controller. Previously files
were sent in one message, requiring them to fit in RAM and be smaller than an
internal message size sanity check. Transfers from controller to targets have
been streaming since 0.2.0.
* :gh:commit:`7ae926b3`: the :ans:mod:`lineinfile` leaked writable temporary
file descriptors between Ansible 2.7.0 and 2.8.2. When :ans:mod:`~lineinfile`
created or modified a script, and that script was later executed, the
execution could fail with "*text file busy*". Temporary descriptors are now
tracked and cleaned up on exit for all modules.
Core Library
~~~~~~~~~~~~
* Log readability is improving and many :func:`repr` strings are more
descriptive. The old pseudo-function-call format is migrating to
readable output where possible. For example, *"Stream(ssh:123).connect()"*
might be written *"connecting to ssh:123"*.
* In preparation for reducing default log output, many messages are delivered
to per-component loggers, including messages originating from children,
enabling :mod:`logging` aggregation to function as designed. An importer
message like::
12:00:00 D mitogen.ctx.remotehost mitogen: loading module "foo"
Might instead be logged to the ``mitogen.importer.[remotehost]`` logger::
12:00:00 D mitogen.importer.[remotehost] loading module "foo"
Allowing a filter or handler for ``mitogen.importer`` to select that logger
in every process. This introduces a small risk of leaking memory in
long-lived programs, as logger objects are internally persistent.
* :func:`bytearray` was removed from the list of supported serialization types.
It was never portable between Python versions, unused, and never made much
sense to support.
* :gh:issue:`170`: to improve subprocess
management and asynchronous connect, a :class:`~mitogen.parent.TimerList`
interface is available, accessible as :attr:`Broker.timers` in an
asynchronous context.
* :gh:issue:`419`: the internal
:class:`~mitogen.core.Stream` has been refactored into many new classes,
modularizing protocol behaviour, output buffering, line-oriented input
parsing, option handling and connection management. Connection setup is
internally asynchronous, laying most groundwork for fully asynchronous
connect, proxied Ansible become plug-ins, and in-process SSH.
* :gh:issue:`169`,
:gh:issue:`419`: zombie subprocess reaping
has vastly improved, by using timers to efficiently poll for a child to exit,
and delaying shutdown while any subprocess remains. Polling avoids
process-global configuration such as a `SIGCHLD` handler, or
:func:`signal.set_wakeup_fd` available in modern Python.
* :gh:issue:`256`, :gh:issue:`419`: most :func:`os.dup` use was eliminated,
along with most manual file descriptor management. Descriptors are trapped in
:func:`os.fdopen` objects at creation, ensuring a leaked object will close
itself, and ensuring every descriptor is fused to a `closed` flag, preventing
historical bugs where a double close could destroy unrelated descriptors.
* :gh:issue:`533`: routing accounts for
a race between a parent (or cousin) sending a message to a child via an
intermediary, where the child had recently disconnected, and
:data:`~mitogen.core.DEL_ROUTE` propagating from the intermediary
to the sender, informing it that the child no longer exists. This condition
is detected at the intermediary and a :ref:`dead message <IS_DEAD>` is
returned to the sender.
Previously since the intermediary had already removed its route for the
child, the *route messages upwards* rule would be triggered, causing the
message (with a privileged :ref:`src_id/auth_id <stream-protocol>`) to be
sent upstream, resulting in a ``bad auth_id`` error logged at the first
upstream parent, and a possible hang due to a request message being dropped.
* :gh:issue:`586`: fix import of
:mod:`__main__` on later versions of Python 3 when running from the
interactive console.
* :gh:issue:`606`: fix example code on the
documentation front page.
* :gh:issue:`612`: fix various errors
introduced by stream refactoring.
* :gh:issue:`615`: when routing fails to
deliver a message for some reason other than the sender cannot or should not
reach the recipient, and no reply-to address is present on the message,
instead send a :ref:`dead message <IS_DEAD>` to the original recipient. This
ensures a descriptive message is delivered to a thread sleeping on the reply
to a function call, where the reply might be dropped due to exceeding the
maximum configured message size.
* :gh:issue:`624`: the number of threads used for a child's automatically
initialized service thread pool has been reduced from 16 to 2. This may drop
to 1 in future, and become configurable via a :class:`Router` option.
* :gh:commit:`a5536c35`: avoid quadratic
buffer management when logging lines received from a child's redirected
standard IO.
* :gh:commit:`49a6446a`: the
:meth:`empty` methods of :class:`~mitogen.core.Latch`,
:class:`~mitogen.core.Receiver` and :class:`~mitogen.select.Select` are
obsoleted by a more general :meth:`size` method. :meth:`empty` will be
removed in 0.3
* :gh:commit:`ecc570cb`: previously
:meth:`mitogen.select.Select.add` would enqueue one wake event when adding an
existing receiver, latch or subselect that contained multiple buffered items,
causing :meth:`get` calls to block or fail even though data existed to return.
* :gh:commit:`5924af15`: *[security]*
unidirectional routing, where contexts may optionally only communicate with
parents and never siblings (so that air-gapped networks cannot be
unintentionally bridged) was not inherited when a child was initiated
directly from another child. This did not effect Ansible, since the
controller initiates any new child used for routing, only forked tasks are
initiated by children.
Thanks!
~~~~~~~
Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Andreas Hubert <https://github.com/peshay>`_,
`Anton Markelov <https://github.com/strangeman>`_,
`Dan <https://github.com/dsgnr>`_,
`Dave Cottlehuber <https://github.com/dch>`_,
`Denis Krienbühl <https://github.com/href>`_,
`El Mehdi CHAOUKI <https://github.com/elmchaouki>`_,
`Florent Dutheil <https://github.com/fdutheil>`_,
`James Hogarth <https://github.com/hogarthj>`_,
`Jordan Webb <https://github.com/jordemort>`_,
`Julian Andres Klode <https://github.com/julian-klode>`_,
`Marc Hartmayer <https://github.com/marc1006>`_,
`Nigel Metheringham <https://github.com/nigelm>`_,
`Orion Poplawski <https://github.com/opoplawski>`_,
`Pieter Voet <https://github.com/pietervoet/>`_,
`Stefane Fermigier <https://github.com/sfermigier>`_,
`Szabó Dániel Ernő <https://github.com/r3ap3rpy>`_,
`Ulrich Schreiner <https://github.com/ulrichSchreiner>`_,
`Vincent S. Cojot <https://github.com/ElCoyote27>`_,
`yen <https://github.com/antigenius0910>`_,
`Yuki Nishida <https://github.com/yuki-nishida-exa>`_,
`alexhexabeam <https://github.com/alexhexabeam>`_,
`DavidVentura <https://github.com/DavidVentura>`_,
`dbiegunski <https://github.com/dbiegunski>`_,
`ghp-rr <https://github.com/ghp-rr>`_,
`migalsp <https://github.com/migalsp>`_,
`rizzly <https://github.com/rizzly>`_,
`SQGE <https://github.com/SQGE>`_, and
`tho86 <https://github.com/tho86>`_.
.. rubric:: Footnotes
.. [i615] Peak RSS of controller and target as measured with ``/usr/bin/time
-v ansible-playbook -c local`` using the reproduction supplied in
:gh:issue:`615`.