Xnvme

------

A handful of improvements for Windows, additional steps toward removing
third-party vendoring / bundling, addition of experimental tunable knobs for
io_uring/SQPOLL, and several improvements to testing and CI infra.

* API
- Removed SLIST from API, although "sys/queue.h" are commonly available on
Linux/FreeBSD, then they are not part of toolchain on Windows.

* Third-party
- Bumped SPDK to v22.09, and with that removed mutliple out-of-tree patches
for DPDK.

* CLI
- The xNVMe command-line library (libxnvmec) and all the cli-tools using it
are refactored to use common sets of command-line arguments. Along with
this came a consistent set of CLI-arguments for admin/sync/async.

* Backends
- ramdisk: The ramdisk got support for write-zeroes, iovec payloads and added
to CI testing.
- linux: support for a buffer-allocator using HUGEPAGES and tunable knobs for
controlling the behavior of io_uring SQPOLL via environment variables.
- windows: support for the experimental IORING Windows SPDK API and support
for block devices (SCSI and SATA).
- spdk: when controllers are re-used for device-handles, events are
processed as a means to check whether the controller is still "alive"

* CI
- scan-build now runs for each PR
- basic tests are now running post-building testing using the RAMDISK
- Testing of fabrics with TCP transport is now part of the setup

------

A bit of expansion in the application of xNVMe with support for macOS, a
ramdisk backend, revival of Python language bindings and a refresh of the docs
on NVMe-over-Fabrics.

* API
- Removed helpers for SGL
- Add 'subnqn' to 'xnvme_ident', useful for NVMe-oF
- Add 'hostnqn' to 'opts', useful for NVMe-oF
- Add support directive-receive and write-with-directives
- Spec adjusted for NVMe 2.0, still more work needed in this area

* Third-party
- Bumped fio to v3.32
- Bumped SPDK to v22.05
- Removed liburing, now relies on on-system library instead of
vendoring/bundling, documentation is updated to assist with library
installation

* Tooling
- Re-working testing using cijoe 0.9+, that is, switching to CIJOE/pytest for
testing an CIJOE workflows for instrumentation
- liburing is no longer bundled with xNVMe, that is, xNVMe now links with
liburing as discovered on the system via pkg-config. This is done to avoid
symbol collisions for applications linking or loading liburing and xNVMe.

* be:linux:async:libaio
- When 'opts.poll_io' is set then poke() will return immmediatly and now wait
for completions. This allows the use of trading CPU for more IOPS and lower
per command latency.

* Additional user-space NVMe driver support via libvfn
- Added 'be:vfio' providing another user-space driver via libvfn

* Preliminary support for macOS
- Initial implementation using the "core" I/O mechanisms of sync-io, async
emulation and the threadpool
- Does enumeration of NVMe devices through the limited interface provided for
user-space by the macOS kernel
- Utilizes what is avaiable for admin-command submision

* Prelimiary support for a "ramdisk" device
- be:ramdisk: added a backend mimicing the behavior of an NVMe NVM namespace
- Intended as a test-vehicle providing a "device" without requiring anything
but the consumption of main memory of the system
- I/O is "stored" using main-memory

* Revival of the xNVMe Python language bindings
- A re-introduction of the Python bindings, these are now generated and thus
provide access to the entire xNVMe C API
- They are added to the testing infrastructure ensuring that they are aligned
with the library
- In addition to simple ctypes bindings, then cython headers and bindings
based on Cython are provided

* Documentation
- Refreshed the NVMe-over-Fabrics tutorial
- Expanded with descriptions on installing liburing
- Expanded with a section for the WIP Python bindings

------

This is a release with the sole purpose of changing the liburing subproject
from tracking 'master' to the next stable release that is liburing-2.2.

------

This main feature of this release is the alignment of the ``io_uring_cmd``
implementation with the ``io_uring`` big-sqe/big-cqe approach to asynchronous
passthru of NVMe commands.

NOTE: the tracking of the liburing repository/subproject is changed from the
fixed tag ``liburing-2.1`` to the ``master`` branch. Thus, in case you
experience liburing related build-issues with this release, then it is most
likely due to changes on ``master``. As soon as ``liburing-2.2`` is released,
xNVMe will be released as well going back to stable tracking. Thus, do not pin
your project to the xNVMe project tag for ``v0.3.0`` if you rely on the
``io_uring`` functionality.

* Asynchronous Passthru of NVMe Commands via ``io_uring``
- There are no API changes to adjust to, the changes are encapsulated inside
the implementation of ``be:linux:async:ucmd`` aka ``async=io_uring_cmd``.
- The previous version of ``io_uring_cmd`` used indirect-commands, that is,
the io_uring-sqe contained a pointer to the NVMe-command. This approach of
passthrough via ``io_uring`` has been superseeded by the
``big-sqe/big-cqe`` approach with the NVMe-sqe embedded within the
io_uring-sqe, and similar for the NVMe-cqe inside the io_uring-cqe.
- This requires changes to how the ``io_uring`` is setup, this task is
delegated to ``liburing`` and the subproject-wrap now tracks liburing
``master`` to do this.

* API
- Fixed ``xnvme_enumerate()`` when ``NULL`` was passed as ``opts``, it now
uses ``xnvme_default_opts()`` when no ``opts`` are given
- Misc. fixes to docstrings missing descriptions

* cmd:
- Fixed missing full-guard on full-guard in xnvme_cmd_passv()

* be:async:{emu,thrpool}: several fixes to command-processing
- Fixed missing setup of completion errors
- Fixed missing empty-guard in cmd_io{v}()

* fio IO engine
- 3p:fio: bumped to v3.30
- tools:fioe: fixed issue with iovec-payloads
- tools:fioe: cleanup and alignment with upstream xNVMe fio IO engine
- docs: removed deprecated information and re-written with usage examples

* tests:io_worker
- Added a basic io_worker to verify the behavior of the
submit-upon-completion

* tools:xdd
- The ``xdd`` tool now provides an ``offset`` argument (in bytes), previously
it started from 0

* Documentation
- Re-introduced the ``tutorial`` section containing a guide to dynamically
load xNVMe from C and Python
- Added a Contributors section containing notes useful for first-time
Contributors

------

Main feature introduction is vectored I/O across a wider set of system
interfaces, that is via ioctl(), io_uring (ucmd) and preadv()/pwritev()
fallback.

* Support for vectored I/O via Linux: ioctl(), psync, and io_uring_cmd

* API
- add xnvme_cmd_passv()
- rename rename xnvme_queue_wait() to xnvme_queue_drain()

* be:io_uring_cmd:
- Enabled NVME_IOCTL_IO64_CMD by default, when available for cmd_io()
- Added support NVME_IOCTL_IO64_CMD_VEC over io_uring via cmd_iov()

* be:linux:nvme:
- Enabled NVME_IOCTL_IO64_CMD by default, when available for cmd_io()
- Added support NVME_IOCTL_IO64_CMD_VEC via cmd_iov()
- Normalized error-handling for NVMe-ioctl interfaces, ioctl() as well as
io_uring_cmd

* be:thrpool:
- Added handling of cmd_iov(), providing a threadpool based fallback when
io_uring_cmd is not available

* be:emu:
- Added handling of cmd_iov(), providing a pseudo-async fallback when
io_uring_cmd is not available

* Re-worked git-pre-commit using the 'pre-commit' framework
- mk: added helpers invoking 'pre-commit', 'make format'/'make format-all'
- mk: removed auto-setup of git-hooks
- git: removed .githooks/pre-commit

* xNVMe fio io-engine
- tools:fioe: use calloc instead of malloc
- tools:fioe: changes according to fio coding conventions

* Library introspection
- fix incorrect generation of third-party information
- replace ``xnvme_3p`` with ``xnvme_libconf``
- add all build-configs to ``xnvme_libconf``

* Command-line argumenter parser
- xnvmec: fix missing setup of --direct

* CLI-fixes
- zoned: fix description for identify namespace command

* Toolbox
- mk: add script generating help-text on Makefile targets
- meson: only do whole-archive in pkg-config when SPDK is enabled
- scripts: replace astyle with clang-format
- pcf: the pre-commit-framework is available for xNVMe

------

Another infrastructure / fixes release.

* 3p:liburing
- Bumped to 2.1
- This breaks old distros: Debian Stretch and CentOS 7 but adds support for
the latest Arch, Fedora, Tumbleweed, and Ubuntu

* docs
- Added scripts and docs for: openSUSE, Fedora, CentOS Stream