Libnvme

The highlights of this release are

- allocated payloads aligned (Tomas Bzatek)
- refactoring and improvements of RAE handling in get log pages (Caleb Sander)
- auto cleanup of resources (Caleb Sander)
- build instruction update and improving build situation for older distros
- support for TP8018 (Hannes Reinecke)
- use sysfs only for topology scan (Daniel Wagner)

The last point addresses a long standing problem. Whenever 'nvme list' is used it could add error log entries for certain devices. With the corresponding kernel changes this will not happen anymore. It also unlocks some possible new features. Note this depends on the upcoming 6.8 kernel.

Obviously there were lots of bug fixes everywhere and contributions in form of bug reports etc. Many thanks to everyone who contributed to the project: Alfred Wingate, Brandon Paupore, Caleb Sander, Chris Patterson, Daniel Wagner, Hannes Reinecke, Joy Gu, Martin Belanger, Martin George, Maurizio Lombardi, Nate Thornton, Prashanth Nayak, Sam James, Tokunori Ikegami and Tomas Bzatek.

ALPSS 2023 Release

A couple of noteworthy features made it to this release:
- A parser for the NBFT table (per NVMe-oF boot specification v1.0).
The [Timberland SIG](https://github.com/timberland-sig) provided the code for this.
- Ignoring fabric connect attempts if the execution context doesn't match.
This feature allows to make different users of libnvme to coexist, e.g. nvme-cli, nvme-stas or blktests without trying
to manage the same resource

And as usual there are plenty of fixes all over the place.

A big thanks to contributions from Benjamin Drung, Caleb Sander, Hannes Reinecke, Jeremy Kerr, Lior Weintraub, Martin Belanger, Maurizio Lombardi, Steven Seungcheol Lee, Stuart Hayes, Tokunori Ikegami, Tomas Bzatek, Umer Saleem.



Benjamin Drung (1):
examples: Fix wrong indentation in discover-loop.py

Caleb Sander (3):
fabrics: handle /dev/nvme-fabrics read failure
fabrics: check genctr after getting discovery entries
ioctl: fix RAE bit on last Get Log Page command

Daniel Wagner (27):
tree: Fix argument check in nvme_bytes_to_lba
fabrics: Do not pass unsupported options to kernel
tree: Fix offset argument check in nvme_bytes_to_lba
build: Disable fallback on default
build: Update wrap mode defaults
build: Make json-c dependency lookup not fail
build: Extend summary section
build: Simple muon build configuration
doc: Update README
python: Update test data
ioctl: Explicitly initialize all members of struct nvme_ns_mgmt_args
build: Install missing s390 library
test: Add unit test for ctrl lookups
util: Add ignored error code
fabrics: Filter discovery ctrls out during application context check
tree: Ignore NULL address pointer for phy slot lookup
fabrics: Relax match on well known disc ctrl lookup
scripts: Move helper scripts to a central place
scripts: Call update doc script from top level dir
scripts: Sync release script
build: Move CI build steps into a scripts
scripts: Use spaces instead of tabs
util: Provide empty nvme_ipaddrs_eq for static builds
build: Use containers with matrix build
build: Update cross instruction and drop verbose test flag
build: Update version to v1.5
doc: Regenerate all docs for v1.5

Hannes Reinecke (3):
libnvme: add 'application' setting to the subsystem
libnvme: add 'application' setting to nvme_root
fabrics: filter out subsystems with non-matching application string

Jeremy Kerr (1):
mi: Add nvme_mi_ctrl_id to retrieve controller ID

Lior Weintraub (1):
examples: fix incorrect controller status in MI info output

Martin Belanger (12):
nbft: Doc typo - Use nvme_nbft_free() instead of nbft_free()
Python: Add NBFT support
NBFT: Remove documentation from nbft.c since it's also in nbft.h
Python: make NBFT data more pythonic
Python: Suppress swig warnings about unnamed struct
python/swig: Wrap swig-sensitive struct inside ifwdef SWIG
python/swig: Check swig version to determine whether -py3 is needed
Python: Fix crash during garbage collection
util: Add ipaddrs_eq() to check whether two IP addresses are equal
util: rename ipaddrs_eq() to nvme_ipaddrs_eq() and make public.
test: Add more code coverage for nvme_ipaddrs_eq()
tree: Use nvme_ipaddrs_eq() to compare IP addresses

Maurizio Lombardi (1):
fabrics: fix potential invalid memory access in __nvmf_supported_option()

Steven Seungcheol Lee (4):
types.h: Add FDP related Status Code
types: Add ns-mgmt host software specified fields
types: Add IO command set specific field on nsmgmt
ioctl: io management send, receive args fix

Stuart Hayes (1):
nbft: add NBFT v1.0 table support

Tokunori Ikegami (3):
fabrics: Add EADDRNOTAVAIL error mapping
util: Use HAVE_NETDB instead of HAVE_LIBNSS
json: Use memory block allocated by realloc() instead printbuf

Tomas Bzatek (4):
nbft: Move added symbols to LIBNVME_1_5
nbft: Add a simple unit test
nbft: Fix nbft_ssns_flags endianness test
nbft: Parse the {HOSTID,HOSTNQN}_CONFIGURED flags

Umer Saleem (1):
tree: Add PCI physical slot number for controller

This release doesn't bring a lot of new exciting features. Hannes provided the initial TLS support for the TCP transport. This is really fresh code an might actually change. There kernel part is not there yet, consider this all as technology preview.

On the bug fixing front we have a bunch of smaller contributions. Thanks Tomas, Steven, Keith and Minwoo!

And a bigger one for the Python binding from Martin, thanks too!

The rest of the changes are mostly build related stuff. Eg I made all the non necessary dependency optional (which reduces also the functionally).

Note:
- The Python configuration flag type changed to feature, that means it accepts 'auto', 'disabled' and 'enabled.
- New optional dependency introduced (TLS feature) on libkeyutils (https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/keyutils.git/)


Daniel Wagner (28):
Merge pull request 570 from tbzatek/nss
build: Use prefixdir directly on sysconfdir
build: Make json-c dependency optional again
build: Update dbus.wrap
Merge pull request 575 from igaw/update-dbus-wrap
fabrics: Do not ignore one char whitespace when string sanitizing
Merge pull request 576 from igaw/fix-strchomp-callers
Merge pull request 573 from igaw/fix-muon-build
Merge pull request 574 from igaw/make-json-c-optional
build: Set defaults for libdbus to disabled
build: Rename meson.yml to build.yml
build: Add muon build
Merge pull request 577 from igaw/build-updates
ioctl: Set file descriptor in nvme_get_log_page()
mi: Add nvme_mi_admin_get_log_page
build: Streamline workflows with nvme-cli
build: Enable test in muon minimal build
build: Initialize sysconfdir
build: Upload all log files on failure
build: Add build action back
build: Update json-c wrap
build: Update OpenSSL wrap
doc: Update build script to use correct meson verbs
doc: Do not hardcode default location for config files
fabrics: Add define for DMI sysinfo
fabrics: Filter out invalid UUIDs from DMI
build: Update version to v1.4
doc: Regenerate all docs for v1.4

Hannes Reinecke (6):
linux: add nvme_lookup_keyring()
nvme: add 'nvme_insert_tls_key()' function
linux: add key helper functions
fabrics: add configuration option 'keyring'
fabrics: add configuration option 'tls_key'
Print out correct TREQ strings for discovery

Keith Busch (1):
fix endians

Martin Belanger (11):
python: Fix segmentation fault during garbage collection
python: Return PyObject* from __str__() instead of a static char[]
python test: Check there is no SIGSEGV during garbage collection
python: Reformat SWIG input file
python, meson: Assert that deps are present for -Dpython=true
Python: Add hostnqn_from_file() and hostid_from_file()
Python: Add support for setting/getting Host DHCHAP key
build, meson: Print option summary
python: Remove redundant host.set_key() method
python: Add setter property for controller DHCHAP Key
python: Change python option from combo to feature

Minwoo Im (2):
types: add opcode field to error log page (TP4113)
tree: fix generic device open failure

Steven Seungcheol Lee (1):
doc: fix kernel-doc-check warning

Tomas Bzatek (1):
build: Fix nss code snippet includes

zerone (1):
types: Update position of elbatm in struct nvme_copy_range{_f1}

I am glad to announce the v1.3 release of libnvme. As usual it consist out of a few new features and a lot of bug fixes all over the place. Thanks everyone who is contributing to this project (code, bug reports, testing)!

- Support added for Flexible Data Placement (TP4146)
- Fabrics: added support for Persistent Discovery Controllers.
- libnvme-mi added a quirk framework to workaround special behavior of certain devices
- The python binding behaves more friendly in a threading environment by release the GIL on blocking calls.
- Replaced the libsystemd dependency with libdbus
- A lot of small fixes everywhere (leaks, wrong ordering of arguments, ...)


Benjamin Drung (1):
doc: Fix spelling mistakes found by lintian

Daniel Wagner (27):
build: Remove usused pkg-config template
ioctl: Move nvme_get_log_page() to ioctl.h
ioctl: Use nvme_get_log_page() in helpers
doc: Generate rst documentation on the fly for readthedocs
build: Drop meson version test for 0.48
build: Allow static linking
build: Always set CONFIG_LIBSYSTEMD variable
build: Run cross build test under qemu
build: Do not install libuuid in cross builds
build: Add s390x cross build target
build: Drop --werror for fallback builds
build: Remove unused uuid.wrap
build: Add D-Bus fallback
build: Add libdbus to build default build environment
build: Set projects default to debug and prefix /usr/local
ioctl: Initialize all members in nvme_fdp_reclaim_unit-handle_status
util: Add ECONNECTREFUSED to enum nvme_connect_err
fabrics: Change string for NVMF_DISC_EFLAGS_NONE
fabrics: Add nqn to connect/disconnect log entry
tree: Add unique discovery controller flag
ioctl: Initialize non-trivial designated initializers
gitignore: Ignore clang build artifacts
fabrics: Always sanitize discovery log entries
tree: Add persistent_discovery_ctrl config flag
fabrics: Avoid buffer overrun in strchomp
build: Update version to v1.3
doc: Regenerate all docs for v1.3

Hao Jiang (2):
mi: Allow zero as a controller ID in nvme_mi_scan_ep
mi: add timeout functions to .map

Jeremy Kerr (14):
mi: Issue a NSID-specific Controller List Identify in nsid_ctrl_list()
mi: Introduce quirks and endpoint probing
mi: Allow probe suppression via an environment variable
mi: Add inter-command-delay quirk
mi: Add model detection via identify in nvme_mi_ep_probe()
mi: Print applied quirks
doc: Add documentation for MI probe behaviour
mi: Add inter-command-delay quirk for Samsung MZUL2512 devices
mi: Fix status extraction from cdw3 field
mi: endian fixes
mi: call nvme_mi_close on endpoint error path
MI: switch from libsdbus to libdbus
test: mi: Explicitly zero args for test API invocations
test: mi: Provide a full request buffer for invalid admin req tests

Jinliang Wang (6):
mi: add nvme_mi_admin_admin_passthru
mi: fix rc checking

Jeremy Kerr was busy adding new MI commands and fixing bugs in this area. Since we have also unit tests for all these commands it all looks in excellent shape. Thanks a lot Jeremy!

There are also a lot of small bug fixes all over the place from various contributors; Hannes, Hao, Jinliang, Martin Belanger, Martin George, Maurizio, Simon and Tokunori. Thanks for you work too!

One 'feature' which I'd like to mention, is that we dropped the dependency on libuuid. This is one step forward towards being able to build the library complete static. There are still some small fallouts but I am sure the next release will have the necessary fixes.

Anyway, happy packaging!


Daniel Wagner (20):
workflow: Only publish proper releases to PyPI
build: Support release candidate version strings
build: Update release version rules
build: Update version before regenerating docs
util: Add LINE_MAX define
build: Extend release script to support micro version releases
doc: Install pre-compiled man pages to the correct path
docs: Relax regex in identifier match in list-man-pages.sh
json: Enforce correctly formatted JSON config files
json: Verify JSON config file starts with an array
fabrics: Use fallthrough statement
util: Do not expose fallthrough defines
util: Add simple UUID type
ioctl: Set log page offset for nvme_get_log_telemetry_host
mi: Set log page offset for nvme_get_log_telemetry_host
build: Update min version of meson to 0.48
fabrics: Duplicate strings when merging configs
types: Move enum nvme_data_tfr to types
build: Update version to v1.2
doc: Regenerate all docs for v1.2

Hannes Reinecke (4):
libnvme: accessors for dhchap_key variables
tree: rename controller 'dhchap_key' to 'dhchap_ctrl_key'
Parse dhchap_host_key on controller level
Update json config schema for missing dhchap host key

Hao Jiang (3):
mi: Fix C++ compiler errors
Add more details for return code of MI admin cmds
mi: Init ctrl_id within xfer

Jeremy Kerr (23):
mi: Implement Get & Set Features Admin commands
mi: Add identify helpers for namespace lists
mi: Add identify helpers for namespaces
mi: Add identify helper for nsid-capable Controller List
mi: Implement Namespace Management command and create/delete helpers
mi: Implement NS attach command and helpers
mi: Implement Format NVM command
mi: Implement Sanitize command
mi: Add Get Log Page helpers
example/mi-conf: Add an example for MI transport configuration
mi: Allow Admin-message sized More Processing Required responses
examples/mi-mctp: fix warning for possibly uninitialised variable
github workflows: Add an armhf cross build
github workflows: Add a powerpc64le cross build
mi: Add Identify function for secondary controller list
mi: Introduce a helper for response status, unify values with ioctls
mi: Move post-1.1 symbols to 1_2 section of libnvme-mi.map
mi: Add firmware download and commit commands
mi: fix get_log_page chunked offset check
mi: Distinguish MI status from NVMe (CDW3) status
mi: add nvme_mi_status_to_string()
examples/mi-conf: Initialise sd_bus_error
examples/mi-conf: account for MCTP header in route MTU

Jinliang Wang (1):
mi: Add Identify helper for ns-descs and primary-ctrl-caps

Martin Belanger (6):
examples: Add missing free()
fabrics: Add new TP8010 definitions
fabrics: Add nvmf_get_discovery_wargs()
python: Use nvmf_get_discovery_wargs()
fabrics: Filter out empty strings in add_argument()
examples: LID supported must be shifted right by 16

Martin George (3):
nvme-tree: avoid segfault if auth keys are unavailable
json-schema: add dhchap_key details to host section
libnvme.map: add nvme_ctrl_is_persistent()

Maurizio Lombardi (1):
mi: fix a memory leak in nvme_mi_open_mctp()

Simon A. F. Lund (1):
Makefile: add 'make uninstall'

Tokunori Ikegami (1):
util: Add get feature length 2 API to support direction parameter

dependabot[bot] (1):
build(deps): bump actions/checkout from 2 to 3