Announcement
https://discuss.linuxcontainers.org/t/lxd-4-21-has-been-released/12860
What's Changed
* DB: Support go-dqlite v1.10.1 NULLable fields using sql.NullTime by tomponline in https://github.com/lxc/lxd/pull/9485
* Cluster: Fix self-removal of non-leader member from cluster by tomponline in https://github.com/lxc/lxd/pull/9483
* API: Prevent update of cluster.https_address in doApi10Update by tomponline in https://github.com/lxc/lxd/pull/9487
* Operations: Don't duplicate local operations in operationsGet when running in single member cluster by tomponline in https://github.com/lxc/lxd/pull/9486
* Cluster: Join leader address validation improvements by tomponline in https://github.com/lxc/lxd/pull/9488
* Doc: fix typo for hardware offload by hnakamur in https://github.com/lxc/lxd/pull/9490
* Doc: fix remove non-existent anchor in link to network-peers.md by hnakamur in https://github.com/lxc/lxd/pull/9491
* lxd/instances: Use correct project in live migration by stgraber in https://github.com/lxc/lxd/pull/9502
* DB: Fix upgrade from LXD 2.0/3.0 when using go-dqlite v1.10.1 NULLable fields by tomponline in https://github.com/lxc/lxd/pull/9500
* lxd/device: Clean up mdev vGPU on failure by monstermunchkin in https://github.com/lxc/lxd/pull/9501
* Network: Add support for OVN PTMU discovery for external ingress traffic by tomponline in https://github.com/lxc/lxd/pull/9503
* VM: fix usb pass-through with more than one device by zyga in https://github.com/lxc/lxd/pull/9497
* Switch to original go-udev by stgraber in https://github.com/lxc/lxd/pull/9505
* doc: Elaborate on pongo2 date-time syntax for snapshots by RKBethke in https://github.com/lxc/lxd/pull/9493
* lxc/list: Changed units to be IEC consistent by RKBethke in https://github.com/lxc/lxd/pull/9494
* Switch to go-httprequest by stgraber in https://github.com/lxc/lxd/pull/9507
* DB: Always defer close until after statement has returned to avoid panicking on error by tomponline in https://github.com/lxc/lxd/pull/9514
* Default profile units by simondeziel in https://github.com/lxc/lxd/pull/9513
* Migrate: Only use pointers to migration.MigrationControl to avoid shallow copies by tomponline in https://github.com/lxc/lxd/pull/9512
* Storage: Disk mounting cleanup and improvements by tomponline in https://github.com/lxc/lxd/pull/9510
* Stricter regexps by simondeziel in https://github.com/lxc/lxd/pull/9517
* Container: Fix liblxc handle leak in renderState by tomponline in https://github.com/lxc/lxd/pull/9520
* Device: Switch VM disk devices to use opened file descriptors for passing disk sources to QEMU by tomponline in https://github.com/lxc/lxd/pull/9521
* Warning entry for missing instance driver by presztak in https://github.com/lxc/lxd/pull/9522
* lxd/network/ovn: Support SSL by stgraber in https://github.com/lxc/lxd/pull/9524
* Network: Separate IP neighbour and neighbour proxy management functions by tomponline in https://github.com/lxc/lxd/pull/9527
* Support trust certificate self renewal by monstermunchkin in https://github.com/lxc/lxd/pull/9474
* NIC: Reworks routed NIC to not depend on liblxc's router NIC type by tomponline in https://github.com/lxc/lxd/pull/9528
* NIC: Adds routed NIC support for VMs by tomponline in https://github.com/lxc/lxd/pull/9529
* doc: Fix misspelling in server.md by cavcrosby in https://github.com/lxc/lxd/pull/9530
* Device: Switch to using string slice for mount option handling by tomponline in https://github.com/lxc/lxd/pull/9532
* Project: Adds restricted.devices.disk.paths setting by tomponline in https://github.com/lxc/lxd/pull/9496
* Support for moving instances and custom volumes between projects by presztak in https://github.com/lxc/lxd/pull/9509
* tests: Split cluster and standalone by stgraber in https://github.com/lxc/lxd/pull/9537
* lxd: add forkusernsexec() by brauner in https://github.com/lxc/lxd/pull/9536
* Project: Separate checking if a project can support snapshots from the project DB lookup by tomponline in https://github.com/lxc/lxd/pull/9539
* VM: Use socket file descriptors for virtfs-proxy-helper and virtiofsd by tomponline in https://github.com/lxc/lxd/pull/9538
* VM: Close unnecessary file handles for VM disk proxy processes by tomponline in https://github.com/lxc/lxd/pull/9540
* forkusernsexec: close file descriptors before exec by brauner in https://github.com/lxc/lxd/pull/9541
* forkusernsexec: fixes by brauner in https://github.com/lxc/lxd/pull/9550
* util_linux: ensure that O_NOCTTY is raised when opening terminals by brauner in https://github.com/lxc/lxd/pull/9544
* Move to cron/v3 by stgraber in https://github.com/lxc/lxd/pull/9551
* Project: Validation cleanup by tomponline in https://github.com/lxc/lxd/pull/9549
* Support for lxc storage volume {copy,move} --target and --destination-target flag by RKBethke in https://github.com/lxc/lxd/pull/9534
* Allow passing context to DownloadFileHash by monstermunchkin in https://github.com/lxc/lxd/pull/9559
* VM: Add raw.idmap support for running disk device proxy processes in user namespace by tomponline in https://github.com/lxc/lxd/pull/9543
* Simple doc fixes by ru-fu in https://github.com/lxc/lxd/pull/9562
* VM: Move readonly config mount out of config directory and into devices directory by tomponline in https://github.com/lxc/lxd/pull/9565
* Network: Ensures OVN networks only consider only compatible networks for use as uplink by tomponline in https://github.com/lxc/lxd/pull/9567
* Instance: Improve and unify stop and shutdown error handling by tomponline in https://github.com/lxc/lxd/pull/9563
* Instance: Reset operation lock timeout when unmounting instance and increase ZFS unmount wait by tomponline in https://github.com/lxc/lxd/pull/9564
* First class cloud-init support by monstermunchkin in https://github.com/lxc/lxd/pull/9515
* forkusernsexec: use unix.Stderr as standard --keep-fd-up-to value by brauner in https://github.com/lxc/lxd/pull/9569
* Network: OVN instance peer route bug fix by tomponline in https://github.com/lxc/lxd/pull/9570
* doc: Initial sphinx build by stgraber in https://github.com/lxc/lxd/pull/9568
* lxd/device/disk: Support snap packages when opening restricted disk paths by tomponline in https://github.com/lxc/lxd/pull/9571
* Generator: Accept sql.NullTime as a valid type in the generator by masnax in https://github.com/lxc/lxd/pull/9572
* doc: Fix typo in ToC entry by stgraber in https://github.com/lxc/lxd/pull/9573
* NIC: Routed test tweaks by tomponline in https://github.com/lxc/lxd/pull/9574
* shared/termios: manually copy termios settings between Go and C by brauner in https://github.com/lxc/lxd/pull/9581
* shared/termios: actually copy the values not the index by brauner in https://github.com/lxc/lxd/pull/9582
* Shared: Revert recent termios fixes by tomponline in https://github.com/lxc/lxd/pull/9586
* Shared: Remove requirement on cgo when setting raw mode by tomponline in https://github.com/lxc/lxd/pull/9589
* doc: add a cheat sheet for documentation by ru-fu in https://github.com/lxc/lxd/pull/9585
* Instance: Use exitErr.ExitCode function for containers by tomponline in https://github.com/lxc/lxd/pull/9598
* Increases recommended subuid and subgid ranges in machine setup. by markylaing in https://github.com/lxc/lxd/pull/9601
* lxc/exec: Always connect control websocket by tomponline in https://github.com/lxc/lxd/pull/9595
* Instance: Add more exec tests by tomponline in https://github.com/lxc/lxd/pull/9591
* Instance: Detect disconnect of non-interactive exec stdout websocket and kill command by tomponline in https://github.com/lxc/lxd/pull/9594
* Instance: Rework VM exec to always expect control connection for websocket sessions by tomponline in https://github.com/lxc/lxd/pull/9597
* doc: clean up the content on the index page of the documentation by ru-fu in https://github.com/lxc/lxd/pull/9587
* doc: css: temporary fixes for theme issues by ru-fu in https://github.com/lxc/lxd/pull/9603
* Instance: Fix VM exec pts leak by tomponline in https://github.com/lxc/lxd/pull/9599
* Instance: Adds ExitStatus() to properly extract exit status when a command is signaled from interactive session by tomponline in https://github.com/lxc/lxd/pull/9602
* Replace bionic with focal by simondeziel in https://github.com/lxc/lxd/pull/9605
* doc: add redirect to index.html by ru-fu in https://github.com/lxc/lxd/pull/9606
* Shared: Some minor tweaks by tomponline in https://github.com/lxc/lxd/pull/9607
* doc: Update Ceph docs with details for erasure coded pools by AlexandarY in https://github.com/lxc/lxd/pull/9604
* Instance: Adds non-websocket exec tests by tomponline in https://github.com/lxc/lxd/pull/9613
* lxd: log basic idmapped mount support by brauner in https://github.com/lxc/lxd/pull/9612
* lxd/device: Improves error message for custom storage volumes when shift is true. by markylaing in https://github.com/lxc/lxd/pull/9611
* lxd/device: Fix golint by stgraber in https://github.com/lxc/lxd/pull/9615
* Generator: Refactor handling of reference tables. by masnax in https://github.com/lxc/lxd/pull/9364
* Add `network.nat` property to zones by stgraber in https://github.com/lxc/lxd/pull/9616
* Instance: Make VM WaitForWS for lxd-agent exec explicit by tomponline in https://github.com/lxc/lxd/pull/9619
* Instance: Clear up OVS integration bridge port earlier in stop sequence for OVN NICs by tomponline in https://github.com/lxc/lxd/pull/9621
* Added link for installing.md by dillonsamra9 in https://github.com/lxc/lxd/pull/9624
* Add cluster member role to indicate the leader by presztak in https://github.com/lxc/lxd/pull/9609
* Storage: Enable nodatacow on BTRFS subvolume for block volumes by tomponline in https://github.com/lxc/lxd/pull/9627
* Storage: Improve logging during backup unpack by tomponline in https://github.com/lxc/lxd/pull/9629
* Storage: Add warning to docs about using VMs on BTRFS storage pools by tomponline in https://github.com/lxc/lxd/pull/9635
* doc/storage: fix 2 typos by simondeziel in https://github.com/lxc/lxd/pull/9637
* Added --sub-command flag by dillonsamra9 in https://github.com/lxc/lxd/pull/9634
* Events: Switch events websocket heartbeat to counter rather than using absolute deadline times by tomponline in https://github.com/lxc/lxd/pull/9632
* Storage: Restore VM filesystem size.state quota on backup restore by tomponline in https://github.com/lxc/lxd/pull/9628
* Project: Account for disk size.state when checking project limits.disk setting by tomponline in https://github.com/lxc/lxd/pull/9631
* lxd/storage: Switch to deviceConfig.DefaultVMBlockFilesystemSize by monstermunchkin in https://github.com/lxc/lxd/pull/9638
* Add swagger to doc by ru-fu in https://github.com/lxc/lxd/pull/9636
* doc: Clarify btrfs resize documentation by archevan in https://github.com/lxc/lxd/pull/9640
* Ability to retrieve instances for all projects through API by presztak in https://github.com/lxc/lxd/pull/9590
* Optimise proxy device driver nat rules by markylaing in https://github.com/lxc/lxd/pull/9633
* lxc/copy: Replaces profiles when -p is set. by markylaing in https://github.com/lxc/lxd/pull/9648
* Cleanup ucred handling, fix cluster permission and more reliable tests by stgraber in https://github.com/lxc/lxd/pull/9644
* doc: fix link to rest-api.yaml by ru-fu in https://github.com/lxc/lxd/pull/9646
* doc/api-extensions.md: fixed typo in gpu_mig section by dillonsamra9 in https://github.com/lxc/lxd/pull/9652
* doc: Reformatted Storage Documentation to Organize Keys by Storage Type by jeff2814 in https://github.com/lxc/lxd/pull/9626
* Cluster server grouping by monstermunchkin in https://github.com/lxc/lxd/pull/9583
* GetNonPendingNetworkIDs Project Aware by C-Stout in https://github.com/lxc/lxd/pull/9653
* lxd/storage: Enable LO_FLAGS_DIRECT_IO by default for loop devices by archevan in https://github.com/lxc/lxd/pull/9643
* Cluster: Include endpoint address in dqlite connect errors by tomponline in https://github.com/lxc/lxd/pull/9655
* test: Give exec chance to finish before capturing output by tomponline in https://github.com/lxc/lxd/pull/9654
* Record net.Conn in request context by markylaing in https://github.com/lxc/lxd/pull/9658
* Cluster: Heartbeat ordering by tomponline in https://github.com/lxc/lxd/pull/9649
* Add ipv{n}.routes setting for routed NIC type by hQavi in https://github.com/lxc/lxd/pull/9645
* Use all-projects parameter in events api by markylaing in https://github.com/lxc/lxd/pull/9657
* lxd/device/nic/bridged: Allow ipv{n} filtering when ip is set to none by hQavi in https://github.com/lxc/lxd/pull/9641
* Cluster: Make event listener refresh event driven by tomponline in https://github.com/lxc/lxd/pull/9656
* lxd/storage: Add a `ceph.rbd.du` storage option to disable the use of `rbd du` by liamnickell in https://github.com/lxc/lxd/pull/9659
* doc/cluster: Clarify join question handling by stgraber in https://github.com/lxc/lxd/pull/9660
New Contributors
* zyga made their first contribution in https://github.com/lxc/lxd/pull/9497
* RKBethke made their first contribution in https://github.com/lxc/lxd/pull/9493
* cavcrosby made their first contribution in https://github.com/lxc/lxd/pull/9530
* markylaing made their first contribution in https://github.com/lxc/lxd/pull/9601
* AlexandarY made their first contribution in https://github.com/lxc/lxd/pull/9604
* dillonsamra9 made their first contribution in https://github.com/lxc/lxd/pull/9624
* archevan made their first contribution in https://github.com/lxc/lxd/pull/9640
* jeff2814 made their first contribution in https://github.com/lxc/lxd/pull/9626
* C-Stout made their first contribution in https://github.com/lxc/lxd/pull/9653
* hQavi made their first contribution in https://github.com/lxc/lxd/pull/9645
* liamnickell made their first contribution in https://github.com/lxc/lxd/pull/9659
**Full Changelog**: https://github.com/lxc/lxd/compare/lxd-4.20...lxd-4.21