Virtme-ng

While the changelog looks extensive, this release is quite small and stable with only a few bug fixes. The big changelog is due to the merging of `virtme-ng-init` into the main `virtme-ng` project (while keeping the `virtme-ng-init` git history intact). This merge allows us to simplify the project structure and eliminate the need to manage a separate git submodule.

As a result, you may need to run the following commands to remove the submodule also from your local Git repository (in case somehting isn't working in your regular workflow):

$ git submodule deinit virtme_ng_init
$ git rm -rf virtme_ng_init


After that, just fetch, pull, or reset to the main virtme-ng branch as usual.

Most noticeable changes in this release include restricting the SSH port to localhost (to mitigate potential security risks when `--ssh` is used), support for running `amd64` vng instances on non-x86 platforms (e.g., running an x86 vng instance on arm64), and a number of minor fixes here and there.

Big thanks to all the contributors!

Detailed ChangeLog:

Andrea Righi (53):
initial commit
virtme-ng-init: initial implementation in Rust
doc: add a proper README.md
create rust.yml
properly handle GUI support
workaround to be able to start xinit directly from a console session
doc: README.md: project merged into virtme-ng
virtme-ng-init: preserve host path
virtme_ng_init: properly synchronize udev and network startup
virtme-ng: allow to run snaps inside virtme-ng instances
virtme-ng: provide --enable-snaps
virtme-ng-init: initialize XDG_RUNTIME_DIR
guest-tools: add a script to automatically start the sound subsystem
virtme-ng: try to automatically fix permissions of snapd state.json
virtme: init process refactoring
virtme-ng: pass --exec command via /proc/cmdline
virtme-ng-init: support 32-bit architectures
virtme-ng-init: use usize instead of u64 in do_mount()
virtme-ng-init: minor coding style fixes
virtme-ng-init: relax 'nix' and 'users' crate dependencies
remove Cargo.lock
update README.md
github: add coding style checks to the workflows
virtme-ng-init: support base64 encoded commands
cargo: use fixed versions for nix and users crate
virtme-ng-init: fail gracefully when executing external commands
virtme-ng-init: support long commands in graphic mode
virtme-ng-init: make virtme-script a constant
virtme-ng: honor virtme_user when running user script
virtme-ng-init: always set XDG_RUNTIME_DIR
virtme-ng-init: set proper ownership of the default console device
virtme-ng-init: set the proper ownership on the virtio-ports devices
virtme-ng-init: fix build error with older versions of rustc
virtme-ng-init: add default secure_path to /etc/sudoers
virtme-ng-init: drop usage of group when chwon'ing
virtme-ng-init: docker host support
virtme-ng-init: channel the return code of a command to the host
virtme-ng-init: allow virtme-ng to specify a console device
virtme-ng-init: always override sudoers
virtme-ng-init: handle command line option nr_open
virtme-ng-init: apply limits after mounting kernel filesystems
virtme-ng-init: hide additional sudo settings
virtme-ng-init: print a hint to exit from virtme-ng
virtme-ng-init: allow to mount legacy cgroupfs (v1)
virtme-ng-init: properly configure terminal line settings
add Cargo.lock
virtme-ng-init: move all temp files to /run/tmp
virtme-ng-init: overlayfs: fall back to mounting without xino option
ssh: support virtme.ssh
virtme-ng-init: always check return code when executing bash
virtme-ng: enable amd64 architecture support on non-x86 platforms
virtme: always print kernel panic / oops in interactive mode
virtme-ng v1.33

Andreas Grapentin (1):
properly quoting exec string to avoid breaking virtme-run invocation on more complex guest calls

David Carlier (1):
virtme_ng_init: replace Err(...) instances for is_err()

Marcos Paulo de Souza (1):
main.rs: Enable lvm usage

Matthieu Baerts (NGI0) (17):
init: set the HOME env var if root
net: support multiple interfaces
net: setup the loopback iface in a thread
poweroff: fix irrefutable 'if let' pattern warning
vsock: socat service for remote console
vsock: mount virtme_vsockmount if needed
ssh: only export the port to localhost
virtme-ng-init: remove the submodule
gh: import and adapt rust workflow
console/ssh: properly quote the remote cmd
vng: run: support O= in interactive mode
vng: run: support O= with full paths
vng: run: support KBUILD_OUTPUT
vng: support --debug with --qemu-opts
vng: support --nvgpu and --qemu-opts
vng: fail if --arch is used without --root
vng: arch: list supported arch in case of errors

Neill Kapron (1):
virtme-ng-init: do not remount /run

Ondrej Mosnáček (2):
virtme_ng_init: use uzers instead of users (which is unmaintained)
virtme_ng_init: update dependencies

Thibault Ferrante (2):
README.md: add building and installation section
virtme-ng-init: allow /tmp to be mounted from host

Uli Schlachter (34):
Use ? on Option where possible
Add a simple test for extract_user_script()
Simplify extract_user_script()
Remove use of HashMap
Remove Option return from run_misc_services()
Simplify an env var check
Simplify(?) some environment lookup
Simplify lookup of $virtme_user
Simplify(?) some command construction
Remove useless Some-wrapping
Remove unnecessary 'static
Replace e.g. libc::S_IRWXU with Mode::S_IRWXU
Fix clippy warnings around logging
Fix clippy warnings about patterns
Fix clippy warning about unnecessary return
Fix clippy warnings about reading lines
Fix clippy warning about "loop never loops"
Fix clippy warning about "push() after creation"
Fix clippy warning about unnecessary matching
Simplify opening of TTYs
Use uname() from nix instead of libc
Remove useless to_string()
Simplify /etc/shadow generation
Use utils::create_file() to write some file contents
Remove an unnecessary Vec
Refactor logic in find_udevd()
Allow run_cmd() with non-utf8-strings
Avoid allocations for static strings for run_shell()
Use nix' flags instead of libc's in one more place
Avoid temporary allocation for PathBuf
Add a log!() macro
(Needlessly) optimise logging
Use write_all() instead of write()
Remove unnecessary trim_end_matches('\n')

Zev Weiss (3):
utils: Don't log mount failures due to ENOENT
Don't try to run systemd-tmpfiles on non-systemd systems
Add guest hostname to /etc/hosts

Happy new year! virtme-ng v1.32 is out!

We're kicking off with an exciting new release with some new features. The most notable being the remote console (vsock), thanks to matttbe, and SSH support.

The remote console (`--console / --console-client`) allows you to open a console session on a running vng instance from another terminal, which can be really useful for inspecting tests/workloads while they're running in the primary console.

The SSH support (`--ssh / --ssh-client`) enables direct ssh/scp access to a running vng instance, making it particularly useful to transfer data, such as large test outputs, from the guest to the host and vice-versa. Since these are all new features, I expect to see some bugs, so don't hesitate to report them.

Additionally, there's a new command `--numa-distance`, that can be used to simulate heterogeneous NUMA nodes with different distance costs.

Enjoy!

Detailed ChangeLog:


Andrea Righi (6):
numa: introduce --numa-distance
virtme-init: Start udev after mounting all filesystems
virtme: enable ssh support
virtme-ng: introduce --console and --ssh shortcuts
sshd: generate a custom sshd_config
virtme-ng v1.32

Marcos Paulo de Souza (1):
virtme-init: Enable lvm usage

Matthieu Baerts (NGI0) (24):
net: support multiple interfaces
udhcpc: add default route if announced
net: allow to specify the bridge iface
net: option to force net MAC address
github: pylint: new supported versions
feat: vsock support for remote console access
vsock: connect: generate helper script
vsock: connect: pass command to execute
vsock: connect: avoid issues with modified script
make: build virtme-ng-init by default
vsock: connect: support commands without tty
vsock: connect: execute given cmd with right user
vsock: connect: only use 'su' if needed
vng: pass '--user root' to virtme-run
vsock: connect: respect '--user' if defined
vsock: connect: respect '--pwd' and '--cwd' if defined
vsock: 'socat' is required
vsock: rename options to --client/--server
console: port: switch to 2222 by default
console: --client and --server are mutually excl
console: client: support --dry-run
README: mention 'socat' in the requirements
github: restrict builds on push to the main branch
sshd: look for overlayfs before modifying auth keys

Michal Suchanek (1):
setup.py: Do not fail when argparse-manpage is not available

Tamir Duberstein (2):
Revert "Merge pull request 184 from hramrach/main"
Restore bash completions

jeromecst (1):
Update README.md

This is a minor release to fix a packaging issue, after an attempt to modernize the build system we realized that we were not installing the bash completion file anymore, so we have temporarily reverted the change to cut this new release.

This is mostly for distro that may have missed the bash completion and the default config file with the previous release (v1.30).

In addition to this packaging fix, there is also the initial support to run virtme-ng on macOS, which is pretty cool (thanks tamird)!

Detailed ChangeLog:

Andrea Righi (2):
setup.py: update contact email
virtme-ng v1.31

Michal Suchanek (1):
Revert "Modernize build system"

Tamir Duberstein (4):
Make it build on macOS
Remove redundant initialization
Remove usage of os.uname
Make qemu work on macOS

The most noticeable changes in this new version are the initial support for NVIDIA GPUs passthrough (thanks to ianm-nv), the possibility to use pre-compiled -rc kernels from Ubuntu mainline builds, the possibility to use virtiofs natively on arm64, some improvements to run virtme-ng cross-architecture and cross-distro and the usual set of bug fixes.

Future plans include the generalization of PCI passthrough support (not only GPUs) and improving the cross-architecture support.

Detailed ChangeLog:

Andrea Righi (13):
vng: allow to use -rc kernels from Ubuntu mainline builds
virtme: fix lint warnings in architectures.py
virtme: fix regex lint warning
virtme: fix "too many arguments" warning in cpiowriter
virtme-ng: fix "too many positional arguments" lint warnings
architectures: GPU passthrough
virtme-ng: fix incorrect cross-compile override
virtme-ng: handle undefined release with --root
virtme-ng: auto-detect host architecture
virtme-ng: allow to specify multiple --rwdir and --rodir
doc: update contact email
readme: add Linux Foundation mentorship session about virtme-ng
virtme-ng v1.30

Andrew Kreimer (2):
readme: fix a typo
virtme: fix typos

Anisse Astier (2):
virtme: fix regex expecting a group
virtme-ng: only match a single downloaded kernel on arm64

Ian May (2):
vng: add support for passthrough NVIDIA GPUs
doc: update README.md with gpu passthrough setup

Jean-Baptiste Denis (1):
Add Kernel Recipies 2024 talk link in the README

Jens Axboe (1):
virtme: enable virtiofs support for arm64

Marcos Paulo de Souza (4):
virtme: commands: run.py: Make get_kernel_version more accurate
virtme: Enable -r argument to work on more architectures
virtme: commands: run.py: Search for kernel version of the file name as fallback
virtme: architectures: Add "image" as s390x image name

Matthieu Baerts (NGI0) (2):
Revert "fix command to install vng_init in pipx"
version: fallback if no git tags

Ricardo B. Marliere (1):
Add hint when trying to create root filesystem outside a Debian host

Tamir Duberstein (1):
Modernize build system

patso (2):
add setuptools to requirements.txt
fix command to install vng_init in pipx

virtme-ng v1.29 has been released just with a minor packaging fix, addressing an issue where the `requirements.txt` file was missing from the source tarball published on PyPI.

This won't affect anyone, unless you're trying to build virtme-ng directly from the tarball available on PyPI.

No major changes in this release, just a few bug fixes that are worth for cutting a new release and a small boot time improvement, so, it's pretty safe to update.

Detailed ChangeLog:


Andrea Righi (4):
setup.py: include dependencies parsing requirements.txt
virtme-ng: fix typo in command help
virtme-configkernel: disable nvram support
virtme-ng v1.28

Jim Cromie (6):
configkernel: config comment cosmetics
configkernel: act more like kernel's make O=outdir
vng-run: get kdir from O=outdir
vng: propagate --verbose to configkernel
vng: alias --custom to --config
run.py: add a --qemu-opts='...' option bundling hint to help

Ricardo Ribalda (1):
init: Always create /run/tmp folder

Stephen Brennan (2):
Don't require ".git" to be a directory
Specify "refs/heads/__virtme__" in git push