Archinstall

This new release of archinstall addresses the many issues raised by the community.
We would like to thank every single one who submitted issues, gave feedback and most importantly suggested or contributed to solutions to all these issues.

This is by no means a perfect release, a lot of work remains ahead. But we believe this is a step in the right direction with both accessibility improvements as well as stability and some security enhancements for the guided template that we ship. Below is a list of all the changes since the previous version, and we will begin with the breaking changes to highlight those.

Breaking changes

* `Filesystem()`'s context manager *(using `with Filesystem()`)* no longer wipes the disk on entry, it no longer detects or handles differences in partition table vs expected partition table format (gpt vs mbr), instead those operations have moved to individual functions, see below in the `Filesystem` header as well as the `Installer` header, where mount logic has moved to.
* Going from `*` imports to targeted imports. This should not cause issues, but if there's any API call not available do let us know.
* `BlockDevice().json()` now only returns the string representation of path to the blockdevice *(`"/dev/sda"` for instance)*
* `BlockDevice().__dump__()` has one additional dimension to the JSON dictionary, where it represent itself as `"/dev/sda" : {"info": "value"}` rather than a flat structure.
* `BlockDevice().device_or_backfile` now return the backfile when the device is a loopdevice, `BlockDevice().device` still returns limited information of the type *(raid, crypt and others)*.
* `Partition().size` is now a read-only value and the parameter `Partition(size=X)` has hence been removed
* `Partition().allow_formatting` has been removed/deprecated, this is handled via direct function calls instead of automatically wiping a partition.
* `Partition().safe_to_format()` has been deprecated and removed since no automatic formatting or changes are done. Any destructive changes has to be explicitly called, and there for that is the safeguard instead.
* `Partition().format()` no longer has the argument `allow_formatting` as it's implicitly set whenever the function is called. This means that "testing" which file systems are supported is no longer supported, but instead we have a curated list of supported file systems that can be used.
* `Partition().format()` no longer uses the nomenclature `vfat` for the `filesystem` argument, instead `fat32` is used. This to better reflect the `mount` call later where `fat32` is specified and better reflects `man parted`'s expectations.
* `archinstall.JSON` serializer now correctly removes any key in a dictionary that starts with `!`, for instance `"!password" : "moo"` will get filtered out. To serialize a structure without loosing values, use `archinstall.UNSAFE_JSON` which is unsafe to use unless you know what you're doing.
* The `gfx_driver` argument in profiles etc no longer has supported for `Nvidia`, and instead of using sub-menu's under the `Nvidia` category there is now two top level choices of either `Nvidia (open-source)` and `Nvidia (proprietary)`. They both do the same as before, just on a top level of choices and as a single argument in JSON configurations/profiles.
* The default installation mountpoint is no longer `/mnt` but instead `/mnt/archinstall`
* Disk layout and configuration has moved from `user_configuration.json` into `user_disk_layouts.json`
*(It's still supported for as long as possible to define these in `user_configuration.json` for backwards compatibility reasons)*
* User credentials have moved from `user_configuration.json` into `user_credentials.json`
*(It's still supported for as long as possible to define these in `user_configuration.json` for backwards compatibility reasons)*
* Keywords in the JSON configuration has changed in the guided installer:
* `keyboard-language` has been re-named to `keyboard-layout`
* `harddrive` has been re-named to `harddrives` *(and changed structure)*
* `disk_layouts` have been added and replaces `harddrives` partially
* `users` have changed to `!users`
* `superusers` have changed to `!superusers`

Highlighted new features

* GRUB and Disk Encryption now properly works.
* BTRFS subvolumes *(very early beta feature, hence some limitations for initial tests. Supports only a [nested structure](https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Nested) and some [issues are known](https://github.com/archlinux/archinstall/issues/718) with this approach)*
* Archinstall now detects if `espeakup.service` is active on the installation ISO and copies the setup over to the installed medium automatically.
* Now supports multiple encrypted volumes *(slightly limited, it's either on or off, but multiple partitions will be encrypted)*. All encrypted partitions are protected by the passphrase as usual and stored in key-slot 0, bu every partition that is not the root partition will also have a computer generated passphrase stored on the root device under `/dev/cryptsetup-keys.d/` and that passphrase is stored in key-slot 1 on the partition. It's a unique passphrase per volume.
* More reliable disk operations *(partitioning, encryption and mounting)*
* A rudimentary [plugin-support](https://github.com/archlinux/archinstall/blob/master/archinstall/lib/plugins.py), allowing users to create their own plugins. There are four ways of activating the plugin support, either via `--plugin=url|location`, via en entry in `--config` as `{"plugin": "url|location"}` or via the API call `archinstall.load_plugin()` or finally via `pip install yourplugin` assuming that the plugin registers itself via Pythons builtin entry-point system as `archinstall.plugin`. These plugins are triggered/called at various points in the installation and more documentation on this specific will come and be continuously updated. One such step is `archinstall.pacstrap()` which calls `on_pacstrap(package_list)` in the plugin and the return value should be a curated list of packages, this enables for instance custom plugins to be written and deal with for instance AUR packages and deal with the custom operations needed and filter out those packages from the rest of the install process, leaving only official packages to be dealt with by the official installation process. *(again, just an example)*.
* [swap](https://wiki.archlinux.org/title/Swap) has been added into the mix using [zram](https://www.kernel.org/doc/html/latest/admin-guide/blockdev/zram.html) and [zram-generator](https://wiki.archlinux.org/title/Swap#zram-generator).
* Manual partitioning has been re-worked *(but released partially unfinished, the reason being it was impossible to release this version where this specific change was removed - as it's so intertwined with everything else that's been re-worked)*

Detailed changelog

Parameters

* More static arguments have been added to the initialization making for a more traditional argument parsing of known arguments.
* `--creds` is a new argument that take the previous `"users"`, `"superusers"` and `"root-password"` keys & values, this allows for separation of JSON machine configuration and user credentials which might want to be stored separately for security reasons.
* `--disk_layouts` is a new argument that takes the previous structure of `"harddrive"` which later was renamed `"harddrives"`. The old JSON structure of `"harddrives"` still remains in the `--config` input as an indicator of which harddrives to use and find from `--disk_layouts`.
* `--dry-run` has been added, which will abort before any changes are made on hardware. This allows for generating a configuration file which can then be passed to either of `--config`, `--creds` and `--disk_layouts`.
* `--mount-point` has been added to enable choosing where the installed system gets mounted during installation/configuration.
* `--plugin` has been added for loading external archinstall plugins via parameter.

Library changes

* `archinstall/lib/disk.py` has been split up into `archinstall/lib/disk/*.py` since it grew out of hand. The exposure of API calls should remain the same as they're exposed via `__init__.py`, but for contributors this change is worth noting when finding functions and files.
* Going from `*` imports to targeted imports. This should not cause issues, but if there's any API call not available do let us know.

General changes

* A dummy `epoll()` has been added in order to be able to run certain tests on Windows, for instance "reach mirror"-tests etc without causing import issues.
* `archinstall.generate_password(length)` has been added to generate a rudimentary password for disk encryption and other things. This using the `secrets` library and `string.printable` as a haystack.
* `archinstall.json_dumps(*args, **kwargs)` performs a safe `json.dumps()` call using the `archinstall.JSON` serializer
* `archinstall.UNSAFE_JSON()` has been added to serialize JSON structures without consideration of private information.
* `archinstall.SysCommandWorker()` now outputs the stdout as-is, without any attempt to make it pretty.
* `archinstall.SysCommandWorker()` *(and in extension, even `SysCommand()`)* now logs all commands executed under `/var/log/archinstall/cmd_history.txt` for easier debugging and learning which commands we executed. Currently we don't filter anything, so there will be a lot of spam from `parted()` calls and `lsblk` multiple times *"for no reason"*.
* `archinstall.SysCommand()` has support for retrieving slices using `SysCommand("ls")[:10]` for instance, where the output generated by the internal `SysCommandWorker()` call will be simplified into retrieving the final output and only the first 10 bytes in this example.
* `archinstall.cpuinfo()` has been created to return CPU information from `/proc/cpuinfo` in a more general sense
* `archinstall.has_amd_cpu()` and `archinstall.has_intel_cpu()` now uses the above `cpuinfo()` call.
* `archinstall.meminfo()` has been created to return memory information from `/proc/meminfo` as a whole or a specific key if specified.
* `archinstall.graphics_devices()` now correctly catches `" 3D "` entries from `lspci` correctly.
* `archinstall.cpu_vendor()` now uses `cpuinfo()` instead.
* `archinstall.cpu_model()` now uses `cpuinfo()` instead
* `archinstall.sys_vendor()` has been added which returns the vendor as specified by `/sys/devices/virtual/dmi/id/sys_vendor`
* `archinstall.product_name()` has been added which returns the product name from `/sys/devices/virtual/dmi/id/product_name`
* `archinstall.mem_available()` returns the available memory using `meminfo()`
* `archinstall.mem_free()` returns the available free memory using `meminfo()`
* `archinstall.mem_total()` returns the total amount of memory using `meminfo()`
* `archinstall.virtualization()` returns the used virtualization platform using `systemd-detect-virt`
* `archinstall.sort_mirrorlist()` has been added to sort the `/etc/pacman.d/mirrorlist` according to preference of URL prefix. Defaults to HTTPS before HTTP, while preserving origin of the servers in groups.
* `archinstall.filter_mirrors_by_region()` now takes the `sort_order` parameter, which uses the `sort_mirrorlist()` call to arrange mirrors in a preferred way.
* `archinstall.use_mirrors()` now correctly iterates over multiple regions to support multi-region support for mirror selection *(menu system will be updated in the next release to offer this)*
* `archinstall.list_mirrors()` also supports the `sort_order` parameter as previously mentioned.
* `archinstall.storage` has gotten three new keys, `ENC_IDENTIFIER` which is the prefix for all loopdevices upon opening encrypted volumes, `DISK_TIMEOUTS` is the time in seconds in which archinstall should sleep pending a retry of the same operation, `DISK_RETRY_ATTEMPTS` which is the amount of retries of the same disk operation before quitting *(and usually exception-error out)*
* `archinstall.Boot()` which can temporarily boot the installed system without having to reboot the running machine now properly avoids using output-pagers when outputting the terminal output which is read by archinstall. In the future, we will use `--console=pipe` instead of reading the stdout via *"brute force"*
* `archinstall.Boot()` no longer uses `machinectl` to run commands in the temporary boot environment, instead it uses `systemd-run`
* `archinstall.select_encrypted_partitions()` has been added to mark partitions as encrypted. In the future this function will allow for selecting and manipulating which partitions are encrypted or not. For now, when called it will mark all partitions *(except `/boot`)* as encrypted. There is no toggle for off in this release and no user-interaction.
* A bunch of language improvements throughout the output

Profiles and it's API

* `archinstall.is_desktop_profile()` has been added to determine whether or not a specific profile is of the category *"desktop"*.
* `archinstall.Profile().get_profile_description()` has been added to return a profiles `__description__` variable if one exists.
* [cutefish](https://github.com/cutefishos) has been [added](https://github.com/archlinux/archinstall/blob/2058e61dc7d0662400a99157671d905e8c7c55f7/profiles/cutefish.py) as a desktop environment.
* `lightdm-gtk-greeter` have changed to `lightdm-deepin-greeter`
* `gnome-software-packagekit-plugin` have been added to `gnome` in order to get the Gnome Software app to work properly
* `ark` has been added to `KDE` to allow for archive management
* `xarchiver` have been added to `xfce4` to allow for archive management
* `gvfs` have been added to `xfce4` to fix *"trash"*
* `network-manager-applet` added to `xfce4` to allow for network management using NetworkManager

Guided installation changes

* `archinstall.ask_for_swap()` has been added as a API call, which is used in the guided installer to ask if the users wants swap or not.
* `archinstall.ask_for_bootloader()` have been modified to allow listing *"advanced"* boot loaders via the `advanced_options=` parameter.
* `archinstall.ask_for_audio_selection()` now defaults to pipewire instead of pulseaudio, but still supports choosing either or none.
* `archinstall.ask_for_main_filesystem_format()` now supports the `advanced_options=` parameter, which will unlock `ntfs` as the root filesystem *(hidden and triggered by `--advanced`)*
* `archinstall.get_default_partition_layout()` returns the *"ideal"* disk layout for one or more disk setups.
* In turn, for single disk `archinstall.suggest_single_disk_layout()` has been added to suggest a single disk layout setup
* For multi-disk setups `archinstall.suggest_multi_disk_layout()` has been added to suggest a multi-disk layout
* `archinstall.manage_new_and_existing_partitions()` has been added to handle the menu-operations and alternatives for manipulating and managing partitions during manual configuration. This feature is in an early stage and will contain bugs, dead code paths and other edge cases that we haven't picked up yet. We release it partially unfinished because this release is getting too big as it is and we need more user input. And finding the optimal layouts and suggestions are tricky. **we strongly suggest manually partitioning the disk and using it *"as is"* if you want to get full control of partition layouts** but we provide this tool as a test to see if it would be viable.
* General warnings have been added surrounding AMD, Intel and Nvidia hardware.
* Graphics driver defaults to `All open-source (default)` if no other option is given and a desktop profile has been selected.
* The guided installation will now log and output the relevant hardware specs, which we can use in support arrends assuming the user is willing to include the log when asking for support.
* `load_config()` has been added in guided to facilitate the setup of all the required arguments prior to running the installation steps.
* `perform_filesystem_operations()` have been added to deal with disk operations. A lot of the encryption and formatting have moved away from the guided core and in to the respective classes *(see other parts of the release notes)* to simplify the guided profile a bit and separate it's duties.
* `perform_installation_steps()` have been re-named to `perform_installation()` which now deals with mounting disk layouts and is cleaned up to only contain installation steps and no disk operations *(other than mounting)*

`archinstall.Installer` *(and the installer.py file in general)*

* `archinstall.InstallationFile()` has been added, which is a per-installation helper to create files using the `with open()` context with correct file permissions as seen by the installation. `owner=` will look up the user-id within the installation and set ownership upon closing.
* `archinstall.accessibility_tools_in_use()` returns if `espeakup.service` is active in the live medium, can be used to carry over functionality into the installed system.
* `archinstall.Installer()` now installs `archinstall.__accessibility_packages__` when detected in use on the live medium.
* `archinstall.Installer().partitions()` returns any `Partition()` object in use under the installers mountpoint.
* `archinstall.Installer().mount_ordered_layout()` is a new function to mount a defined JSON structure of partitions and mountpoints, as well as meta-info such as encryption and passwords. This replaces the old way of setting up block devices, partitions and dealing with encryption.
* `archinstall.Installer()`
* `archinstall.Installer().mount_ordered_layout()` will create a encryption key on `/` *(root)* if multiple volumes are detected as encrypted. It will then add that encryption key in key-slot 1 on that encrypted volume. Where key-slot 0 will remain the overall encryption key used for the root device. This ensures a secure automatic-unlock key that the system can use during boot, but allows a fallback manual passphrase for manual mounting from say other operating system instances or recovery modes.
* `archinstall.Installer().activate_ntp()` has been replaced by `archinstall.Installation().activate_time_syncronization()` but the old function will live on for a version or two with a warning.
* `archinstall.Installation().enable_espeakup()` enables text to speech services in the installed system.
* `archinstall.Installation().run_as()` has been improved to deal with quote issues a bit better *(still not perfect)*
* `archinstall.Installation().mkinitcpio()` will now remove `fsck` in the HOOKS if `ntfs` is detected as the root filesystem.
* `archinstall.Installation().setup_swap()` is a new helper function to configure swap, currently only `zram` is supported.
* `archinstall.Installation().add_bootloader()` now correctly configurs and creates boot entries based on the selected kernels
* `archinstall.Installation().add_bootloader()` now has support for [efistub](https://wiki.archlinux.org/title/EFISTUB) when used with your own custom installation scripts or via passing the `--advanced` flag to `archinstall`.
* `archinstall.Installation().chown()` has been added as a convenience function to chown files inside the installation using the installation user-map.
* `archinstall.Installation().create_file()` is a wrapper for `archinstall.InstallationFile()`.

`archinstall.luks`

* `archinstall.luks2()` has two new functions:
* `.add_key()` which adds a keyfile to a slot on a existing encrypting volume
* `.crypttab()` which creates a `/etc/crypttab` entry for a volume in the installation *(the installation has to be passed to the function)*

`archinstall.BlockDevice`

* More detailed output in `__repr__` to represent the class object instance
* Now supports `len()` to get the amount of partitions on the block device.
* Now supports `sorted()` by comparing `self.path` against other objects `.path`.
* `BlockDevice().partition_type` returns the partition table type *(GPT or MBR)*
* `BlockDevice().size` has been added to return the size of the device in GB *(not GiB)*
* `BlockDevice().bus_type` has been added to return the type of underlying bus *(nvme etc)*
* `BlockDevice().spinning` has been added, returns `True` if the underlying physical device is a spinning device
* `BlockDevice().free_space` has been added to report how much available space is free on the device *(still a bit buggy when no partitions exist)*
* `BlockDevice().largest_free_space` has been added and reports the largest continuous free space on the block device.
* `BlockDevice().first_free_sector` has been added and reports the first free sector and is reported in MB, GB or other similar unit.
* `BlockDevice().first_end_sector` has been added and reports the first free sector closest to the first available end, also in units of MB etc *(if partitions exist, it will report the last sector right before that partition)*
* `BlockDevice().get_partition(uuid)` returns the `Partition()` object from the block device partitions table based on the `uuid` *(PARTUUID)` filter.

`archinstall.Filesystem`

* `Filesystem().partuuid_to_index` has been added to give the positional index *(order)* of a specific partition in relation to other partitions based on it's `uuid` *(PARTUUID)*
* `Filesystem().load_layout` has been added to load a specific partition layout, this function will be moved in the future to create better separation of duties, but in the large split/move/change this is where it ended up. The function takes a dictionary structure *(see examples in the examples catalogue or generate with `--dry-run`)* and converts it into disk and partition operations, such as `mklabel`, `mkpart` and encryption calls.
* `Filesystem().add_partition()` has been re-worked in a way to allow for more gracefully letting the kernel come up to speed. it will re-try grabbing the newly formed partitions PARTUUID for a set period of time before giving up.

`archinstall.Partition`

* A new `__dump__` function has been added for use with `archinstall.JSON` serializer
* `Partition().sector_size` has been added to return the logical sector size
* `Partition().start` has been added to return the start sector from `(sfdisk /path).start`
* `Partition().end` has been added to return the end sector from `(sfdisk /path).size`
* `Partition().size` has been added as a read only property which returns the size of the partition in GB
* `Partition().boot` has been added and returns if the partition is set/marked as bootable
* `Partition().partition_type` has been added and returns the partition table type that this partition lives under
* `Partition().uuid` now features retrying the `PARTUUID` retrieval, as the kernel some times hands over to the user before certain disk operations are actually complete. These retry-attempts are configurable both in terms of delay-time between and amount of attempts.
* `Partition()._safe_uuid()` has been added as a compliment to `.uuid` but does not raise any exceptions and does not attempt to do any re-tries. This feature is meant to be a safe way to "get" the PARTUUID without halting other operations, such as printing information or representation of the `Partition()` object.
* `Partition().format()` has support for additional file systems
* `ext2`
* `ntfs` *(as of writing, only works with the latest Linux kernel and not `-lts` kernels. And it's highly experimental and lacks fscheck, but its supported)*
* `Partition().format()` now supports passing options such as compression or setting [case-insensitivity](https://github.com/archlinux/archinstall/issues/380) via the JSON config or API calls.

btrfs and subvolumes
*This feature is released as a BETA feature, do expect issues but feel free to try out the limited support for btrfs subvolumes and report feedback*

* `archinstall.mount_subvolume` has been added as a helper function to mount a btrfs subvolume to a specific location
* `archinstall.create_subvolume` has been added as a helper function to create a subvolume on a existing btrfs mounted volume location

`archinstall.lib.disk.helpers*`, `archinstall.lib.disk.user_guides*` & `archinstall.lib.disk.validators*`

* `archinstall.convert_size_to_gb()` has been added and takes a size in `bytes` and converts it to `gigabytes` *(I sure hope I get the convention right here, bytes to GB and not GiB)*.
* `archinstall.sort_block_devices_based_on_performance()` performs a rudimentary sort of block devices using an *"algorithm"* based on *"performance"*.
* `archinstall.filter_disks_below_size_in_gb()` generates an iterator which filters out blockdevices based on a lowest GB number
* `archinstall.select_largest_device()` selects the largest disk from a collection of disks
* `archinstall.convert_to_gigabytes(string)` is a limited helper function to convert MB and TB to GB
* `archinstall.get_mount_info()` now supports more features
* Traversing the targeted structure backwards when `traverse=True`.
* Added `return_real_path=True` which when True returns two return values rather than the mount information alone, this is so that in junction with `traverse` it will return the path in which it found the mount information. `/mnt/archinstall/opt/` for instance will not return any info, but `/mnt/archinstall` will so that's the path that will be returned together with the mount information at that location.
* `archinstall.encrypted_partitions()` generates an iterator containing all partition blobs in a JSON structure that are encrypted
* `archinstall.find_partition_by_mountpoint()` iterates a JSON structure and find the relative mountpoint, `/boot` for instance can be located *(note: not `/mnt/boot`)*
* `archinstall.partprobe()` A new helper function which simply calls `partprobe`
* `archinstall.convert_device_to_uuid()` A new helper function to convert `/dev/sda` or `/dev/sda2` to a UUID *(not PARTUUID)*
* `archinstall.suggest_single_disk_layout()` Has been added and performs rudimentary guesswork to set up the best partition layout and/or subvolumes for the selected block device.
* `archinstall.suggest_multi_disk_layout()` Has been added and performs rudimentary guesswork to set up the best partition layout and/or subvolumes for multiple selected block devices.
* `archinstall.valid_parted_position()` Has been added as a validation function before sending start/end positions to `parted`
* `archinstall.valid_fs_type()` Has been added and replaces the previous `Partition().format()` of `/dev/null` to detect if a file system is valid or not. Instead, this function will perform a static lookup of hardcoded and supported file systems.

Building and contributing

* Updated [flake8](https://github.com/archlinux/archinstall/blob/e729457b6c12a00b17207254ee72e98b78912f8d/.flake8#L4-L9) to be a bit more strict but stay in line with our contribution guidelines allowing some flexibility to the code.
* Added automatic [ISO builds](https://github.com/archlinux/archinstall/blob/e729457b6c12a00b17207254ee72e98b78912f8d/.gitlab-ci.yml) based on the official Arch Linux ISO configuration [releng](https://wiki.archlinux.org/title/Archiso#Prepare_a_custom_profile) for GitLab, this in perparation for the potential/eventual move to GitLab for all Arch Linux related projects.
* Automatic build of man pages in the [PKGBUILD](https://github.com/archlinux/archinstall/blob/e729457b6c12a00b17207254ee72e98b78912f8d/PKGBUILD#L26)

Highlighted features:
================

* BTRFS subvolumes (limited functionality on this first release, still in beta and very niche usecases)
* Re-worked partitioning
* systemd-timesyncd instead of ntpd

*A more detailed inventory of changes will be put up on the release day, the list is a bit long*

Known issues:
===========
* ~~Encryption with GRUB still broken~~ *(Fixed in this RC2 release)*
* ~~Occasional .format() glitch during partitioning, see 689~~ *(Fixed in this RC2 release)*
* Multi-disk layouts are incomplete, please submit feedback here or on discord even if we know this is an issue

General discussions are welcome in 692

Highlighted features:
* BTRFS subvolumes (limited functionality on this first release)
* Re-worked partitioning
* `systemd-timesyncd` instead of `ntpd`

Known issues:
* Encryption with GRUB still broken
* Occasional `.format()` glitch during partitioning, see 689 *(please submit logs in that issue if this happens)*
* Multi-disk layouts are incomplete with BTRFS, please submit feedback here or on discord even if we know this is an issue

**Known Issues:**
* Partitioning still has issues with certain layouts. **Workaround:** Manually creating partitions and selecting *"use /mnt as is"* or *"Re-use partitions"* *(after a valid file system was created on the manually created partitions)* will solve most issues. Wiping the desired drive before running archinstall is also a workaround for now but remember to backup your data! Work is already underway to correct this, but will not make it in time for this release, follow the progress here: 426
* Dual-Boot with Windows works, but there's an issue with Windows creating a very small `/boot` partition which causes issues, please track the progress here: 527
* 536: The output from `pacman` *(via `pacstrap` also)* looks broken/corrupt/garbage - This is a cosmetic issue only, and installs work fine. Will be fixed!

Top New Features
==============

* Archinstall now supports `GRUB` as a secondary boot loader. It's **still experimental**, so any issues would be appreciated if they got reported.
* Moved the IRC channel from FreeNode to LiberaChat
* `archinstall.Installer()` *(and subsequently the guided installer)* now supports selecting kernels
* `archinstall.Installer()` now supports modifying kernel parameters and other `mkinitcpio` related stuff
* `archinstall.Installer().pacstrap()` now shows a "progress bar" of what's going on
* Archinstall now features a multi-selection mode thanks to `archinstall.generic_multi_select()` - which will help users select multiple options. We've begun by allowing for selecting multiple kernels, but in the future more options that support multiple choices, such as multiple mirror selection will be introduced.
* Archinstall supports loading configuration from a JSON file, if you wish to re-use a setup. We also ship a [example](https://github.com/archlinux/archinstall/blob/1c9adbbedfcbd821fc4f4a74f1e63699ac33d6f5/examples/config-sample.json) JSON file, *(**warning**: It does use `/dev/sda` in the example)*. The JSON structure that is shown on screen during installation and in `/var/log/archinstall/install.log` is valid configuration and can be re-used. There's also an example with [additional custom commands](https://github.com/archlinux/archinstall/blob/1c9adbbedfcbd821fc4f4a74f1e63699ac33d6f5/examples/custom-command-sample.json) that we support in the `--config` structure, which isn't produced by the guided installer log output.
* The guided installer now allows you to optionally activate `NTP` if a time-zone was specified
* A very rudimentary `runas` has been added to `archinstall.Installer().arch_chroot()`, beware that a any quotations might break this function call as it's *experimental* for now. There is also no error checks if the user you try to run exists or not, so make sure to create the user first. Future improvements will be made here.
* `archinstall.Installer().mkinitcpio()` has been added, which can be called after modifications to `Installer().MODULES` or `.HOOKS` have been made.
* `archinstall.Installer().minimal_installation()` now intalls Intel and AMD `ucode` binaries by default.
* Archinstall's guided installer now has a `--advanced` flag, which will allow you to unlock certain additional configuration questions, such as system language. *(because how the `/etc/locale.conf` and `localectl list-locales` work, there's no way to get the full list of supports locales, so you will have to know which locale to write in order for this to work, hence the `--advanced` flag requirement)*
* Keyboard map selection now translates in to `X11`-desktops as well. This thanks to https://github.com/archlinux/archinstall/issues/147#issuecomment-812945222 *(We've moved to `localectl` for setting locales, which vastly improves reliability but also locks us into systemd more than ever)*
* Archinstall will now identify as `ArchInstall` for most of it's web requests, such as getting the mirror-list and similar activities.
* `archinstall.check_mirror_reachable()` has been added to detect if we have a reliable connection to the configured mirrors (locally or remote)
* `archinstall.Boot(<Installer() instance>)` [has been added](https://github.com/archlinux/archinstall/blob/1c9adbbedfcbd821fc4f4a74f1e63699ac33d6f5/archinstall/lib/systemd.py#L49-L121), which can temporally boot up the new installation without having to re-boot from the ISO installation medium. A use case can be where you would need to communicate with systemd to set certain configuration parameters, or communicate via dbus to services etc. As an example, this is how it could be used:
python
with Boot(archinstall.Installer("/mnt")) as session:
session.SysCommand(["localectl", "set-keymap", '""'])


General Guided installation changes
============================
* Archinstall's guided installer will only execute if run as root
* `--help` was added, which simply points to `man archinstall`
* Can now be run in EFI and BIOS mode.
* The `--advanced` flag was added, to unlock certain options that require more experienced users
* Currently only `--sys-language` and `--sys-encoding` is unlocked/hidden from the general view
* `--filesystem` flag now works properly to skip the file system question
* It now asks for a default boot-loader if EFI was detected, in BIOS mode it defaults to GRUB
* Multiple kernel selection has been added *(mentioned above as well)*
* `NTP` option has been added after time-zone selection
* Keyboard layout selection has been stabilized
* `--services` flag/option has been added
* `--custom-commands` flag/option have been added
* `--silent` flag has been added, which mutes the `"Would you like to chroot into" question at the end as well as other user inputs. This requires a `--config` to be given in the place of asking the user for questions.

New Profiles
==========
* server *(lets you select multiple server-category application profiles)*
* deepin
* enlightenment
* sway

Profile changes
---------------------
* i3 has an updated package list, which now includes `lightdm-gtk-greeter`, `lightdm` and `dmenu`
* *(Breaking Change)* Most desktop applications which has little to no additional configuration other than installing packages, have now had it's `profiles/application/<profile>.py` removed, and the contents have moved in to the `profiles/<profile>.py` instead. Since it was just one additional call for "no reason".

New Application Profiles
===================
*Keep in mind that these are only accessibly if you script your own installations or to some extent select the server profile above, they are outside of the scope of the guided installer in general*

* Cockpit
* docker
* httpd *(apache)*
* lighttpd *(v1.0)*
* mariadb
* nginx
* postgresql
* sshd
* tomcat

General API Changes
=================

* With the previous change, we also introduce loading configuration from a JSON file with `--config <config_path.json>`, this configuration equals the same structure you see before the installation begins, which means you can re-use a configuration and inject it with `--config`.
* *(Breaking Change)* The magic global variable `installation` has been removed effect of immediately, instead, `archinstall.storage["installation_session"]` is created whenever `archinstall.Installer()` is instantiated. This is to make it less mysterious and keeping it simple - as well as satisfy the linters which produced over 400 errors because of this alone.
* *(Breaking Change)* Just like `installation` moved, so has `gfx_driver_packages` which is now `archinstall.storage["gfx_driver_packages"]`
* *(Breaking Change)* A lot of function names have been re-named in order to facilitate some stricter PEP-8 syntaxes, this in order to make linters happier and produce less errors.
* `archinstall.sys_command()` -> `archinstall.SysCommand()` *(This function has also been re-worked from the ground up as well, but should be backwards compatible, any issues of incompatibility should be reported)*
* `JSON_Encoder` -> `JsonEncoder`
* `hasUEFI` -> `has_uefi`
* `hasWifi` -> `has_wifi`
* `graphicsDevices` -> `graphics_devices`
* `hasNvidiaGraphics` -> `has_nvidia_graphics`
* `hasAmdGraphics` -> `has_amd_graphics`
* `hasIntelGraphics` -> `has_intel_graphics`
* `archinstall.Installer().copy_ISO_network_config` -> `archinstall.Installer().copy_iso_network_config`
* `archinstall.getHwAddr` -> `archinstall.get_hw_addr`
* `archinstall.enrichIfaceTypes` -> `archinstall.enrich_iface_types`
* `archinstall.wirelessScan` -> `archinstall.wireless_scan`
* `archinstall.GetWirelessNetworks` -> `archinstall.get_wireless_networks`
* `archinstall.` -> `archinstall.`
* `archinstall.` -> `archinstall.`
* *(Deprecated)* `archinstall.LOG_LEVELS` have been deprecated, instead, normal `logging.<level>` can be used. Next version will remove these old level definitions permanently.
* `archinstall.Partition().mount()` now supports passing down mount options when called via `options=<string with mount flags>`
* `archinstall.Filesystem(mode=<archinstall.GPT|archinstall.MBR>)` now supports `MBR`, allowing for older hardware to be used. This is still some what experimental, and `archinstall.Filesystem().use_entire_disk()` will honor `GPT` if UEFI is detected but fall back on `MBR` if EFI vars isn't present. Use manual formatting if you want to override and use `MBR` on EFI compliant hardware.
* `archinstall.disk_layouts()` have been added as a helper function to get an overview of the partition layout at any given moment. This function is called before and after an installation and added to the logs for easier support. If you wish to hide any disk information from the logs, make sure to remove this if you do not wish to expose your total disk setup. Only the disk SIZE and TYPE are added to the logs other than the identifiers themselves.
* `locate_binary()` now raises a `RequirementError` when the binary can not be found.
* *(Breaking Change)* `SysCommand()` Has been re-worked a lot, the following logic has been split into a new class called `SysCommandWorker()`
* It no longer has the ability to emulate a command, this might be brought back later if the need for it is great.
* `.raw_cmd` no longer exists, it's just `cmd`
* `.callback` and `.start_callback` has been merged into `.callbacks` which now is a dictionary with two keys currently: `on_start` and `on_end` *(currently not in use, but that's the future)*
* `.peak_output` is a boolean to control if we should trail the command output *(we don't strip `\n`, but we do stay on the same line as long as no new line was given)*
* `.status` has been removed, instead you can rely on `.is_alive()` and `.exit_code`.
* `.is_alive()` is a new function that does what it sounds like
* It now supports the `in` operator, `if b"/home" in SysCommand("pwd")` for instance.
* `for line in SysCommand()` as previously mentioned now only iterates the latest output since the last iteration.
* `.write()` has been added to communicate with a process
* `.tell()` will give you the current position in the trace log.
* `.seek()` will move to a position in the trace log
* `.peak()` is the internal function called on bytes to print it, something you wouldn't normally use, but you can override it.
* `.poll()` will perform emptying of buffers and perform work on the process, it also updates if the process is alive or not.
* `.execute()` will initate execution of the command and setup all the registers and stuff
* `.decode(encoding='utf-8')` will return the trace log decoded in the given encoding
* `SysCommand()` has been reworked - but should work the same way as before, but while utilizing `SysCommandWorker()`:
* `.trace_log` is now a property that fetches the `SysCommandWorker()`'s instance variable `._trace_log`
* If `SysCommand()` is put in a `with` context, it will return the inner `SysCommandWorker()` for raw processing, rather than returning itself.
* `SysCommand()` will not raise exceptions, instead it will log errors, instead `handle.exit_code` should be used. This was done to weed out bad behavior, since some commands use exit codes as status messages rather than actual errors.
* Using `SysCommand()` in a string, for instance like `f"This path is cool: {SysCommand('pwd')}"` will automatically decode into a `UTF-8` string of the output from the command.
* `SysCommand()` can also be JSON-dumped with the `JsonEncoder`, without the trace log.
* Also has the `.decode()` function
* `archinstall.pid_exists()` has been added to find out if a pid exists or not
* `archinstall.run_custom_user_commands()` have been added as a helper function to execute custom commands in the `--config` file.
* Most modules have now gotten a `__packages__` definition, for instance in `hardware.py` a `__packages__` has been created with the different drivers that it offers. This will allow us to do package inventory externally. Which will help with giving feedback of what we support. It can also be used in projects like [archoffline](https://github.com/Torxed/archoffline) to source packages during build time.
* hardware now also maintains a list of graphics drivers, called `archinstall.lib.hardware.AVAILABLE_GFX_DRIVERS`
* `archinstall.has_amd_cpu()` has been added
* `archinstall.has_intel_cpu()` has been added
* `archinstall.is_vm()` has been added
* A lot of docstring and typing has been done to help IDE users navigate the code.
* Listing keyboard mappings now uses `localectl list-keymaps`
* `archinstall.set_keyboard_language` now uses `localectl` instead of `loadkeys`
* *(Breaking Change)* `archinstall.Profile()` will now reset the namespace on every `with` context opening. This to ensure that it's started with the correct name-space, make sure to override `.original_namespace` if you want to take back control, otherwise whatever the namespace was on creation will be reverted on each instance.
* *(Breaking Change)* `archinstall.Profile().is_top_level_profile` has been re-worked to make sure not just a comment saying ` top_level_profile = True` exists, but it has to be an actual declared variable now.
* An error in Profile lookups have been fixed, where the search paths *(`PROFILE_PATH`)* was some what broken due to a previous re-work.
* `archinstall.generic_multi_select()` has been added which allows for selecting multiple options.
* `archinstall.MiniCurses` has been added, to emulate `ncurses` some what, without having to pull in all that it entails.
* `archinstall.ask_for_bootloader()` has been added.
* `archinstall.select_profile()` will now only show top level profiles, but entering any valid profile should also be supported.
* `archinstall.select_driver()` has been added, to select available graphics drivers
* `archinstall.select_kernel()` has been added, to select given kernels, uses `generic_multi_select` for this
* A lot of changes to the documentation, mainly surrounding `--config` has been re-worked or added.
* Introduced linting and automated ISO builds on commits and pull requests for easier contributions.
* Preprations to Introduced `__description__` in profiles have been introduced, this will later show a short description in the selection process

**Known Issues:**
* Partitioning still has issues with some layouts, manually creating partitions and selecting *"use /mnt as is"* or wiping the desired drive before running archinstall is two workarounds for now *(Remember to backup your data)*. Work is already underway to correct this, but will not make it in time for this release.

Top New Features
==============

* Archinstall now supports `GRUB` as a secondary boot loader. It's **still experimental**, so any issues would be appreciated if they got reported.
* `archinstall.Installer()` *(and subsequently the guided installer)* now supports selecting kernels
* `archinstall.Installer()` now supports modifying kernel parameters and other `mkinitcpio` related stuff
* `archinstall.Installer().pacstrap()` now shows a "progress bar" of what's going on
* Archinstall now features a multi-selection mode thanks to `archinstall.generic_multi_select()` - which will help users select multiple options. We've begun by allowing for selecting multiple kernels, but in the future more options that support multiple choices, such as multiple mirror selection will be introduced.
* Archinstall supports loading configuration from a JSON file, if you wish to re-use a setup. We also ship a [example](https://github.com/archlinux/archinstall/blob/1c9adbbedfcbd821fc4f4a74f1e63699ac33d6f5/examples/config-sample.json) JSON file, *(**warning**: It does use `/dev/sda` in the example)*. The JSON structure that is shown on screen during installation and in `/var/log/archinstall/install.log` is valid configuration and can be re-used. There's also an example with [additional custom commands](https://github.com/archlinux/archinstall/blob/1c9adbbedfcbd821fc4f4a74f1e63699ac33d6f5/examples/custom-command-sample.json) that we support in the `--config` structure, which isn't produced by the guided installer log output.
* The guided installer now allows you to optionally activate `NTP` if a time-zone was specified
* A very rudimentary `runas` has been added to `archinstall.Installer().arch_chroot()`, beware that a any quotations might break this function call as it's *experimental* for now. There is also no error checks if the user you try to run exists or not, so make sure to create the user first. Future improvements will be made here.
* `archinstall.Installer().mkinitcpio()` has been added, which can be called after modifications to `Installer().MODULES` or `.HOOKS` have been made.
* `archinstall.Installer().minimal_installation()` now intalls Intel and AMD `ucode` binaries by default.
* Archinstall's guided installer now has a `--advanced` flag, which will allow you to unlock certain additional configuration questions, such as system language. *(because how the `/etc/locale.conf` and `localectl list-locales` work, there's no way to get the full list of supports locales, so you will have to know which locale to write in order for this to work, hence the `--advanced` flag requirement)*
* Keyboard map selection now translates in to `X11`-desktops as well. This thanks to https://github.com/archlinux/archinstall/issues/147#issuecomment-812945222 *(We've moved to `localectl` for setting locales, which vastly improves reliability but also locks us into systemd more than ever)*
* Archinstall will now identify as `ArchInstall` for most of it's web requests, such as getting the mirror-list and similar activities.
* `archinstall.check_mirror_reachable()` has been added to detect if we have a reliable connection to the configured mirrors (locally or remote)
* `archinstall.Boot(<Installer() instance>)` [has been added](https://github.com/archlinux/archinstall/blob/1c9adbbedfcbd821fc4f4a74f1e63699ac33d6f5/archinstall/lib/systemd.py#L49-L121), which can temporally boot up the new installation without having to re-boot from the ISO installation medium. A use case can be where you would need to communicate with systemd to set certain configuration parameters, or communicate via dbus to services etc. As an example, this is how it could be used:
python
with Boot(archinstall.Installer("/mnt")) as session:
session.SysCommand(["localectl", "set-keymap", '""'])


General Guided installation changes
============================
* Archinstall's guided installer will only execute if run as root
* `--help` was added, which simply points to `man archinstall`
* Can now be run in EFI and BIOS mode.
* The `--advanced` flag was added, to unlock certain options that require more experienced users
* Currently only `--sys-language` and `--sys-encoding` is unlocked/hidden from the general view
* `--filesystem` flag now works properly to skip the file system question
* It now asks for a default boot-loader if EFI was detected, in BIOS mode it defaults to GRUB
* Multiple kernel selection has been added *(mentioned above as well)*
* `NTP` option has been added after time-zone selection
* Keyboard layout selection has been stabilized
* `--services` flag/option has been added
* `--custom-commands` flag/option have been added
* `--silent` flag has been added, which mutes the `"Would you like to chroot into" question at the end as well as other user inputs. This requires a `--config` to be given in the place of asking the user for questions.

New Profiles
==========
* server *(lets you select multiple server-category application profiles)*
* deepin
* enlightenment
* sway

Profile changes
---------------------
* i3 has an updated package list, which now includes `lightdm-gtk-greeter`, `lightdm` and `dmenu`
* *(Breaking Change)* Most desktop applications which has little to no additional configuration other than installing packages, have now had it's `profiles/application/<profile>.py` removed, and the contents have moved in to the `profiles/<profile>.py` instead. Since it was just one additional call for "no reason".

New Application Profiles
===================
*Keep in mind that these are only accessibly if you script your own installations or to some extent select the server profile above, they are outside of the scope of the guided installer in general*

* Cockpit
* docker
* httpd *(apache)*
* lighttpd *(v1.0)*
* mariadb
* nginx
* postgresql
* sshd
* tomcat

General API Changes
=================

* With the previous change, we also introduce loading configuration from a JSON file with `--config <config_path.json>`, this configuration equals the same structure you see before the installation begins, which means you can re-use a configuration and inject it with `--config`.
* *(Breaking Change)* The magic global variable `installation` has been removed effect of immediately, instead, `archinstall.storage["installation_session"]` is created whenever `archinstall.Installer()` is instantiated. This is to make it less mysterious and keeping it simple - as well as satisfy the linters which produced over 400 errors because of this alone.
* *(Breaking Change)* A lot of function names have been re-named in order to facilitate some stricter PEP-8 syntaxes, this in order to make linters happier and produce less errors.
* `archinstall.sys_command()` -> `archinstall.SysCommand()` *(This function has also been re-worked from the ground up as well, but should be backwards compatible, any issues of incompatibility should be reported)*
* `JSON_Encoder` -> `JsonEncoder`
* `hasUEFI` -> `has_uefi`
* `hasWifi` -> `has_wifi`
* `graphicsDevices` -> `graphics_devices`
* `hasNvidiaGraphics` -> `has_nvidia_graphics`
* `hasAmdGraphics` -> `has_amd_graphics`
* `hasIntelGraphics` -> `has_intel_graphics`
* `archinstall.Installer().copy_ISO_network_config` -> `archinstall.Installer().copy_iso_network_config`
* `archinstall.getHwAddr` -> `archinstall.get_hw_addr`
* `archinstall.enrichIfaceTypes` -> `archinstall.enrich_iface_types`
* `archinstall.wirelessScan` -> `archinstall.wireless_scan`
* `archinstall.GetWirelessNetworks` -> `archinstall.get_wireless_networks`
* `archinstall.` -> `archinstall.`
* `archinstall.` -> `archinstall.`
* *(Deprecated)* `archinstall.LOG_LEVELS` have been deprecated, instead, normal `logging.<level>` can be used. Next version will remove these old level definitions permanently.
* `archinstall.Partition().mount()` now supports passing down mount options when called via `options=<string with mount flags>`
* `archinstall.Filesystem(mode=<archinstall.GPT|archinstall.MBR>)` now supports `MBR`, allowing for older hardware to be used. This is still some what experimental, and `archinstall.Filesystem().use_entire_disk()` will honor `GPT` if UEFI is detected but fall back on `MBR` if EFI vars isn't present. Use manual formatting if you want to override and use `MBR` on EFI compliant hardware.
* `archinstall.disk_layouts()` have been added as a helper function to get an overview of the partition layout at any given moment. This function is called before and after an installation and added to the logs for easier support. If you wish to hide any disk information from the logs, make sure to remove this if you do not wish to expose your total disk setup. Only the disk SIZE and TYPE are added to the logs other than the identifiers themselves.
* `locate_binary()` now raises a `RequirementError` when the binary can not be found.
* *(Breaking Change)* `SysCommand()` Has been re-worked a lot, the following logic has been split into a new class called `SysCommandWorker()`
* It no longer has the ability to emulate a command, this might be brought back later if the need for it is great.
* `.raw_cmd` no longer exists, it's just `cmd`
* `.callback` and `.start_callback` has been merged into `.callbacks` which now is a dictionary with two keys currently: `on_start` and `on_end` *(currently not in use, but that's the future)*
* `.peak_output` is a boolean to control if we should trail the command output *(we don't strip `\n`, but we do stay on the same line as long as no new line was given)*
* `.status` has been removed, instead you can rely on `.is_alive()` and `.exit_code`.
* `.is_alive()` is a new function that does what it sounds like
* It now supports the `in` operator, `if b"/home" in SysCommand("pwd")` for instance.
* `for line in SysCommand()` as previously mentioned now only iterates the latest output since the last iteration.
* `.write()` has been added to communicate with a process
* `.tell()` will give you the current position in the trace log.
* `.seek()` will move to a position in the trace log
* `.peak()` is the internal function called on bytes to print it, something you wouldn't normally use, but you can override it.
* `.poll()` will perform emptying of buffers and perform work on the process, it also updates if the process is alive or not.
* `.execute()` will initate execution of the command and setup all the registers and stuff
* `.decode(encoding='utf-8')` will return the trace log decoded in the given encoding
* `SysCommand()` has been reworked - but should work the same way as before, but while utilizing `SysCommandWorker()`:
* `.trace_log` is now a property that fetches the `SysCommandWorker()`'s instance variable `._trace_log`
* If `SysCommand()` is put in a `with` context, it will return the inner `SysCommandWorker()` for raw processing, rather than returning itself.
* `SysCommand()` will not raise exceptions, instead it will log errors, instead `handle.exit_code` should be used. This was done to weed out bad behavior, since some commands use exit codes as status messages rather than actual errors.
* Using `SysCommand()` in a string, for instance like `f"This path is cool: {SysCommand('pwd')}"` will automatically decode into a `UTF-8` string of the output from the command.
* `SysCommand()` can also be JSON-dumped with the `JsonEncoder`, without the trace log.
* Also has the `.decode()` function
* `archinstall.pid_exists()` has been added to find out if a pid exists or not
* `archinstall.run_custom_user_commands()` have been added as a helper function to execute custom commands in the `--config` file.
* Most modules have now gotten a `__packages__` definition, for instance in `hardware.py` a `__packages__` has been created with the different drivers that it offers. This will allow us to do package inventory externally. Which will help with giving feedback of what we support. It can also be used in projects like [archoffline](https://github.com/Torxed/archoffline) to source packages during build time.
* hardware now also maintains a list of graphics drivers, called `archinstall.lib.hardware.AVAILABLE_GFX_DRIVERS`
* `archinstall.has_amd_cpu()` has been added
* `archinstall.has_intel_cpu()` has been added
* `archinstall.is_vm()` has been added
* A lot of docstring and typing has been done to help IDE users navigate the code.
* Listing keyboard mappings now uses `localectl list-keymaps`
* `archinstall.set_keyboard_language` now uses `localectl` instead of `loadkeys`
* *(Breaking Change)* `archinstall.Profile()` will now reset the namespace on every `with` context opening. This to ensure that it's started with the correct name-space, make sure to override `.original_namespace` if you want to take back control, otherwise whatever the namespace was on creation will be reverted on each instance.
* *(Breaking Change)* `archinstall.Profile().is_top_level_profile` has been re-worked to make sure not just a comment saying ` top_level_profile = True` exists, but it has to be an actual declared variable now.
* An error in Profile lookups have been fixed, where the search paths *(`PROFILE_PATH`)* was some what broken due to a previous re-work.
* `archinstall.generic_multi_select()` has been added which allows for selecting multiple options.
* `archinstall.MiniCurses` has been added, to emulate `ncurses` some what, without having to pull in all that it entails.
* `archinstall.ask_for_bootloader()` has been added.
* `archinstall.select_profile()` will now only show top level profiles, but entering any valid profile should also be supported.
* `archinstall.select_driver()` has been added, to select available graphics drivers
* `archinstall.select_kernel()` has been added, to select given kernels, uses `generic_multi_select` for this
* A lot of changes to the documentation, mainly surrounding `--config` has been re-worked or added.
* Introduced linting and automated ISO builds on commits and pull requests for easier contributions.

================================

**Known issues**:
* Installing **to** a USB device is still a bit iffy at times and some partition layouts might still confuse the installer and these will be fixed in v2.2.0 due to it requiring some re-work of the partition logic - which is to big of a change for this version.
* MBR and GRUB support will be coming in v2.2.0 (next release)

For more issues, see the [issues](https://github.com/archlinux/archinstall/issues) tab. And thank you for your patience and constructive feedback, it's very much appreciated!

Experimental features:
* *"Use `/mnt` as is"* 124 has been added as a new option in the formatting process. This function is tested, but internet is filled with people who use exotic partitioning layouts and such. So this feature is bound to break at some points. Feedback is very welcome especially around this new feature. `lsblk` output has been added to the logs, so submitting that log should be sufficient for us to replicate certain layouts and file formats.

This release contains the following:
* *(breaking change)* [Installer()](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-283d51110d004467e254215ad93e4dfcf0188beb22d9f878bb7088c12076dd2eL37-R37) has been re-worked quite a bit to accomodate 124. This meant that `partition` and `boot_partition` are no longer used, instead, a single installation directory is given which adds more degrees of freedom for the installer - as it no longer *"cares"* what partitions are being used. As long as it can run `pacstrap` in said directory, the installer will try to figure out what packages to source additionally to deal with the file system's in play. It also no longer sets a hostname by default, instead `.set_hostname()` should be called on the installation. *(see the minimal example for information on how the new workflow should look like)*.
* `argon2id` is being used by default instead of `argon2i` after some discussions here on GitHub, with a future plan of making this configurable through `--PBKDF=<implementation name>`.
* 124 has been implemented and allows for more complex partition layouts. See note above surrounding this feature.
* Added more verification to time zone selection
* Added a warning for one experimental feature
* The install log now contains `lsblk` output to better help diagnose potential partitioning issues.
* Added information regarding unattended installations in the readme
* IP addresses are now verified when entering them
* Time zone input will now loop instead of default to UTC when a **invalid** time zone is given *(leaving blank will however default to UTC still as mentioned below)*
* `generic_select` have been improved quite a bit. It now supports `allow_empty_input=False` which will loop over the choices until a valid is given as well as a `sort=True` which sorts the listing. Lastly `options_output=False` have been added to hide any output of the list, enabling the programmer to print their own list prior to selecting the options *(options still has to be given to `generic_select`, but these can be different from the output printed, enabling for more programatically understandable options returned from `generic_select` while allowing for presenting user friendly options)*
* "Keep partitions" now forces the user to select at least `/` and `/boot`
* A `.editorconfig` to help contributors use and follow some guidelines when contributing
* Added a mission statement to the project
* Simplified and consolidated the build process.
* [minimal.py](https://github.com/archlinux/archinstall/blob/master/examples/minimal.py) and the [script your own installation](https://github.com/archlinux/archinstall#scripting-your-own-installation) examples have been updated properly.
* The [test instructions](https://github.com/archlinux/archinstall#testing) and [build ISO instructions](https://github.com/archlinux/archinstall/wiki/Building-and-Testing) have been added and modified.
* The entry-point for the module have moved in to [__init__.py](https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/archinstall/__init__.py)
* Partial support for hardware raids [have been added](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-de72788d774e24383a05298142e3cb72b53329e0cef82eb19fd9716bfc667c58R82-R84) *(still very much an experimental feature)*
* `Partition()` now has a [.uuid](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-de72788d774e24383a05298142e3cb72b53329e0cef82eb19fd9716bfc667c58R206-R215) property for easy access to the PARTUUID of `lsblk`
* `BlockDevice()` now has a [.uuid](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-de72788d774e24383a05298142e3cb72b53329e0cef82eb19fd9716bfc667c58R129-R140) property for easy access to the UUID of `lsblk` for the device.
* `Partition()`[.real_device](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-de72788d774e24383a05298142e3cb72b53329e0cef82eb19fd9716bfc667c58R234-R237) now always returns the parent device, not only when it's encrypted. This means that `/dev/mapper/luksloop` will return `/dev/sda2` for instance, and `/dev/sda2` will return `/dev/sda`. For a more consistent behavior.
* Relaxed the `Partition()`[.safe_to_format()](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-de72788d774e24383a05298142e3cb72b53329e0cef82eb19fd9716bfc667c58R273-R281) a bit to allow for formatting "odd" boot partitions that doesn't make sense.
* `Partition()`[.mount()](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-de72788d774e24383a05298142e3cb72b53329e0cef82eb19fd9716bfc667c58R383) will now create the destination by default.
* `archinstall` has a new function called [.get_partitions_in_use(mountpoint)](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-de72788d774e24383a05298142e3cb72b53329e0cef82eb19fd9716bfc667c58R590-R607) which can look up a folder/mountpoint and analyze which block devices are mounted there (recursively). This helps the `Installer()` to understand which partitions is mounted and what packages to source.
* `archinstall.sys_command()` now take [environment_vars](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-4b1ae0a69adb92795bf24c05708559d253512bd3b6fe994b2f9f7a88309d2696R79) as a new argument, which can be used to pipe in specific environment variables. It also properly disposes of [any open filehandles](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-4b1ae0a69adb92795bf24c05708559d253512bd3b6fe994b2f9f7a88309d2696R266-R270) once done executing.
* Some general code clean up such as [this](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-d39c5771ecacdcd8a8216c41e300b2c0df140c7a871e99f2b29994a85b78cc86L6-R6)
* `Installer()` [no longer installs nano](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-283d51110d004467e254215ad93e4dfcf0188beb22d9f878bb7088c12076dd2eL37-R37) by default as a base package. That is now an optional package in profiles instead.
* `Installer()`[.partitions](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-283d51110d004467e254215ad93e4dfcf0188beb22d9f878bb7088c12076dd2eR51) have been added to facilitate easy access to the current partitions in play during this installation.
* `Installer().enable_service()` now takes [an arbitrary number of services](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-283d51110d004467e254215ad93e4dfcf0188beb22d9f878bb7088c12076dd2eR182-R186) instead of just one
* `Installer()`[.drop_to_shell()](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-283d51110d004467e254215ad93e4dfcf0188beb22d9f878bb7088c12076dd2eR194-R196) has been added to the API for easy access into the new installation.
* `Installer()`[.detect_encryption()](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-283d51110d004467e254215ad93e4dfcf0188beb22d9f878bb7088c12076dd2eR263-R269) has been added to the API to crudely but conveniently detect if there's encryption in play at the given pacstrap location.
* `Installer().minimal_installation()` now [patches mkinitcpio](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-283d51110d004467e254215ad93e4dfcf0188beb22d9f878bb7088c12076dd2eR276-R300) instead of blindly writing a string. This is prep-work for the future, where there will be API hooks such as `on_mkinitcpio` that will pipe these variables to external functions for patching.
* `archinstall.list_keyboard_languages()` should [no longer be limited to QWERTY](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-349e0306916a31b2d10e2e2c6be53ee68427130fe66150a64f26a70575ae0f7bR7) layouts. This should enable layouts like `be-latin1` to be discoverable which was mentioned in 149.
* Some [NVMe sleeps](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-77fb925cf17751bd67029d4b5d84614fc2e01b1a78be872405a2fb2426db4f66R130-R131) have been added to combat the issues during formatting/encryption/unlocking where commands release us before the kernel is done updating.
* `list_mirrors()` now support [offline mode](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-3a34aa93b1dbcbf57a9ecf6a0dc5737b8bee9a2bfaf6032e1c50efd799a0f6c1R79-R83) is detected. This is useful when building your own ISO's that is targeted at offline environments. An example usage of this is [archoffline](https://github.com/Torxed/archoffline) that utilize this as a test.
* [profiles/](https://github.com/archlinux/archinstall/tree/master/profiles) now has the ability to masquerade as [top_level_profile](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-c020f06bfd45245769ec3b87d62176bb0e3ad7e54751caa91099f28f2611aeb5R198-R203) enabling it to show up in profile listing. And subsequently hide themselves from the top level menu in [guided](https://github.com/archlinux/archinstall/blob/master/examples/guided.py). This helps a lot with maintaining a clean list of profiles at the first presentation. All desktop profiles have now been moved under the [desktop](https://github.com/archlinux/archinstall/blob/master/profiles/desktop.py) option.
* `Profiles()`[.packages](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-c020f06bfd45245769ec3b87d62176bb0e3ad7e54751caa91099f28f2611aeb5R205-R223) is a new property in the API to access potential package definitions. *(experimental feature and will not take full effect until `v2.2.0`)*
* [service_state()](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-8b88989818da3b8e89018b3bff85119ab8bf6f20a95c86f4594151e1966a9da9R10) has been patched and now properly returns a string representing the state, without any ANSI escape codes.
* Usernames are now [properly verified](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-991834f970d8b0af702afcd0581ed8edc8fd87c0aec535af2d375e04b0518a43R22-R30) before continuing in the guided installer.
* The formatting countdown has been moved into [archinstall.lib.user_interactions](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-991834f970d8b0af702afcd0581ed8edc8fd87c0aec535af2d375e04b0518a43R32-R65) *(and exposed as `archinstall.do_countdown()`)*
* Most function logs no longer outputs a [black background](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-991834f970d8b0af702afcd0581ed8edc8fd87c0aec535af2d375e04b0518a43R71).
* Timezones will now [default to UTC](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-991834f970d8b0af702afcd0581ed8edc8fd87c0aec535af2d375e04b0518a43R133-R135) when skipped.
* [Audio selection](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-991834f970d8b0af702afcd0581ed8edc8fd87c0aec535af2d375e04b0518a43R145-R151) will be added to all desktop profiles supporting both Pulse Audio and Pipewire.
* [NetworkManager](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-991834f970d8b0af702afcd0581ed8edc8fd87c0aec535af2d375e04b0518a43R161-R162) has been added as a network configuration step. *(Note that the [gnome (arch package group)](https://archlinux.org/groups/x86_64/gnome/) still contains `NetworkManager` as a dependency and will be installed, but not activated unless this new option is chosen)*
* Some general fixes for `out of range` issues [have been fixed](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-991834f970d8b0af702afcd0581ed8edc8fd87c0aec535af2d375e04b0518a43R234-R237) in various ways.
* Searching for a keyboard layout that doesn't exist, [no longer crashes but will ask you until a valid result is found](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-991834f970d8b0af702afcd0581ed8edc8fd87c0aec535af2d375e04b0518a43R343-R351).
* Until the next release when MBR/BIOS/GRUB support is fixed and added, guided will now [issue a warning about EFI not being used](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-98d75a109b5337cd7d7c948d2cfc2379bcc51be22dfa3ca6491765f0e0bcaaabR6-R8) to avoid confusion.
* [keyboard layout](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-98d75a109b5337cd7d7c948d2cfc2379bcc51be22dfa3ca6491765f0e0bcaaabR17-R22), [mirror selection](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-98d75a109b5337cd7d7c948d2cfc2379bcc51be22dfa3ca6491765f0e0bcaaabR31-R36) and [package selection](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-98d75a109b5337cd7d7c948d2cfc2379bcc51be22dfa3ca6491765f0e0bcaaabR191-R208) now longer causes hard crashes when faulty configuration is entered, instead it will loop back and ask for input again until valid input is given. These crashes were a design choice when releasing this project to avoid potential issues slipping through. This reduced complexity of the code and logic and ensured that no issues fell through the cracks, but it came at the cost of user experience so this is being changed one function at a time to allow for proper testing before being "made pretty".
* [audio selection](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-98d75a109b5337cd7d7c948d2cfc2379bcc51be22dfa3ca6491765f0e0bcaaabR323-R332) will only be performed if a desktop profile was chosen, or if the `--audio` flag was given to guided.
* Guided [will now ask if you want to drop into a shell](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-98d75a109b5337cd7d7c948d2cfc2379bcc51be22dfa3ca6491765f0e0bcaaabR352-R358) for post configuration upon a successful installation.
* Gnome and KDE have been polished a bit to enable a more minimal installation, which should make the footprint on the disk much smaller.
* A lot of profiles have been added: [alacritty](https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/profiles/applications/alacritty.py), [budgie](https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/profiles/applications/budgie.py), [cinnamon](https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/profiles/applications/cinnamon.py), [i3](https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/profiles/i3.py), (https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/profiles/applications/i3-wm.py), [lxqt](https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/profiles/applications/lxqt.py), [mate](https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/profiles/applications/mate.py), [sway](https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/profiles/applications/sway.py) and [xfce4](https://github.com/archlinux/archinstall/blob/c9e1d4a8c3435401220c1108ac938971ad517a37/profiles/applications/xfce4.py)
* [awesome](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-78ecad89c47a929ee91dfd09bea208fadee359b5dfb07954cd68ec6cd6c30ac7R9) no longer installs a browser and things by default.
* [desktop](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-77e411262e1038a5da0dd3815b55c2ebc5747dbe29c8929635efba3af5c33cd3R9) will install some helper packages to make life easier.
* [A new build method](https://github.com/archlinux/archinstall/compare/v2.1.3...v2.1.4-RC1#diff-fa602a8a75dc9dcc92261bac5f533c2a85e34fcceaff63b3a3a81d9acde2fc52R1-R39) to make building the python library a bit more future compliant
* Updated references to the old project URL to the new Arch Linux official GitHub account
* A lot of spelling errors have been corrected