Rdiff-backup

=== Changes

* CHG: (API 201 only) no more increments are created for files where
only metadata changed, this spares some disk space and inodes, thanks
to rknichols for the idea
* CHG: rdiff-backup is now compiled with Python 3.10.4 and Visual
Studio 2022, users shouldn't notice any difference beside the fact that
rdiff-backup doesn't support (or even work) on Windows 7 and older
* CHG: rdiff-backup with API lower than 201 might fail on unlocked
repository because it doesn't know about the rdiff-backup-data/lock.yml
file
* CHG: remove support for python v2 from rdiff-backup-delete
* CHG: return codes have changed and are now more detailed, see
man-page for details
* CHG: start to introduce more specific return codes (1 for error, 2
for warning, 4 for single file error, etc), which can be combined (e.g.
3 for error _and_ warning), this will take time to introduce everywhere
* CHG: the Python library psutil has become an optional dependency of
rdiff-backup to implement secure locking
* DEV: Add psutil feature to utils to gather process information, as
well known psutil wasn't fit for purpose and this avoids an unnecessary
dependency.
* DEV: get rid of unused RepoShadow.update_quoting function
* DEV: introduce properly reference path, index, inc and type
(partially renamed from restore_....)
* DEV: remove RepoShadow.set_rorp_cache from API, make internal to
simplify
* DEV: split RET_CODE_FILE in RET_CODE_FILE_ERR AND _WARN so that file
warnings can be used to detect comparaison differences
* DEV: use file system object for two file systems in comparaison
action, closes 643
* DOC: add a man-page for rdiff-backup-delete, closes 301
* DOC: add note about old versions of Windows not being supported due
to Python support matrix, closes 715
* DOC: add workflow of actions and locations to architecture
documentation
* DOC: Clarify / correct the FAQ instructions on how to set the
timezone for `rdiff-backup` on Windows, and slightly reword the entry,
closes 692
* DOC: Clarify global options vs. action-specific options in new CLI
syntax in examples, closes 679
* DOC: document how to backup between case aware file systems (VFAT or
NTFS), closes 11
* DOC: include more docs in the Windows distribution, see issue 679
* DOC: new standard for graphics using diagrams.net/draw.io
* FIX: add message about temporary directory to no space left on
device, this impacts only cases where tempdir isn't explicitly set,
closes 624
* FIX: allow --buffer option while testing by logging to terminal as
string, closes 546
* FIX: (API 201 only) do not compress metadata files if the
--no-compression option is given, BEWARE that such a repo can't be read
by rdiff-backup 2.0, closes 402
* FIX: (API 201 only) when removing the first hardlink and adding a new
one, all hardlinks remain linked together in repo, closes 272
* FIX: ignore failing creation of a device and applying of ACLs in all
circumstances, makes rdiff-backup more robust on CIFSv1, closes 678
* FIX: --min/max-file-size options were acting like inclusion functions
where they should be considered exclusion ones, closes 466
* FIX: rdiff-backup would complain about enforced quoting overriding
suggested quoting even though they were the same
* FIX: Regression ErrorLog has no attribute write was fixed by
reintroducing the write function (as internal _write method), thanks to
desseim for reporting, closes 686
* FIX: regress option in API 201 was actually still using the old API
function
* FIX: restoring from sub-path while selecting is now forbidden to
avoid data loss, workaround documented in FAQ, closes 463
* NEW: (API 201 only) regress action can be forced on a sane
repository, i.e. the last backup can be removed, closes 10
* NEW: (API 201 only) repositories are now locked to avoid concurrent
write actions, i.e. backup, regress and remove, it should work on NFS
as well, closes 415, closes 122
* NEW: define (imperfect) programmatic usage of rdiff-backup, see FAQ
for details, closes 703

=== Authors

* Eric L
* fireartist
* Guillaume Legrand
* Logan Stromberg
* maffe
* rknichols

=== Changes

* CHG: all messages have a proper prefix corresponding to their
severity (ERROR, WARNING, etc...)
* CHG: human readable list of increments with size has slightly changed
and is in the same order as the list _without_ size for consistency
* CHG: placeholder for version parts in remote schema are Vx, Vy and Vz
to align with -V for --version (and reserve small v for verbosity)
* CHG: rdiff-backup-delete: improve performance on gzip files by
directly using gzip command
* CHG: remove the possibility to change chars-to-quote between backup
calls, this feature never really worked properly anyway
* CHG: requoting a repository can be enforced using --chars-to-quote,
use with care
* DEV: align Windows Vagrant build pipeline with GitHub build pipeline
* DEV: Added patch for Python 3.11 as per bpo-39573 to replace Py_TYPE
with Py_SET_TYPE, closes 633
* DEV: align more actions and meta plugins interfaces, add generic
plugins module
* DEV: change default version string to make it parsable
* DEV: documented how to get information about missing code lines in
coverage
* DEV: for API 201, the fs_abilities module moved to locations as they
are only used from repositories and directories
* DEV: generic user/group functions have been moved to utils/usrgrp
* DEV: improved execution output during tests so that commands can be
more easily reused
* DEV: improve plugins documentation and add meta plugins docs
* DEV: mapping modules (filenames quoting, hardlinks, longnames and
owners users/groups) have been moved to locations/map package for
better encapsulation
* DEV: new test function commontest.rdiff_backup_action using the new
CLI interface
* DEV: remote functions in Main and backup are deprecated from the API
and replaced by class methods in _repo_shadow and _dir_shadow
* DEV: RORPath and RPath classes are now PathAlike and can be safely
output as string, closes 84
* DEV: server process can be remotely debugged using rpdb and netcat
* DEV: the restrict mode and path options are specific to the server
action, and not generic
* DEV: Vagrant / Ansible build pipeline relies on ansible-core 2.12 and
collections
* DEV: Windows is now part of the test pipeline even if tests are
limited, closes 347
* DOC: add FAQ regarding support of too slow file systems like exFAT
and ZFS on SMB, closes 595
* DOC: add FAQ to document GUI and WebUI for rdiff-backup, closes 594
* DOC: Convert documentation from Markdown to AsciiDoc, closes 537
* DOC: docs/DEVELOP.md link in readme was 404.
* DOC: updated installation and migration instructions for CentOS, RHEL
& Co using COPR or EPEL
* DOC: updated Windows developer guide for python 3.9 and x64, linking
error explained, thanks to rstarkov
* FIX: backslashes were removed too eagerly in locations, making the
use of Windows paths impossible, closes 585
* FIX: calling with remote-schema containing the new server action
would fail with NoneType not being iterable, closes 565
* FIX: catch properly long name errors under Windows, closes 558
* FIX: describe implications of undetected case sensitivity and
trailing space/period and reduce severity, closes 583
* FIX: do not return an error if no increment is old enough to be
removed, closes 616
* FIX: exclude symlinks in first place under Windows to avoid symlink
not being of correct type, closes 608
* FIX: handling of RDIFF_BACKUP_VERBOSITY was broken after recent
changes
* FIX: ignore bad file descriptor (errno 9/EBADF) error impacting a
single file, closes 611
* FIX: improve handling of connection errors with clearer message,
closes 564
* FIX: issues with trailing spaces/periods on NTFS file system under
Linux are now properly detected and such characters quoted, closes 579
* FIX: rdiff-backup-delete: delete metadata from
win_access_control_lists
* FIX: rdiff-backup-delete: permissions error when trying to delete
file or folder
* NEW: output runtime information with parsed arguments in debug mode,
to help support
* NEW: parsable list of increments is in YAML format for easier parsing
* NEW: parsable output in YAML for the compare action

=== Authors

* Eric L
* Frank Crawford
* Patrik Dufresne
* Roman Starkov
* ToM
* user-na

=== Changes

* FIX: 64 bits version compiled with PyInstaller for Windows couldn't
find its module rdiff_backup, closes 555

=== Authors

* Eric L

=== Changes

* CHG: Add no-compression defaults for videos .webm and tar
zStandart-compressed files .tzst
* CHG: depend on importlib-metadata instead of setuptools to get
rdiff-backup veersion, closes 418
* CHG: man page rdiff-backup-old(1) describes the old CLI,
rdiff-backup(1) the new one
* CHG: option --test-server will test all servers even if one fails,
returning 1 in such case, 2 if the parameters were incorrect, output
has also slightly changed.
* CHG: option --version outputs extended version information when used
in API versions above 200
* CHG: Pickle protocol raised from 1 to 4, it shouldn't impact older
versions of rdiff-backup as protocol 4 is known since Python 3.4 and
the protocol version is recognized automatically on the receiving end
* CHG: rdiff-backup now supports the newly released Python 3.9 and
stops supporting the obsolete Python 3.5.
* CHG: restoring a specific increment requires now the use of
'--restore' parameter
* CHG: the host placeholder in the remote schema is now '\{h}', '%s' is
deprecated.
* CHG: the old command line interface without explicit actions is
considered deprecated
* CHG: the Windows build uses Python 3.9 instead of Python 3.7 (nobody
should notice)
* DEV: action plugins are described and implemented as context manager
* DEV: add coding conventions under docs/CODING.md to be followed by
developers and reviewers. This is a living document which will be
expanded over time.
* DEV: add docs/api folder with API description to be followed and API
v200.
* DEV: added coding rules for sorting of items like functions,
variables, methods, classes, etc.
* DEV: add Globals.PICKLE_PROTOCOL constant and raise it's version from
1 to 4
* DEV: add new package rdiffbackup.locations for directory and
repository modules.
* DEV: add requirements.txt to help GitHub detect our dependencies and
warn about security flaws, closes 434
* DEV: all API interfaces are marked directly in the code with API to
simply recognition while coding.
* DEV: document docstrings and import recommendations.
* DEV: documented that compatibility functions are to have a postfix
`_compat<API>`.
* DEV: Explain or remove many asserts throughout the code, closes 398
* DEV: fix issue in ACL tests when user isn't named like group
* DEV: increase crossversion check to old version 2.0.5
* DEV: introduction of an 'actions' plug-in interface described in the
architecture documentation.
* DEV: Make flake8 check python scripts and simplify
rdiff-backup-statistics
* DEV: make it easier to use and test rdiff-backup directly from the
Git repo under Windows using Vagrant
* DEV: man page can be generated from markdown
* DEV: migrate from Travis-CI (thanks for all the fish) to GitHub
actions for our CI/CD pipeline
* DEV: pin specific version of pyenv-win in Travis CI so that changes
don't make the pipeline without control
* DEV: prefix all internal functions, variables and classes with
underscore to get more clarify in the code
* DEV: reduce max complexity to 20 by simplfiying more functions,
mostly using mapping dictionaries
* DEV: reduce max complexity to 30 and rename CompareRecursive to
compare_recursive.
* DEV: Re-write tox.ini to make sure that also sub-processes are part
of the coverage calculation, raises test coverage above 80%
* DEV: TempFile.new(_in_dir) is replaced by RPath.get_temp_rpath
* DEV: there is a new namespace 'rdiffbackup' for new/clean code
according to strategy.
* DOC: add architecture documentation for better understanding of the
overall code structure
* DOC: add hint on how to use batch file under Windows
* DOC: add how to use Microsoft's OpenSSH from 32-bits rdiff-backup,
closes 494, closes 496
* DOC: clarify in the man page(s) that only slashes are allowed in
selection patterns under Windows, closes 531
* DOC: clarify selection principles in man-page that pattern matching
doesn't resolve relative vs. absolute paths and that it is done on the
complete path, closes 533
* DOC: clarify that the host part belongs together with the double
colons, closes 480
* DOC: comparaison of old and new Command Line Interface added to the
migration documentation
* DOC: comparaison of old and new Command Line Interface added to the
migration documentation
* DOC: docs/migration.md describes how to install rdiff-backup side by
side and use old versions 'forever', closes 523
* DOC: document how to use Putty as SSH client thanks to xastor in 496
* DOC: document more clearly that rdiff-backup 1.x and 2.x are
incompatible, closes 513
* DOC: explain the prefixes used in the changelog with focus on
potentially incompatible __CH__an__G__es, closes 436
* DOC: make the installation instructions for other Linux and UN*X-OID
e.g. BSD systems using PyPI more complete, considering build
dependencies. Closes 487
* DOC: man page has been clarified regarding --no-hard-links option
* FIX: avoid breaking on non-readable files, causing ListError, closes
34, closes 245
* FIX: avoids MemoryError on Windows when compiling for 64 bits, closes
453
* FIX: cross device link error on ZFS with project quota, closes 519
(522)
* FIX: get rid of spurious resource warnings due to subprocess still
running, closes 165
* FIX: longnames are correctly reversed when regressing a failed
back-up, closes 9
* FIX: PID handling when process is interrupted now works properly
under Windows.
* FIX: setting tempdir under Windows might fail with error about mix of
bytes and str, closes 540
* FIX: support long paths under Windows 10 v1607 or later, once enabled
in registry/GPO (see Windows README for details), closes 236
* FIX: When using the --remove-older-than option with --tempdir, the
--tempdir
* NEW: both 32 and 64 bits version of rdiff-backup are now built for
Windows
* NEW: new action 'info' to output system information, try
'rdiff-backup info'
* NEW: option --api-version to explicitly set the actual API version,
maximum version is 201, default is 200, compatible with 2.0.x
* NEW: rdiff-backup has a `--help` parameter, closes 280
* NEW: rdiff-backup has a new interface with actions and sub-options,
use `--new --help` to get the help
* NEW: rdiff-backup has the concept of API version between client and
server
* NEW: rdiff-backup-statistics has --help and --version options
* NEW: the current rdiff-backup version can be used in the remote
schema with 'x.y.z' being split as placeholders '\{vx}', '\{vy}', '\{vz}'
so that one can install (via pip) and use a specific major/minor
version of rdiff-backup (see the migration docs for details).

=== Authors

* Andrea Veri
* dgasaway
* Eric L
* Felix Yan
* Patrik Dufresne
* t9t

=== Changes

* CHG: development status now set to stable in PyPI classifiers
* CHG: increased version of bundled Python Windows version from 3.7.5
to 3.7.7. (426)
* DEV: add measurement of test coverage to tox.ini and limit to 70% for
further improvement, closes 113
* DEV: make CI pipeline faster by joining small jobs together to avoid
VM creation overhead.
* DOC: add few development notes about profiling rdiff-backup for time
and memory consumption

=== Authors

* Eric L

=== Changes

* CHG: explicitly refuse to back-up to exFAT because it doesn't handle
properly case insensitive deletion of files, closes 38
* CHG: setuptools is a runtime dependency for installation and tests so
that version appears correctly instead of DEV, closes 305
* CHG: testing explicitly for existence of tempdir might make certain
setups fail now because tempdir was silently ignored
* DEV: Add a misc script to setup an ArchLinux as development platform
* DEV: add a new Vagrant configuration to do some smoke tests between
the current/development version and any older one
* DEV: Add samba server with pre-defined shares to Windows vagrant
setup to allow for more extensive tests on shares
* DEV: fix compatibility in rollsum and sum-size with rdiff 2.2/2.3
leading to errors in librsynctest, closes 304
* DEV: function rpath.getdevnums now also returns the device type,
block or char
* DEV: replace deprecated xattr.<verb>xattr with xattr.<verb> function,
closes 177
* DOC: added clearer instructions for installing weak dependencies to
support ACLs and EAs under CentOS and RHEL
* DOC: fix semi-broken nongnu.org links in manpages of rdiff-backup and
rdiff-backup-statistics
* FIX: add python3-setuptools as a run time dependency to Debian
package so --version works and doesn't output DEV, closes 305.
* FIX: address `PY_SSIZE_T` deprecation warning appearing under Python
3.8 in the C code, closes 374
* FIX: avoid error module 'errno' has no attribute 'EDEADLOCK' under
MacOSX, closes 366
* FIX: avoid issue with backslash at the end of file path under
Windows, closes 395
* FIX: avoid TypeError: a bytes-like object is required, not 'str' when
logging error message by fixing encoding, closes 380
* FIX: explicitly test existence of tempdir and avoid "Can't mix
strings and bytes in path components" error, closes 367
* FIX: failed on certain device files with no such file or directory
error, closes 401
* FIX: Force encoding of log file to be UTF-8 on all platforms and be
lenient to avoid codec errors on logging, closes 356
* FIX: Improve handling of files in use under Windows, closes 392
* FIX: more meaningful error message when trying to test-server a local
path, closes 396

=== Authors

* Andreas Olsson
* Eric L
* Jirka Vejrazka
* Neha S
* Otto Kekäläinen
* Patrik Dufresne