Npbackup

- This is a major rewrite that allows using multiple repositories, adds repository groups and implements repository settings inheritance from group settings


Features
- New viewer mode allowing to browse/restore restic repositories without any NPBackup configuation
- Allows setting repository via environment variables, config file or directly in GUI
- Multi repository support
- Group settings for repositories
- Operation center
- GUI operation center allowing to mass execute actions on repos / groups
- CLI operation center via `--group-operation --repo-group=somegroup`
- Implemented retention policies
- Optional time server update to make sure we don't drift before doing retention operations
- Operation planifier allows to create scheduled tasks for operations
- Implemented repo quick check / full check / repair index / repair packs /repair snapshots / recover / unlock / forget / prune / dump / stats commands
- Implemented new housekeeping command which is a shorthand for check, forget and prune commands
- Added per repo permission management
- Repos now have backup, restore and full privileges, optionally allowing to restrict access for end users
- Added snapshot tag to snapshot list on main window
- Split npbackup into separate CLI and GUI programs (GUI can also act as CLI)
- Status window has been refactored so GUI now has full stdout / stderr returns from runner and backend
- Implemented file size based exclusion
- CLI can now fully operate in API compatible mode via --json parameter
- Parses non json compliant restic output
- Always returns a result boolean and a reason when failing
- CLI now accepts --stdin parameter to backup streams sent to CLI
- Added minimum backup size upon which we declare that backup has failed
- All bytes units now have automatic conversion of units (K/M/G/T/P bits/bytes or IEC bytes)
- Refactored GUI and overall UX of configuration
- New option --show-config to show inheritance in CLI mode (GUI has visual indicators)
- Allow using external restic binary via --external-backend-binary parameter in CLI mode
- Implemented stdin_from_command backup process which can now get exit codes of programs which output is backed up (ex: mysqldump)
- GUI: Restore window
- Restore window now also shows symlinks and irregular files
- Restore window now uses 40% less memory
- Restore window showing up 60% faster
- Symlinks and irregular files are now shown properly
- Prometheus metrics:
- Metrics are now created for all types of operation
- Metrics have npbackup execution state
- Metrics now include auto upgrade state
- Dry mode now works for all operations where restic supports dry-mode
- Implemented scheduled task creator for Windows & Unix
- Added --no-cache option to disable cache for restic operations (neeeded on RO systems)
- Added CRC32 logging for config files in order to know when a file was modified
- Missing exclude files will now search in current binary directory for a excludes directory
- Splitted releases between legacy and non legacy
- Updated legacy tcl8.6.13 to tc8.6.15
- Updated legacy Python 3.7 to Python 3.11 (with openssl 3.1.3)



Fixes
- Default exit code is now worst log level called unless specific errors are triggered
- Fix deletion failed message for en lang
- Fix Google cloud storage backend detection in repository uri
- Backup admin password is now stored in a more secure way
- GUI: Restore window now closes correctly when using the window control button
- GUI: Show anonymized repo uri in GUI
- GUI: Exec error should always stay on top

Misc
- Current backup state now shows more precise backup state, including last backup date when relevant
- Concurrency checks (pidfile checks) are now directly part of the runner
- Allow a 30 seconds grace period for child processes to close before asking them nicely, and than not nicely to quit
- Fully refactored prometheus metrics parser to be able to read restic standard or json outputs
- Reimplmented auto upgrade after CLI/GUI split
- Added initial tests
- Exclude lists have been updated
- Removed Windows installer from the project. We need to come up with a better solution
- Added manual --init command (already executed by first backup)
- Updated internal restic binary to restic 0.17.1 compiled with go1.23.1
- Added heartbeat log for long commands

- Fixed backup paths of '/' root partitions
- Properly display repository init errors
- Logs didn't show proper error status
- Fixed wrong init detection for S3 backend

- Added snapshot deletion option in GUI
- Fix ignore_cloud_files parameter did not work on some non systems (still an issue, see https://github.com/restic/restic/issues/4155)
- Configured default backend connections to 2 for local, 8 for remote when default value '0' is given
- Fine adjust GOMAXPROCS environment variable (n for 1 core, n-1 for 2-4 cores, n-2 for 5+ cores)
- Updated internal restic binary to 0.16.0 compiled with go1.20.6
- Compiled with newer Nuitka 1.7.10 that allows use of Python 3.11
- Updated python to version 3.11 for Windows x64 targets
- Basic MacOS X compile support

- Added a full auto-upgrade solution
- Aupgrade client integrated into NPBackup, that can be called manually via --auto-upgrade or automatically run every n backups
- Upgrade server which servers files and their metadata
- Added a gui to create a scheduled task under Windows
- Added a gui to configure backup admin password (was compile time setting before)
- Improved setup.py to provide launch scripts for both Linux and Windows platforms
- Made windows cloud file filter optional (enabled by default)
- Added default configuration settings
- Rewrite configuration gui into a tabbed gui
- Updated restic binary to restic 0.15.1 compiled with go1.19.5
- Changed backup interval unit from seconds to minutes

- Added execution time information
- Code cleanup
- Refactor runner into a class
- Refactor npbackup into package
- Adding missing typing
- Code is now formatted with black
- Code must now pass pylint and flake8
- Workaround for painfully slow sg.TreeData window.close()
- Simplify npbackup shortcut
- Shift default interval from 23H30 to 24H
- Fixed missing translations for admin password popups

- First open public release
- Refactor GUI so actions become non-blocking
- Added loader animation for gui events
- Added backup window with current state
- Added internationalization support (currently en and fr)
- Refactor restic.ls function to return a generator since restic ls command gives chunks of json, which is bad memory wise when parsing each chunk into a big list
- On Windows, "The cloud operation is not supported on a read-only volume." and "The media is write protected." errors will not make backup fail, see 4155
- This fix can be removed once 4155 is resolved, see TEMP-FIX-4155 comment in restic_wrapper package
- Added Additional backup parameter option
- Added optional backup tags
- Added optional prometheus group label and additional labels entries
- Make sure backend connections number parameter works with all backends
- Added priority configuration for restic operations, defaults to low priority to not disturb operations
- Added WindowsApps folder (reparse points) to exclusion list (fix "The file cannot be accessed by the system." error)
- Fixed backup did use verbose option twice
- Updated restic binary to restic 0.15.0 compiled with go1.19.5 on windows/amd64
- Made theme configurable
- Make sure we exit all subprocesses when exiting npbackup
- Improve config GUI for supporting elder config files properly
- Fix restic metrics when 0 bytes of data have been added to the repo
- Fix restoration must ignore case on Windows
- Fix backup_job variable substitution in config file
- Fix for ruamel.yaml global objects being modified out of current scope encrypting current config while saving
- Fix npbackup could not start when no config file was present
- Drastically improve ls operation speed by changing command_runner method from poller to monitor (no live output) except for backup operation
- Compiler
- Compile python 3.7 and 3.10 targets so we also get to use Windows 7 which can't run python > 3.7
- Installer
- Moved task creation process to npbackup so we can recreate tasks on systems where npbackup is already installed