Unmanic

Service
- [NEW] Introduce worker groups to specify unique schedules and tags for groups of workers.
- [NEW] Introduce new "tagging" feature. Apply tags to libraries and worker to further customise that partnership. Eg. Tag a library with "priority" and keep a matching tagged worker spare for priority tasks.
- [NEW] Add the ability to link with an installation that is behind a proxy (traefik, swag/nginx, etc.) configured with basic authentication.
- [NEW] Add ability to set the maximum number of days that the completed tasks should be kept. If enabled, Unmanic will remove successfully completed tasks after the configured number of days.
- [IMPR] Session reloads with the Unmanic site API are now reduced and persisted between application restarts. Unmanic can operate in a supporter tier without an internet connection for up to seven days.
- [IMPR] All sub-processes executed by workers will now be executed at a lower priority than the main application. This should help improve application responsiveness during times of high CPU load.
- [IMPR] When configured to send tasks to remote installations, Unmanic will now send one transfer at a time, but be available to receive up to two transfers back from that remote installation. This should help to saturate network bandwidth for people with faster networks.
- [IMPR] Remote linking will now poll for updates from a remote installation every 10 seconds (previously 30). This will reduce the delays between a task finishing and the source installation seeing it as available to download.
- [IMPR] Reduce the number of file copies during a task lifcycle by first attempting an atomic movement for the final file movement of the post-processor from the cache directory. If the cache directory and destination directory are on different mounts, Unmanic will fall back to a file copy.
- [IMPR] Carry out post-processor file movements with a '.unmanic.part' suffix added to the end of the file as it is moved.
- [IMPR] Improvements to application logging.
- [IMPR] Improvements to the how application settings are saved to file.
- [FIX] Fix a minor issue with the way that the main service exits when a termination request is given.
- [FIX] Fix for bug in post-processor where, under certain circumstances, if a file movement failed it could cause the post-processor thread to crash.
- [FIX] Fix a bug where removing a library while a task is about to be processed by a worker could crash the main worker foreman thread.

Plugin executor
- [FIX] Fix error causing plugin settings to fail during write if no global settings file exists

Front-end
- [NEW] Display release notes on dashboard the first time the UI is opened after a release update.
- [NEW] Add form elements presenting worker group settings.
- [NEW] Add form elements for configuring library and worker tags.
- [NEW] Add form elements for configuring basic auth credentials in remote installation linking.
- [NEW] Add form elements for configuring completed tasks management.
- [NEW] Add ability to filter the completed tasks list by success status.
- [NEW] Add ability to filter the completed tasks list by the date that the task was completed.
- [IMPR] Display more information on the list of configured libraries in settings so we can see at a glance how each library is configured.
- [IMPR] Update all front-end dependencies for improvements, bug-fixes and security patches.

Service
- [NEW] Add ability to set a priority score modifier per library. Setting a higher score will cause tasks to be added further up the queue. Use a negative value to drop the task closer to the bottom of the queue.
- [NEW] Added support for executing subprocesses when installed natively on Windows.
- [IMPR] The plugin repo will be reloaded before a library config import attempts to installing plugins.
- [IMPR] Workers now have the ability to execute commands as raw strings if a plugin provides it with a string rather than a list.
- [IMPR] Added checks to ensure a library ID is provided as a number when fetching a plugin's settings.
- [IMPR] Switch the primary Unmanic site API domain to api.unmanic.app.
- [IMPR] Remove '.' (periods) from the generated cache paths.
- [IMPR] Improvements to repo updater functionality.
- [IMPR] Tweaks to SQLite DB migrations during upgrades to allow different types of joining tables.

Plugin executor
- [NEW] Add ability to share common data between file testing plugins. This should speed up library scans as plugins adopt this feature.
- [FIX] Fix issue where worker plugins that requested multiple passes were being given the same input file rather than the output file from the previous pass.
- [FIX] Fix issue where if plugins were configured for the first time from the library, a FileNotFoundError exception would be thrown as the global settings.json file was missing.
- [FIX] Fix issue where during second pass plugin execution, a settings object was being retained for subsequent executions rather than being read fresh every time.

API
- [NEW] Add ability to set a priority score when creating a task via the API in order to modify its place in the task queue.

Front-end
- [NEW] Add form elements for configuring a library's priority score modifier.

Changes:
- Add support for multiple libraries
- Add ability to create, read, update and delete multiple libraries
- Update library scanner and file monitor to handle multiple libraries
- Move the "enabled plugins" feature to be per-library
- Add ability to configure plugins per-library or globally and add ability to reset library config to global defaults
- Move plugin flow configuration to be per-library
- Include a library ID when requesting data from plugins (requires bump in plugin compatibility version and all plugins to be updated)
- Update all plugin executors to only execute plugins enabled on a library
- Delete all pending tasks matching a library that is being deleted
- Add ability to export and import library config
- Add ability to clone a library
- Add ability to lock a library to prevent it from being accidentally deleted
- Add ability to login to Unmanic with GitHub
- Unlocks all "supporter perks" if your GitHub account has committed anything to the Unmanic project in the past month
- Add ability to login to Unmanic with Discord
- Unlocks all "supporter perks" if your Discord account has received reputation points for being helpful in the past month
- Update developer tools for multi-library support
- Give post-processor plugins the ability to prevent Unmanic's default file movement completely
- Remove NVIDIA GPU probe from system info
- Improve Unmanic installation linking
- Add library name when sending files
- Do not send a task to remote installations that do not have a library name matching that of the task
- Improve pre-loading of tasks on remote installations to help with smaller files and greater worker counts
- Allow sending smaller files of less than 100MB in parallel
- Enable network transfers to multiple remote installations at a time
- Improvements to debug logging
- Migrate the directory info `.unmanic` files from INI to JSON for better filename support
- Add new plugin runner for managing web-hooks and API calls
- Add new API endpoints
- Create, read, update and delete a library
- Export a library config
- Import a library config
- Terminate all workers with one request
- Trigger a library scan
- Create a new local pending task from a given path

Docker
- Improve the Docker image inbuilt developer tools for multi-library support
- Bump packages:
- libgl1-mesa-dri `21.0.3-0ubuntu0.3~20.04.5` -> `21.2.6-0ubuntu0.1~20.04.2`
- mesa-va-drivers `21.0.3-0ubuntu0.3~20.04.5` -> `21.2.6-0ubuntu0.1~20.04.2`
- Add EXPOSE port 8888 in Dockerfile

Front-end
- Add support for multiple libraries
- Add required screen elements to manage multiple libraries
- Move file monitor enable to be per-library
- Hide the plugin settings for plugins that are not configurable
- Add library selector when adding completed task back to pending task queue
- Add ability to install plugins from the library config screen
- Improvements to layout and text
- Improve mobile browsing
- Rewrite the plugin settings screen to be a list rather than a table
- Ensure the file browser dialog is full-screen when viewed on a mobile
- Add dark mode
- Ensure the web-socket warning is dismissed when new data arrives
- Update all translations
- Add ability to terminate all workers from the dashboard
- Add button to trigger a library scan from the dashboard

Bug fixes:
- When specifying a custom config path via the `unmanic` command, the old config file was still being imported before this param was being applied.
- The web-socket connection warning would not disappear if the web-socket started receiving data again.
- When Unmanic detected an NVIDIA GPU, but the driver was not installed or the NVIDIA Docker runtime was not in use, it would throw an error.
- One of the logs "No libraries are configured to run a library scan" log was being run no matter what conditions were met

Changes:
- Always remove remote pending tasks on application startup
- Improvements to the library scanner
- Run file tests in a pool of threads to increase speed
- Disable plugin reloading during a library scan
- Pre-load all failed results from the database into memory rather than fetching them when testing each file
- Improve logging for the library scanner

Docker
- Ensure startup.sh line endings are always Unix line endings and not windows
- Bump packages:
- libgl1-mesa-dri '21.0.3-0ubuntu0.3~20.04.3' -> '21.0.3-0ubuntu0.3~20.04.5'
- mesa-va-drivers '21.0.3-0ubuntu0.3~20.04.3' -> '21.0.3-0ubuntu0.3~20.04.5'
- Added packages:
- sqlite3

Front-end
- Add configuration options for the number of threads to start for testing files during a library scan
- Display warning message on the data panels screen when there are no data panel plugins installed
- Update footer layout

Bug fixes:
- If a link is removed in settings while a task was in progress, the remote task manager threads would not be stopped
- Atomic file movements were not being picked up by the event monitor

Changes:
- Add options to link multiple installations together
- Installations once linked now have the options to:
- Send tasks to a remote installation
- Receive tasks from a remote installation
- Share a distributed pool of workers among themselves
- Increase the Plugin limit for non-supporters from 5 to 20

Front-end
- Add configuration options required for the remote installation linking feature

Bug fixes:
- The pending tasks list was not being sorted in order of priority
- Under some conditions, setting the worker count to 0 would incorrectly trigger the post-processor limit status message

Changes:
- Fix authentication polling for people with poor internet connections
- Refactor API to handle asynchronous requests
- Add ability to post a new file to the pending tasks queue via the API
- Remove files are not post-processed
- Remote files can be downloaded on completion from the API
- Remote files are processed entirely within the specified cache directory
- Add API endpoint for fetching task status
- Add a worker event scheduler
- Able to scaling worker count
- Able to pause workers
- Able to resume workers
- Make a config option keeping pending tasks between restarts


Docker:
- Fixes for Intel Media Drivers

Front-end
- Make the startup library scan independent of the scheduled library scans
- Support setting worker count to 0
- Add worker event schedules
- Add missing ETC tool-tip to language translations

Bug fixes:
- Unmanic library scans were being triggered when the library scanner was disabled
- Worker command log tail would not start until 20 lines had been produced