Padocc

- Two new modes added! (match and status_log)
- Added new display options! (allocations and bands now displayed)
- Bug fixes:
- merge_old_new - issue with indexing different types of lists. (1.2.a1)
- cleanup - now able to delete allocation directories. (1.2.a2)
- progress - now able to match multiple error types. (1.2.a3)

New Software Features:

- Per Project Code (PPC) Error tracking
- Individual log files for each dataset, updated automatically with filehandler updates built into the pipeline - for upcoming job allocation improvements.
- Scanning improvements and identification of types of dimensions.
- Support for virtual dimension additions (file_number)
- BypassSwitch option and default changes.

New Documentation:

- Documentation Updates
- Example CCI Water Vapour files and tutorial
- Kerchunk Powerpoints

- Major documentation overhaul
- Added BypassSwitch for better control of switch options
- Added features to Assessor:
1. Blacklist and proj_code list concatenation with existing files.
2. Slurm error type recognition and better labelling.

- errors.py - contains all custom error classes for Kerchunk pipeline
- logs.py - contains logger content and other utils
- setup.py - installs pipeline scripts into environment
- validate.py - thorough validation added (try subset, then try full fileset)

Bug Fixes:
- Dap link in cached files: Pipeline now writes cache files BEFORE concatenation to ensure no linkage.
- Dimension loading issue: Known issue with remote_protocol set - greater distinction between pre-concatenation and post-concatenation Kerchunk files.

Features added:
- Memory flag: Assign specific memory to parallel job arrays
- In-place validation: Pipeline allows rechecking of 'complete' datasets.
- Multi-dimension concatenation support
- Identical_dims identification within compute phase.
- Added custom error classes for edge cases.

First release (alpha) of CEDA Kerchunk Pipeline. Includes Init, Scan, Compute and Validate phases but no Catalog/Ingest control solution. Kerchunk files created by the pipeline are fully verified so all results from NetCDF solutions will be the same as the Kerchunk method.