Data-safe-haven

New features

- Add apt update server
- Add backup for blob storage
- Add backup for VM disks
- Add DNS server capabilities to DC2
- Enable automated VM updates
- Relicence to BSD 3-Clause
- Simplify deployment configuration
- Simplify NPS setup
- Simplify Powershell modules
- Switch to using DSC when configuring domain controllers
- Unify deployment of repository mirrors/proxies

Bug fixes

- Fix AAD domain verification
- Fix database logic so that either 0,1 or 2 databases can be deployed in an SRE
- Fix DNS recursion on domain controllers
- Fix htmlproofer issues by version pinning
- Fix network/firewall rules that were stopping the installation of gitlab-ce
- Fix NSG rules that were blocking LDAP connections from webapps
- Fix SHM teardown failure
- Fix Tier-3 allowlist scripts
- Fix updating of Guacamole dashboard when reading users from LDAP
- Improve tear down scripts
- Make RDS cipher suite setting more robust
- Make template deployments more robust
- Modify SHM requirements script to optionally install missing modules
- Restrict repository updates to this SRE
- Set Az.Storage minimum version
- Update NVIDIA repository key
- Update QGIS repository key
- Update SRD package versions
- Update to SSIS 16.0 in lockdown script

Security fixes

- Add ClamAV to all Linux VMs
- Drop support for Atom text editor
- Drop support for sbt
- Switch storage to GRS

Documentation updates

- Add administrator documentation for backups
- Add backup test to security checklist
- Add citation file
- Add disclaimer text to main repository README
- Add instructions to remove Conditional Access policies when reusing an AzureAD
- Add user backup instructions
- Fix various typographical errors in the documentation
- Make deployment instructions more visible
- Make documentation less prescriptive
- Update GitHub issue templates
- Update password writeback instructions
- Update SHM deployment instructions
- Update user guide

New features

- Whitelisted SSL Labs for analysing remote desktop entrypage.
- Updated SRD image with new packages and increased automation.
- Re-organised and standardised NSG rules
- Added tier 3 support for Nexus repositories

Bug fixes

- Fixed CoCalc NSG rules.
- Updated PyPI and CRAN allow lists.
- Switched to Mustache for all templating.
- Ensured that allow list generation does not time out.
- Replaced SHM networking ARM template.
- Switched from `AzureAD.Standard` preview to mainline version.
- Switched from `AzureAD.Standard` to `Microsoft.Graph`.
- Deprecated use of `Write-Host`.
- Ensured that `pyenv` virtual environment work correctly.
- Standarised NSG rule naming.
- Fixed overlapping IP ranges in example configs.
- Tidied up cloud-init files, moving scripts into dedicated files where appropriate.
- Switched Guacamole Docker deployment to use a non-root user.
- Simplified domain joining logic.
- Fixed check for tensorflow so that it is only applied if on the required package list.
- Fixed check for CoCalc deployment termination
- Set correct Graph permissions for changing user passwords

Documentation updates

- Fixed broken data classification flowchart.
- Added HTML checker to CI.
- Renamed DSVM to SRD throughout.
- Updated GitHub issue templates.
- Switched to GitHub discussions where relevant.
- Fixed GitHub Actions PR generation.
- Warned against using special characters in usernames.
- Added a Jupyter notebook for interactive testing, together with updates to the documentation.
- Fixed GitHub Actions cron jobs.

Bug fixes
- Allow Tier 0/1 SREs to access the internet as expected
- Correct NSG rule to allow connection to webapps from dashboard
- Ensure that CoCalc VM can connect to the package repositories

Documentation
- Fixed a broken link in the code of conduct

[View and clone the repository at this version](https://github.com/alan-turing-institute/data-safe-haven/tree/v3.3.1)

New features
- Added support for Guacamole remote desktop
- Added single-script SRE deployment (for Guacamole only)
- Added CoCalc webapp
- Added support for more Mustache features when expanding templates
- Added syslog collection for Linux hosts
- Added instructions for migrating users from one SHM to another

Bug fixes
- Allow VMs that were stopped due to lack of credit to be restarted
- Ensure that parameters are passed to remote scripts in a consistent way
- Work-around when using "allow" in the AzurePlatformDNS NSG rule
- Better method of identifying resource groups when tearing down SHM/SRE

Documentation
- Improved style and clarity of deployment documentation
- Improved documentation around image building
- First draft of DSPT documentation
- Better documentation for ingress/egress
- Changed some names to be more inclusive
- Updated security checklist
- Switched to GitFlow and added some explanatory text
- Added automated documentation building

[View and clone the repository at this version](https://github.com/alan-turing-institute/data-safe-haven/tree/v3.3.0)

New features
- Added diagnostic script for DSVM drive mounts
- Added new packages to DSVM
- Added Nexus option for tier-2 mirrors
- Added Powershell code style tests to CI
- Added scripts for deploying a standalone tier1 with CUDA support
- Added support for NFS blob storage for local data
- Added support for SMB blob storage for data ingress
- Dropped support for Python 2.7
- Ensured consistent NTP server across VMs
- Stopped serialising full config files to disk
- Switched to pyenv for installing python

Security
- Blocked DNS tunnelling for DSVMs
- Disabled legacy TLS on RDS Gateway
- Stopped using FQDN tags in firewall rules

Bug fixes
- Added missing <SHM ID> tags to resource group names
- Added missing logging resource group creation
- Allowed VM deployment after network lockdown
- Ensured firewall is started when updated and when SHM VMs are started
- Fixed SHM certificate generation
- Fixed SHM networking deployment
- Fixed SRE naming convention
- Pinned version of bandersnatch as newer versions are not working
- Refactored networking functions
- Refactored VM startup, shutdown and resize scripts
- Removed hard-coded rule on which IP addresses can connect to the SHM
- Removed multiple references to RDS
- Simplified AzureAD disconnect
- Simplified webapp deployment
- Updated Disconnect_AD to work with firewall

Documentation
- Added design decision documents
- Added documentation of database option
- Added initial draft of DSPT certification answers
- Added issue templates and improve GitHub labels
- Improved the Safe Haven deployment documentation
- Updated release and versioning table

[View and clone the repository at this version](https://github.com/alan-turing-institute/data-safe-haven/tree/v3.2.0)

New features
- Added Azure Firewall with rules to support Windows updates and Azure logging.
- Gather initial set of logs from VMs to centralised Azure Log Analytics workspace.

[View and clone the repository at this version](https://github.com/alan-turing-institute/data-safe-haven/tree/v3.1.0)