Aws-parallelcluster

------

**CHANGES**

------

**CHANGES**
- Pyxis is now disabled by default, so it must be manually enabled as documented in the product documentation.
- Upgrade Python runtime to version 3.12 in ParallelCluster Lambda Layer.
- Remove version pinning for setuptools to version prior to 70.0.0.
- Upgrade libjwt to version 1.17.0.

**BUG FIXES**
- Fix an issue in the way we configure the Pyxis Slurm plugin in ParallelCluster that can lead to job submission failures.
https://github.com/aws/aws-parallelcluster/issues/6459
- Add missing permissions required by login nodes to the public template of policies.

------

**ENHANCEMENTS**
- Add support for custom actions on login nodes.
- Allow DCV connection to login nodes.
- Add support for ap-southeast-3 region.
- Add security groups to login node network load balancer.
- Add `AllowedIps` configuration for login nodes.
- Add new configuration `SharedStorage/EfsSettings/AccessPointId` to specify an optional EFS access point for a mount
- Allow up to 10 login node pools.
- Install enroot and pyxis in official pcluster AMIs

**CHANGES**
- Upgrade Slurm to 23.11.10 (from 23.11.7).
- Upgrade Pmix to 5.0.3 (from 5.0.2).
- Upgrade EFA installer to `1.34.0`.
- Efa-driver: `efa-2.10.0-1`
- Efa-config: `efa-config-1.17-1`
- Efa-profile: `efa-profile-1.7-1`
- Libfabric-aws: `libfabric-aws-1.22.0-1`
- Rdma-core: `rdma-core-52.0-1`
- Open MPI: `openmpi40-aws-4.1.6-3` and `openmpi50-aws-5.0.3-11`
- Upgrade NVIDIA driver to version 550.90.07 (from 535.183.01).
- Upgrade CUDA Toolkit to version 12.4.1 (from 12.2.2).
- Upgrade Python to 3.9.20 (from 3.9.19).
- Upgrade Intel MPI Library to 2021.13.1.769 (from 2021.12.1.8).

**BUG FIXES**
- Fix validator `EfaPlacementGroupValidator` so that it does not suggest to configure a Placement Group when Capacity Blocks are used.
- Fix sporadic cluster creation failures with managed FSx for Lustre.
- Fix cluster deletion failure when placement group is enabled.
- Fix issue with login nodes being marked unhealthy when restricting SSH access.
- Fix `retrieve_supported_regions` so that it can get the correct S3 url.
- Fix `describe_images` so that it uses pagination.
- Fix `No route tables found` bug when specifying default VPC subnet to LoginNodes/Networking/SubnetIds.

------

**BUG FIXES**
- Fix image build failure in China regions.

------

**ENHANCEMENTS**
- Add new configuration section `Scheduling/SlurmSettings/ExternalSlurmdbd` to connect the cluster to an external Slurmdbd.
- Allow build-image to be run in an isolated network.
- Add support for Amazon Linux 2023.
- Add support for `price-capacity-optimized` as an `AllocationStrategy`.
- Add validator to prevent the use of Placement Groups with Capacity Blocks.

**CHANGES**
- CentOS 7 is no longer supported.
- Upgrade Cinc Client to version to 18.4.12 from 18.2.7.
- Upgrade munge to version 0.5.16 (from 0.5.15).
- Upgrade Pmix to 5.0.2 (from 4.2.9).
- Upgrade third-party cookbook dependencies:
- apt-7.5.22 (from apt-7.5.14)
- openssh-2.11.12 (from openssh-2.11.3)
- Remove third-party cookbook: selinux-6.1.12.
- Upgrade EFA installer to `1.32.0`.
- Efa-driver: `efa-2.8.0-1`
- Efa-config: `efa-config-1.16-1`
- Efa-profile: `efa-profile-1.7-1`
- Libfabric-aws: `libfabric-aws-1.21.0-1`
- Rdma-core: `rdma-core-50.0-1`
- Open MPI: `openmpi40-aws-4.1.6-3` and `openmpi50-aws-5.0.2-12`
- Upgrade NVIDIA driver to version 535.183.01 (from 535.154.05).
- Upgrade Python to 3.9.19 (from 3.9.17).
- Upgrade Intel MPI Library to 2021.12.1.8 (from 2021.9.0.43482).

**BUG FIXES**
- Fix Data Repository Associations configuration to make `AutoExportPolicy` and `AutoImportPolicy` optional.
- Fixed an issue during cluster deletion that now completes compute fleet cleanup when instances are either in shutting-down or terminated state.
This is to avoid cluster deletion failures for instance types with longer termination cycles.
- Allow cloudwatch dashboard to be enabled and alarms to be disabled in the `Monitoring` section of the cluster config.
- Allow ParallelCluster Custom Resource to suppress validators using `PclusterCluster/SuppressValidators`.
- Removing `/etc/profile.d/pcluster.sh` so that it's not executed at every user login and
`cfn_bootstrap_virtualenv` is not added in PATH environment variable.
- Fix ParallelCluster API spec by replacing field `failureReason` with `failures` in `DescribeCluster` response.
- Fix ParallelCluster API spec by adding the CloudFormation stack status that were missing:
`IMPORT_*`, `REVIEW_IN_PROGRESS` and `UPDATE_FAILED`.
- Fix an issue that prevented cluster updates from including EFS filesystems with encryption in transit.
- Fix an issue that prevented slurmctld and slurmdbd services from restarting on head node reboot when
EFS is used for shared internal data.
- On Ubuntu systems, remove default logrotate configuration for cloud-init log files that clashed with the
configuration coming from Parallelcluster.
- Fix image build failure with RHEL 8.10 or newer.

------

**ENHANCEMENTS**
- Add support for FSx Lustre as a shared storage type in us-iso-east-1.

**BUG FIXES**
- Remove `cloud_dns` from the `SlurmctldParameters` in the Slurm config to avoid Slurm fanout issues.
This is also not required since we set the IP addresses on instance launch.